How to Avoid AWS Lambda Timeout When Processing HubSpot Records

🧠 The Problem

When using AWS Lambda to process HubSpot objects records (specially for company data), there is a limit. One Lambda function can only run for up to 15 minutes. If it takes longer, it will stop automatically.

This happened in our setup. The Lambda logs showed this error:

Task timed out after 900.10 seconds

We were using batch processing, but it failed because the data was too big.

✅ The Solution

To solve the issue, we used AWS Step Functions. It helps by calling the Lambda function repeatedly, allowing us to fetch HubSpot records in smaller chunks (batches). This way, the Lambda won’t hit the timeout limit.

The Step Function works like this:

Start with default input (offset = 0)
Call the Lambda function to get records
Check if there is more data
If more, call Lambda again
If done, finish the process

⚙️ Step Function Code (State Machine)

{
  "Comment": "State Machine to process HubSpot records in batches",
  "StartAt": "Set Default Input",
  "States": {
    "Set Default Input": {
      "Type": "Pass",
      "Result": {
        "lambdaResult": {
          "offset": 0
        }
      },
      "ResultPath": "$",
      "Next": "Get HubSpot Data"
    },
    "Get HubSpot Data": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:<AWSAccountId>:function:dev-yourproject-lambda:$LATEST",
      "Parameters": {
        "offset.$": "$.lambdaResult.offset"
      },
      "ResultPath": "$.lambdaResult",
      "Next": "Check If More Data"
    },
    "Check If More Data": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.lambdaResult.status",
          "StringEquals": "complete",
          "Next": "End"
        },
        {
          "Variable": "$.lambdaResult.status",
          "StringEquals": "incomplete",
          "Next": "Get HubSpot Data"
        }
      ]
    },
    "End": {
      "Type": "Succeed"
    }
  }
}

Example Output:

How to Avoid AWS Lambda Timeout When Processing HubSpot Records

🧑‍💻 Lambda Function Code (Python)

def lambda_handler(event, context):
    start_time = time.time()
    offset = event.get("offset", 0)

    while True:
        records, next_offset = function_to_get_company_records(offset)

        if not records:
            break

        for record in records:
            function_to_process_company_records(record)

        if not next_offset:
            break

        if time.time() - start_time > TIME_LIMIT:
            return {
                "status": "incomplete",
                "offset": next_offset
            }

    return {
        "status": "complete"
    }

Additional Note: Handling Mid-Page TIME_LIMIT Triggers

While the script above gives you a solid starting point, there’s one scenario you can optimize for:

Problem

If TIME_LIMIT triggers mid-page, your current logic returns next_offset (the cursor for the next page). This causes any unprocessed records remaining on the current page to be skipped.

Two Approaches to Solve This

A) Simple (Reprocess Entire Page)

Return current_offset (instead of next_offset).
This means you’ll start processing the same page again on the next run.
Requirement: Your processing logic must be idempotent, so duplicate processing is safe.
Advantage: Easiest to implement with minimal complexity.

B) Precise Resume (Track Index Within Page)

Return not only the current_offset, but also a resume_index (indicating the next record position in the page).
On the next invocation, re-fetch the same page, skip already processed records (via resume_index), then continue.
Only after processing the full page do you move to next_offset.
Advantage: No duplicate processing, no skipped records—much more precise.

Comparison Table

Approach	Offset Returned	Duplicates Risk	Complexity	Best For
Reprocess Page	`current_offset`	Possible	Low	Simple workflows, safe idempotent logic
Precise Resume	`current_offset + resume_index`	None	Medium–High	Critical workflows, no duplicate tolerance

Choose Approach A if you need a quick fix and can guarantee idempotency. Opt for Approach B if completeness and precision matter more, especially in production environments.

✅ Result

After using AWS Step Functions, the Lambda function now works without timing out. It can handle large HubSpot data by splitting it into smaller parts and processing them one batch at a time.

This setup improves reliability and avoids hitting AWS Lambda limits. It is now scalable and ready for bigger datasets.

📌 Key Takeaways

Lambda has a 15-minute max timeout (900 seconds)
HubSpot data can be large, so use pagination
Step Functions help repeat the Lambda task until finished
This design pattern is serverless, scalable, and cost-effective