How to Avoid AWS Lambda Timeout When Processing HubSpot Records

🧠 The Problem

When using AWS Lambda to process HubSpot objects records (specially for company data), there is a limit. One Lambda function can only run for up to 15 minutes. If it takes longer, it will stop automatically.

This happened in our setup. The Lambda logs showed this error:

Task timed out after 900.10 seconds

We were using batch processing, but it failed because the data was too big.

✅ The Solution

To solve the issue, we used AWS Step Functions. It helps by calling the Lambda function repeatedly, allowing us to fetch HubSpot records in smaller chunks (batches). This way, the Lambda won’t hit the timeout limit.

The Step Function works like this:

Start with default input (offset = 0)
Call the Lambda function to get records
Check if there is more data
If more, call Lambda again
If done, finish the process

⚙️ Step Function Code (State Machine)

{
  "Comment": "State Machine to process HubSpot records in batches",
  "StartAt": "Set Default Input",
  "States": {
    "Set Default Input": {
      "Type": "Pass",
      "Result": {
        "lambdaResult": {
          "offset": 0
        }
      },
      "ResultPath": "$",
      "Next": "Get HubSpot Data"
    },
    "Get HubSpot Data": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:<AWSAccountId>:function:dev-yourproject-lambda:$LATEST",
      "Parameters": {
        "offset.$": "$.lambdaResult.offset"
      },
      "ResultPath": "$.lambdaResult",
      "Next": "Check If More Data"
    },
    "Check If More Data": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.lambdaResult.status",
          "StringEquals": "complete",
          "Next": "End"
        },
        {
          "Variable": "$.lambdaResult.status",
          "StringEquals": "incomplete",
          "Next": "Get HubSpot Data"
        }
      ]
    },
    "End": {
      "Type": "Succeed"
    }
  }
}

🧑‍💻 Lambda Function Code (Python)

def lambda_handler(event, context):
    start_time = time.time()
    offset = event.get("offset", 0)

    while True:
        records, next_offset = function_to_get_company_records(offset)

        if not records:
            break

        for record in records:
            function_to_process_company_records(record)

        if not next_offset:
            break

        if time.time() - start_time > TIME_LIMIT:
            return {
                "status": "incomplete",
                "offset": next_offset
            }

    return {
        "status": "complete"
    }

📸 Screenshots

How to Avoid AWS Lambda Timeout When Processing HubSpot Records

✅ Result

After using AWS Step Functions, the Lambda function now works without timing out. It can handle large HubSpot data by splitting it into smaller parts and processing them one batch at a time.

This setup improves reliability and avoids hitting AWS Lambda limits. It is now scalable and ready for bigger datasets.

📌 Key Takeaways

Lambda has a 15-minute max timeout (900 seconds)
HubSpot data can be large, so use pagination
Step Functions help repeat the Lambda task until finished
This design pattern is serverless, scalable, and cost-effective