docs
/
AI skills

Fix a failed automation

Find the runs that failed or never happened, read each step's result to see why, fix the step, run it again only for the records that missed it (from the failed step, not from the top), and clear runs that are stuck waiting.

An automation can go wrong in three ways: a run failed at a step, a run finished but did the wrong thing, or no run started at all. Each one is diagnosed from what the engine kept: one automation_execution per run, with every step's result. This page reads those, fixes the cause and runs the missed records again. For building an automation in the first place, see Automation when a record is added or updated. For a lead follow-up, see New-lead follow-up.

Ask first

  • Which automation? By name. If there are several with similar names, list them (list_automations) and ask.
  • What should have happened, and for whom? One record that missed it (a name or email) is the fastest way in.
  • May the missed records be run again? A re-run really does every step it runs again: emails, texts, tasks, records. Ask before re-running anything that reaches people.

Steps

1. Automation monitor: where it stands

call_appengine GET /automation/<sk>/monitor (last 14 days; ?days= up to 90). Read:

PartWhat it tells you
automation.statusactive, draft, inactive. Only active runs on events.
health.triggerRegistered, health.triggerKeysWhether it is listening, and for what (data_created:ticket). false on an active automation: nothing can start it.
health.problemsSteps the engine says can't run: { stepId, message }. The same list as GET /automation/<sk>/check.
totalsruns, completed, failed, running, waiting, successRate, lastRunAt, lastFailureAt
errorsFailures grouped by message, each with count, lastSeen and the stepId it failed at
stepsPer step: ran, failed, lastError
liveRuns still going or waiting on a delay or a reply

Then the runs themselves: GET /automation/<sk>/runs?status=failed (also completed, running, paused, cancelled). q= searches run ids, errors and the record's name, email, phone or number; stepId=<id>&outcome=failed narrows to runs that failed at one step; from / to take dates. Each row has executionId, status, failedStepId, error and subject (who the run was about).

Hands on: the automation's sk, the failing stepId, and the executionIds of the runs that missed. Check: the same numbers on screen at /automation?monitor=<sk>: the tiles, Errors with Fix this step →, Steps, and the Runs log.

2. Automation monitor: why it went wrong

Open one run: GET /automation/executions/<executionId>. results[] has one entry per step that ran, in order, with success, error, data (what it did) and nextStepId; errors[] is the run's reason; variables is the data it ran with (data, record, dataId).

A step that isn't in results[] never ran. A run stops at the first failed step: nothing after it ran.

A run failed. Match the step's error:

error saysCauseFix
Action configuration invalid: Required field 'update' is missing (or another field)The step's config lacks a field the action needsAdd it (step 3). The check lists every one.
Action not found / "x" is not something the engine can runA step name the engine doesn't haveUse a name from list_automation_capabilities
Customer should be created using user apicreate_data on customerAdd data.accountType: "contact"
Customer should be updated using user apiadd_tag, remove_tag or create_note on a customerUse update_data with $addToSet / $pull (see the automation page)
Workflow <sk> is not active (status: draft)An event reached it while it wasn't onSwitch it on with Start (step 5), or leave it off on purpose
Unknown operator: xA condition's operator is misspelledUse one listed on the automation page
A record_exists or create_ticket failure on an empty valueA {{…}} it relies on was empty for this recordNarrow the trigger to records that have it, or check exists first
A sending error from send_email / send_smsThe address, number or sending accountFix the recipient field or the account; see Business email and the sending address

A run completed but did the wrong thing.

What you seeCause
A condition's branchPath went the other wayThe value isn't what you expected. Every value is text: compare "20", not 20. Read it in variables.data.
{{data.x}} saved or sent as writtenThat field was empty or misspelled for this record. It is left as text.
A step reports success and nothing changedIt is one of the steps that do nothing (update_ticket, update_lead, qualify_lead, calculate_score, get_data and the others listed on the automation page). Use update_data.
A step is missing from results[] though the steps around it ranIts enabled isn't true (see the warning).
It ran for the record as it was, not as it isAfter a delay, or on a retry, the run uses the record from when it started.

A step without "enabled": true is skipped without a word. The engine runs only steps whose enabled is true, while the check and the monitor treat a missing enabled as on: the monitor lists the step as enabled with ran: 0 (tested). create_automation and Studio's builder set it; a raw save (repo_create, POST /automation with full steps, PUT) may not. Give every step "enabled": true.

No run started at all. Work down this list:

  1. automation.status isn't active, or health.triggerRegistered is false. Saving active doesn't connect the trigger; Start does (step 5).
  2. The trigger doesn't match: wrong datatype (a form's answers are its collection, not the form), wrong operation, a whereField / whereValue that doesn't match as text, or a changedField that wasn't in the save.
  3. The change didn't go through a full save. Status buttons, workflow stage moves, imports, partial updates and the automation's own update_data fire no updated automations.
  4. It was skipped to stop a loop. One automation runs at most 3 times in one chain of runs, each started by the changes of the run before. The skipped run isn't in the runs list: it is a run-skipped line in Automation › Runs & Logs › All Logs (/automation?tab=runs-logs), "Skipped to prevent a loop…" (tested).
  5. Its trigger was changed while it was running. The listening keys (triggerKeys) are set at Start; a new record type or operation isn't heard until it is stopped and started again.

Hands on: the cause, the step it is in, and the list of records that missed it (from subject and variables.dataId of each failed run). Check: you can name, in one sentence each, what failed, at which step, and for whom.

3. Automation: fix the step

call_appengine GET /automation/<sk>, change only the step at fault, and save it back with PUT /automation/<sk> and the body { "datatype": "automation", "sk": "<sk>", "data": { …the whole data, with the fixed step… } }.

  • Send only datatype, sk and data. Sending the record back exactly as it was read (with _id, pk, version, dates) answers 500 Failed to update automation (tested).
  • An active automation is checked on every save: one that can't run is refused with WORKFLOW_NOT_READY and its problems, and stays as it was.
  • Don't fix it in Studio's builder if the step is an Update Record: the builder saves a shape the engine refuses (see the automation page).

Fix this step → on the monitor opens the builder at that step, for a person doing it by hand.

Hands on: the saved automation. Check: GET /automation/<sk>/check answers { "ready": true, "problems": [] }.

4. Automation: run it again for the records that missed it

Try the fix once first, on a test record (names starting test-, emails @test.local), as on the automation page: POST /automation/<sk>/execute with { "variables": { "datatype": "<datatype>", "recordId": "<sk>", "dataId": "<sk>" } }, then read the run.

Then the records that missed it. There are two ways, and they are not the same:

From the failed stepRetry
CallPOST /automation/<sk>/execute with { "startStepId": "<failed stepId>", "variables": { "datatype", "recordId", "dataId" } }POST /automation/executions/<executionId>/retry
RunsOnly the failed step and the steps after itEvery step again, from the top
RecordAs it is now (loaded fresh)As it was when the first run started
On screennoneRetry on a failed run in the monitor

Tested: a retry of a run that failed at its third step made the first step's record a second time, and used the record's old name after it had been renamed. Starting at the failed step ran only that step and the one after it, with the current record.

Use from the failed step whenever a step before it sends, creates or charges. Use Retry only when nothing before the failed step reaches anyone or makes anything, or when the person wants it all done again.

Stop: a re-run does its steps for real. Show the person the list of records, the step it will start from, and every email, text, task or record each re-run will make. Wait for a yes, then run them one record at a time.

Hands on: one new executionId per record. Check: each new run is completed (GET /automation/executions/<id>), and no record got a step twice (look at what the steps made: the records, the sent messages).

5. Automation: listening again, and stuck runs

Not listening (triggerRegistered: false on an active automation): POST /automation/start/<sk>. That is the monitor's Reconnect trigger button. It answers WORKFLOW_NOT_READY with the problems if a step still can't run. Starting changes nothing for records that already missed it: do step 4 for them.

Stop: Start makes it run for every matching record from now on. If it was off on purpose, ask first.

Stuck runs. live on the monitor lists runs running or paused (waiting on a delay, a reply, an open). A paused run resumes with the steps as they are now (it carries on at the step after its wait) and the record as it was when it started. To end one: POST /automation/executions/<executionId>/cancel (Stop run). Its wait is cancelled with it. Stopping the whole automation (POST /automation/stop/<sk>) ends all its live runs.

Check: GET /automation/<sk>/monitor shows triggerRegistered: true and the triggerKeys you expect, and live holds only runs that should still be waiting.

When it's done

Tell the person:

  • what failed, at which step, why, and since when (lastFailureAt, the error's count);
  • what was changed in the step;
  • which records were run again, from which step, and what each re-run did;
  • which runs were stopped;
  • whether the automation is running and listening, and for what.

Open the monitor (/automation?monitor=<sk>).

What it can't do yet

  • Retry from a step. Studio's Retry always starts from the top with the old data. Starting at a step is only through execute with startStepId.
  • Tell you a step was skipped. A step without enabled: true leaves no trace in the run; the monitor even shows it as enabled.
  • Show skipped loop runs with the runs. They are only in All Logs.
  • Catch up by itself. Fixing a step or reconnecting the trigger doesn't run the records that were missed.
  • Re-read the record after a wait or on a retry. Both use the record from when the run started.
  • Retry a step on its own after a passing error. A failing step stops the run at once; there is no retry or "carry on" setting per step.

Don'ts

  • Don't press Retry on a run whose earlier steps email, text, create or charge. Start from the failed step.
  • Don't re-run every failed run in one go. One record at a time, and read the first one.
  • Don't send the whole record back to PUT /automation/<sk>. Send datatype, sk and data.
  • Don't call a step fixed until GET /automation/<sk>/check says ready: true and a test run passes it.
  • Don't trust a step's success without reading what it did.
  • Don't switch an automation on to fix it. Fix it, test it, then Start it on a yes.