Designing automations that fail well

Expect the interruption

An automation is a promise that a small piece of work will happen without someone watching. The interesting design question is what happens when that promise cannot be kept.

A scheduled brief is a useful example. It might read a collection of notes, group them, and save a draft. Each boundary can fail independently. Treating the whole workflow as one opaque action makes recovery harder than it needs to be.

Make steps repeatable

Give each run a stable identity. Derive it from the work being performed, such as the week being summarized, rather than the time a retry happens. A retry should complete the same job without creating a second output.

Record completion at the point where an output is saved. In a real implementation, the check and write need to share a transaction or use a unique database constraint. This sketch shows the idea, not a complete storage implementation.

                  type Run = { key: string; notes: Note[] };

async function createBrief(run: Run) {
  const draft = summarize(run.notes);
  // Atomically insert, or return the existing draft.
  return store.insertOnce(run.key, draft);
}
                

Retry with intention

A temporary connection failure and a missing permission need different responses. Retry temporary failures with a capped delay. Stop on invalid input or missing access and make the reason visible.

Keep a record of the current stage, the last successful boundary, and the input identity. These details are usually more useful than a generic “something went wrong” message.

Leave a way back

Prefer drafting to sending and previewing to publishing when a human decision belongs in the workflow. A recoverable output gives the person using the automation a chance to inspect what happened.

Reliability starts with modest guarantees you can explain. The system does not need to hide its uncertainty; it needs to make its state understandable.