Design for uncertain outcomes
A workflow can fail after an external service accepted an action but before the next step recorded success. Treat that as an unknown outcome: inspect the target system or use a request key that lets the destination recognize duplicates.
Separate preparation from irreversible actions. Validate inputs and calculate the intended change first, then perform the send, payment, publish, or delete step only when its prerequisites are satisfied.
Record progress and verify the effect
Store a business identifier and step status where the workflow can inspect them on retry. After a failure, check the run history and the external resource so you can identify the last confirmed step.
For actions that cannot be made idempotent, add a review or reconciliation step instead of automatic retries. Test the recovery path with a controlled example and confirm that a second run does not create duplicate records or messages.