Handling Errors in Flomenco Workflows
Different approaches to Handling errors in batch and single workflows
Overview
When designing workflows in Flomenco, it is essential to anticipate potential errors and implement robust handling mechanisms. Effective error management ensures seamless execution, prevents workflow disruptions, and enables quick identification and resolution of issues. Additionally, the use of sub-workflows for error reporting and handling is highly encouraged, as it promotes modularity, reusability, and maintainability. For more details on structuring workflows efficiently, refer to the Flomenco Workflow Organization Guide.
This guide explores three primary error-handling strategies in Flomenco.
Error Handling Strategies
1. Failing the Workflow and Executing an Error Workflow
When to Use This Approach
- Workflow execution cannot proceed due to a critical error.
- The workflow follows a linear structure where nodes depend on prior successful execution.
Implementation Steps
To configure an error workflow, define an external error-handling process within Workflow Settings. This workflow will execute automatically upon failure, allowing notifications or remediation actions to take place.
- Create an Error Workflow:
- Initiate a new workflow.
- Add the Error Trigger node as the starting point.
- Assign a relevant name, such as
Error Handler. - Save the workflow.
- Link the Error Workflow to an Existing Workflow:
- Open the target workflow.
- Navigate to Workflow Settings.
- Under Error Workflow, select the previously created error-handling workflow (e.g.,
Error Handler). - Save the settings. Once a failure occurs, Flomenco will trigger the error workflow.
This approach centralizes error handling and enables consistent error reporting across multiple workflows.
Note the following
- If a workflow uses the Error Trigger node, you don't have to activate the workflow.
- If a workflow contains the Error Trigger node, by default, the workflow uses itself as the error workflow.
- You can't test error workflows when running workflows manually. The Error Trigger only runs when an automatic workflow errors.
2. Capturing and Reporting Errors Without Interrupting Execution
When to Use This Approach
- The error does not critically impact workflow execution and can be addressed later.
- The workflow processes large data batches where an individual failure should not halt the entire batch.
Designing a Reusable Error Reporting Workflow
A dedicated error reporting workflow ensures error events are documented and communicated without disrupting the primary workflow execution.
Steps to Create an Error Reporting Workflow
- Set Up an Error Handling Workflow:
- Create a new sub-workflow named
Error Reporting. - Add a Execute Workflow Trigger node to receive error data from workflows.
- Integrate Slack, Email, AWS SQS or a database node to log errors and notify stakeholders.
- Save and activate the workflow. Copy workflow ID from browser URL as you will need it to call this workflow
- Integrate the Error Reporting Workflow with Existing Workflows:
- Enable the Error Output feature on nodes where failures are expected.
- Connect the Error Output to an Execute Workflow node that calls our
Error Reportingworkflow (by workflow ID) - Ensure the primary workflow continues execution for subsequent iterations.
By adopting this method, Flomenco users can maintain workflow continuity while logging errors for later review and resolution.
3. Ignoring Errors and Continuing Execution
When to Use This Approach
- Errors do not significantly impact workflow outcomes.
- The workflow should continue processing even if individual nodes encounter issues.
Implementation Steps
To ignore errors and allow the workflow to continue execution, enable the Continue on Error option in the node settings:
- Open the node where errors may occur.
- Navigate to Node Settings.
- Select the Continue on Error option.
- Save and execute the workflow.
This approach is useful when minor errors can be tolerated, and the workflow should not be interrupted due to isolated failures.
4. Retrying on Error
This strategy can be used on its own or in combination with other strategies, such as routing errors via the Error Output after retries fail, or continuing workflow execution by enabling 'Continue on Error' if retries do not succeed.
When to Use This Approach
- The error is transient and may succeed upon retry (e.g., network issues, rate limits).
- You want to reduce the chance of triggering more complex error workflows for recoverable failures.
Implementation Steps
Flomenco allows automatic retries within individual nodes using the built-in retry mechanism:
- Open the node you want to configure.
- Navigate to Node Settings.
- Under the Retry section, enable Retry on Fail.
- Specify the number of retry attempts and the delay between retries.
- Save the node configuration.
This strategy is ideal for handling intermittent errors automatically without additional logic or interruption to workflow execution.
Conclusion
Implementing structured error-handling mechanisms in Flomenco enhances workflow reliability and operational efficiency. You can choose between failing workflows for immediate resolution, logging errors for deferred troubleshooting or simply continuing workflow execution as if nothing has happened.
Last updated on March 20, 2025