bamaer opened a new issue, #9631: URL: https://github.com/apache/hop/issues/9631
### What would you like to happen? A workflow engine that runs a Hop workflow as an Apache Airflow DAG. Workflows are still designed in Hop GUI; Airflow handles scheduling, retries and per-action task state. - **Translator:** builds a DAG from each `.hwf` with Airflow's Java Task SDK (Airflow 3.4+). The Hop project is the DAG bundle, and Hop is installed on the workers. - **Fragment runner:** runs one action, or a group of actions, in a task. Variables and the previous result are handed over through XCom; the runner returns the result, the changed variables and the result rows. Loops, actions that would run more than once, and "Execute for every input row" run as one fragment, so results don't change. - **"Airflow" run configuration in Hop GUI:** publish, trigger, follow task state on the canvas, and stop. It uses `airflow-connection` from #9630. The engine's `supports()` marks the actions it can't run. On a sample of real-world workflows, about 9 in 10 map one action per task, but this depends on a project's style. The rest run partly as Hop fragments. The goal is that every workflow runs with the same results as on the local engine. About a quarter of workflows pass state between actions, so the XCom handover is needed from the start. The Java SDK reached 1.0.0 in November 2026 and is still experimental. It has no DAG params, which matters because many workflows have parameters, and it doesn't enforce `execution_timeout` yet. Keep it behind one adapter module and pin its version per Hop release. Start with a spike that runs one task per workflow and covers passing parameters, then move to one task per action. ### Issue Priority Priority: 3 ### Issue Component Component: Workflows -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
