Hi Daniel The red table entries concerning airflow are bullshit in some relevant aspects. If you want to have a more reasonable detailed argumentation, please call me up: +49 171 5357916. Airflow supports different executers as an (independent) plugins like celery- or dask- or Kubernetes-executor. In the future prefect might be an option.
To speed up dags life cycle multiple dag processors or python (directly) could be used. A not that bad at all alternative to airflow, especially for HPDA pipelines could be dagster. Forget Argo, Airflow or Dagster are the right solutions for Workflows usable also for hybrid clouds. Argo is good for Kubernetes orchestration scenarios. Am I right ? Or if you want speed and only speed look at Rust Waterweel. Don't forget you might need not only a Workflow Manager but also a Workload Manager like SLURM or LSF. Best regards bs -----Original Message----- From: Daniel Boeckenhoff <daniel.f.boeckenh...@gmail.com> Sent: Tuesday, September 26, 2023 1:00 PM To: users@airflow.apache.org Subject: Comments regarding comparison between airflow, argo and prefect Dear all, I am currently investigating if airflow is the rigth solution for our problem. I found https://medium.com/arthur-engineering/picking-a-kubernetes-orchestrator-airflow-argo-and-prefect-83539ecc69b and realized it was written by an argo developer. I would like to get your opinion on the first table in that article. Is it biased due to its author, is it still up-to-date, are relevant categories not compared in order to let argo shine, ...? I appreciate all comments. Best, Daniel --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@airflow.apache.org For additional commands, e-mail: users-h...@airflow.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@airflow.apache.org For additional commands, e-mail: users-h...@airflow.apache.org
