GitHub user pcserna created a discussion: Apache Airflow vs Hop Server (vs Hop-run)
Please, help me understand why it is favorable using Apache Airflow for orchestration: as I can see, with the DAG python script Airflow set up a new instance of Apache Hop, so if I run many DAGs in parallel, they would eat up resources, not to mention the warm-up time-overhead. Ok, it is a good thing, if your Workflow fails you can start over, you can see the KPIs of your runs and you can set up cron in a pythonic way, but what else? In a production environment you have to set up a backend database, too (which may not be the one you operate). Not to mention - as I see - it is preferred to use docker. So, under the very end Apache Hop will do the ETL, right? If I have to operate many Workflows and Pipelines e.g. in every 15 minutes in parallel, I think I would use as simple backend as I could with the lowest necessary hardware and software need. Do you think that doing many tasks with separated hop-runs have a comparable computing load to one hop-server? Thanks for sharing your experiences! GitHub link: https://github.com/apache/hop/discussions/5654 ---- This is an automatically sent email for users@hop.apache.org. To unsubscribe, please send an email to: users-unsubscr...@hop.apache.org
