GitHub user pcserna added a comment to the discussion: Apache Airflow vs Hop Server (vs Hop-run)
Thanks for your opinions and use cases! @NicoDupont I have lots of workflows on two servers. The parallelism and the load is optimized for reaching the results within a specified timeframe (there are also failsafe procedures). The servers are mounting shared spaces of mirrored fileservers (it is needed to start up backup servers in case of power or internet outages). Here I have the repos, batch files, python programs, settings, DBs, etc. Many of the workflows refer to sub-workflows and Python procedures, other 3rd party programs. If a highlighted workflow fails an email is sent about it and there is no time to restart (because the same procedure will be started within minutes). I use Carte as an orchestrator now and evaluating / migrating to Hop. The pdi.log is another info-source I use. The batch files are scheduled form Cron. I started to use Carte because running workflows separately from a batch took a lot of time to warm up, eat-up resources, no visual feedback … it was straightforward. I have to decide where to go, which technology should I go with. I prefer to use open-source platforms, of course, and I thank you for forking your work to Hop. As I see now, running the same workflows in Airflow takes a lot of work: I should learn, integrate and operate a considerable amount of new technologies and separate the complex workflows in DAGs, and at the moment I do not have the impression / information that it is worth investing in. I have to consider the pros and cons. I just got used to and I tried to maximize the capabilities of Carte – I know, this is the past, but my life would be easier if “**Export linked resources to server**” Hop procedure (for example) would create one (zipped ) xml and could be called from command line. :-) Am I right that each Airflow calls a new running instance of Hop? GitHub link: https://github.com/apache/hop/discussions/5654#discussioncomment-14254925 ---- This is an automatically sent email for users@hop.apache.org. To unsubscribe, please send an email to: users-unsubscr...@hop.apache.org
