We use NiFi for all our ETL, and have done so very happily for several years (including persuading partner organisations to adopt it); we have recently started using Airflow for scheduling periodic machine learning algorithm execution and retraining.
My £0.02 - we probably could do everything we currently do in Airflow using NiFi, but for our Airflow use cases, things like task duration are interesting metrics to monitor and I don't immediately know how we would do this with NiFi. Plus the different user communities make two separate tools no bad thing. On Thu, 15 Aug 2019 at 21:00, Mike Thomsen <mikerthom...@gmail.com> wrote: > > Does anyone see any areas where the two can complement each other and where > we might want to give users the ability to offload processing to Airflow? > Curious since our poking around the docs lead us to conclude it was > probably more "Airflow vs Spark+Oozie" than really competing with NiFi.
