Running Processors Synchronously

2016-02-11 Thread obaidul karim
Hi All, Lets say, I have below processors: 1.listfile > 2.fetchfile > 3.putHDFS > 4.ExecuteSQL I want to run all above processors in sequence. Processor 1 will wait for next run until all 2,3 & 4 completes. Similarly 2 will wait until 3 & 4 completes current run and so on. In other words, can

Re: Running Processors Synchronously

2016-02-11 Thread obaidul karim
Hi, The way I expect NiFi, is a suitable replacement of non-opensource & so-called opensource ETL tools :). In traditional ETL(or ELT) process, we use below process flow to load data in near real time: 1. spool dir > 2. collect available files > 3. load files in a "staging table" DB without

Re: Running Processors Synchronously

2016-02-11 Thread Oleg Zhurakousky
Obaid Thanks for reaching out! Currently it is not possible to wire a flow the way you describe. Wha you are asking is a true Event Driven Consumer paradigm which been discussed internally a lot lately, so it would be very interesting to get your perspective as to why do you believe it is