Re: Wait for child file to process successfully

2016-12-24 Thread Jeff
Brian, Take a look at these two JIRAs [1] [2], specifically the first, which should be released in 1.2.0. The second is mostly an FYI so that you are aware of some of the other aggregation capabilities in NiFi. It has not yet been merged to master. There is a way to implement a flow that would ha

Re: Wait for child file to process successfully

2016-12-23 Thread BD International
Jeff, Thanks for that just tried it out and it works perfectly! On a similar topic I have a flow which picks up a CSV and turns each row into an AVRO object and posts that to a web service I've setup. I would like to do something similar where I dont delete the original CSV file until i have succ

Re: Wait for child file to process successfully

2016-12-22 Thread Jeff
Brian, You can use MergeContent in Defragment mode. Just be sure to set the number of bins used by MergeContent equal to or greater than the number of concurrent merges you expect to have going on in your flow, and to route successfully processed and failed flowfiles (after they've been gracefull

Wait for child file to process successfully

2016-12-22 Thread BD International
Hello, I've got a data flow which picks up a zip file and uses UnpackContent to extract the contents. The subsequent files are them converted to json and stored in a database. I would like to store the original zip file and only delete the file once all the extracted files have been stored correc

Wait for child file to process successfully

2016-12-22 Thread BD International
Hello, I've got a data flow which picks up a zip file and uses UnpackContent to extract the contents. The subsequent files are them converted to json and stored in a database. I would like to store the original zip file and only delete the file once all the extracted files have been stored correc