Re: Merge multiple flowfiles

2016-06-03 Thread Oleg Zhurakousky
Huagen, I also want to apologize for my spell-checker butchering your name ;) Cheers Oleg On Jun 3, 2016, at 8:03 AM, Oleg Zhurakousky > wrote: Huge Just to close the loop on this one, I also wanted to point out this JIRA

Re: Merge multiple flowfiles

2016-06-03 Thread Oleg Zhurakousky
Huge Just to close the loop on this one, I also wanted to point out this JIRA https://issues.apache.org/jira/browse/NIFI-1926 for general purpose aggregation processor which indeed would support multiple connections, configurable aggregation, release and correlation strategies. It would be

Re: Merge multiple flowfiles

2016-06-02 Thread Andy LoPresto
Huagen, Sorry, I am a little confused. My understanding is that you want to combine n individual logs (each with a respective flowfile) from a specific hour into a single file. What is confusing is when you say “Even with that [a 5* confirmation loop], I occasionally still get more than one

Merge multiple flowfiles

2016-06-01 Thread Huagen peng
Hi, In the data flow I am dealing with now, there are multiple (up to 200) logs associated with a given hour. I need to process these fragment hourly logs and then concatenate them into a single file. The approach I am using now has an UpdateAttribute processor to set an arbitrary