Re: Proposal: standard record metadata attributes for data sources

2018-05-24 Thread Otto Fowler
I commented on the PR, but I’ll add this to the thread here. Wouldn’t something like this lend itself to a ReportingTask? If not the current structure, a like structure for records? That would allow the destination to do time series analysis etc. That is not saying there isn’t a case to have it

Re: Put data to Elastic with static settings or index template

2018-05-24 Thread Bobby
Yep, i used index template and it worked well...If my calculation is right, the size decreased about 12 - 18%..huge gain!!! Consider this issue solved.. Thank you everyone - - Bobby -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Proposal: standard record metadata attributes for data sources

2018-05-24 Thread Mike Thomsen
I wrote a processor that's inspired by one of the Groovy scripts we use at that client. PR is here if anyone wants to take a look: https://github.com/apache/nifi/pull/2737 It's called "RecordStats" and provides both a general record count attribute and lets you specify record path operations to