Thanks Dan. The blogs encourage us to keep building. Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Sep 5, 2018, at 4:25 PM, dan young <[email protected]> wrote: > > Heya Andy, > > yes, that seems legit...we'll make it work on our side... > > Keep up the awesome work on NiFi, powers all of our ETL here now :) > > Dano > > On Wed, Sep 5, 2018 at 5:14 PM Andy LoPresto <[email protected] > <mailto:[email protected]>> wrote: > Dan, > > Does the proposal I submitted meet your requirements? > > Andy LoPresto > [email protected] <mailto:[email protected]> > [email protected] <mailto:[email protected]> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Sep 5, 2018, at 4:09 PM, dan young <[email protected] >> <mailto:[email protected]>> wrote: >> >> We're using it as well, in the same/similar fashion as being discussed in >> the thread... >> >> Dano >> >> On Wed, Sep 5, 2018, 10:07 AM Brandon DeVries <[email protected] >> <mailto:[email protected]>> wrote: >> Andy, >> >> We use it pretty much how Joe is... to create a unique composite key. It >> seems as though that shouldn't be a difficult functionality to add. >> Possibly, you could flip your current dynamic key/value properties. Make >> the key the name of the attribute you want to create, and the value is the >> attribute / attributes (newline delimited) that you want to include in the >> hash. This does mean you can't use "${algorithm.name >> <http://algorithm.name/>}" in the name of the created hash attribute, but I >> don't know if you'd consider that a big loss. In any case, I'm sure there >> are other solutions, this is just a thought. >> >> Brandon >> >> On Wed, Sep 5, 2018 at 10:27 AM Joe Percivall <[email protected] >> <mailto:[email protected]>> wrote: >> Hey Andy, >> >> We're currently using the HashAttribute processor. The use-case is that we >> have various events that come in but sometimes those events are just updates >> of previous ones. We store everything in ElasticSearch. So for certain >> events, we'll calculate a hash based on a couple of attributes in order to >> have a composite unique key to upsert as the ES _id. This allows us to >> easily just insert/update events that are the same (as determined by the >> hashed composite key). >> >> As for the configuration of the processors, we're essentially just >> specifying exact attributes as dynamic properties of HashAttribute. Then >> passing that FF to PutElasticSearchHttp with the resulting attribute from >> HashAttribute as the "Identifier Attribute". >> >> Joe >> >> On Mon, Sep 3, 2018 at 9:52 PM Andy LoPresto <[email protected] >> <mailto:[email protected]>> wrote: >> I opened PRs for 2980 [1] and 2983 [2] which add more performant, >> consistent, and full-featured processors to calculate cryptographic hashes >> of flowfile content and flowfile attributes. I would like to deprecate and >> drop support for HashAttribute, as it performs a convoluted calculation that >> was probably useful in an old scenario, but doesn’t “hash attributes” like >> the name implies. As it blocks the new implementation from using that name >> and following our naming convention, I am hoping to find anyone still using >> the old implementation and understand their use case. Thanks for your help. >> >> [1] https://github.com/apache/nifi/pull/2980 >> <https://github.com/apache/nifi/pull/2980> >> [2] https://github.com/apache/nifi/pull/2983 >> <https://github.com/apache/nifi/pull/2983> >> >> >> >> Andy LoPresto >> [email protected] <mailto:[email protected]> >> [email protected] <mailto:[email protected]> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> >> >> -- >> Joe Percivall >> linkedin.com/in/Percivall <http://linkedin.com/in/Percivall> >> e: [email protected] <mailto:[email protected]>
signature.asc
Description: Message signed with OpenPGP using GPGMail
