Heya Andy, yes, that seems legit...we'll make it work on our side...
Keep up the awesome work on NiFi, powers all of our ETL here now :) Dano On Wed, Sep 5, 2018 at 5:14 PM Andy LoPresto <[email protected]> wrote: > Dan, > > Does the proposal I submitted meet your requirements? > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Sep 5, 2018, at 4:09 PM, dan young <[email protected]> wrote: > > We're using it as well, in the same/similar fashion as being discussed in > the thread... > > Dano > > On Wed, Sep 5, 2018, 10:07 AM Brandon DeVries <[email protected]> wrote: > >> Andy, >> >> We use it pretty much how Joe is... to create a unique composite key. It >> seems as though that shouldn't be a difficult functionality to add. >> Possibly, you could flip your current dynamic key/value properties. Make >> the key the name of the attribute you want to create, and the value is the >> attribute / attributes (newline delimited) that you want to include in the >> hash. This does mean you can't use "${algorithm.name}" in the name of >> the created hash attribute, but I don't know if you'd consider that a big >> loss. In any case, I'm sure there are other solutions, this is just a >> thought. >> >> Brandon >> >> On Wed, Sep 5, 2018 at 10:27 AM Joe Percivall <[email protected]> >> wrote: >> >>> Hey Andy, >>> >>> We're currently using the HashAttribute processor. The use-case is that >>> we have various events that come in but sometimes those events are just >>> updates of previous ones. We store everything in ElasticSearch. So for >>> certain events, we'll calculate a hash based on a couple of attributes in >>> order to have a composite unique key to upsert as the ES _id. This allows >>> us to easily just insert/update events that are the same (as determined by >>> the hashed composite key). >>> >>> As for the configuration of the processors, we're essentially just >>> specifying exact attributes as dynamic properties of HashAttribute. Then >>> passing that FF to PutElasticSearchHttp with the resulting attribute from >>> HashAttribute as the "Identifier Attribute". >>> >>> Joe >>> >>> On Mon, Sep 3, 2018 at 9:52 PM Andy LoPresto <[email protected]> >>> wrote: >>> >>>> I opened PRs for 2980 [1] and 2983 [2] which add more performant, >>>> consistent, and full-featured processors to calculate cryptographic hashes >>>> of flowfile content and flowfile attributes. I would like to deprecate and >>>> drop support for HashAttribute, as it performs a convoluted calculation >>>> that was probably useful in an old scenario, but doesn’t “hash attributes” >>>> like the name implies. As it blocks the new implementation from using that >>>> name and following our naming convention, I am hoping to find anyone still >>>> using the old implementation and understand their use case. Thanks for your >>>> help. >>>> >>>> [1] https://github.com/apache/nifi/pull/2980 >>>> [2] https://github.com/apache/nifi/pull/2983 >>>> >>>> >>>> >>>> Andy LoPresto >>>> [email protected] >>>> *[email protected] <[email protected]>* >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>> >>> >>> -- >>> *Joe Percivall* >>> linkedin.com/in/Percivall >>> e: [email protected] >>> >> >
