Dan, Does the proposal I submitted meet your requirements?
Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 5, 2018, at 4:09 PM, dan young <danoyo...@gmail.com> wrote: > > We're using it as well, in the same/similar fashion as being discussed in the > thread... > > Dano > > On Wed, Sep 5, 2018, 10:07 AM Brandon DeVries <b...@jhu.edu > <mailto:b...@jhu.edu>> wrote: > Andy, > > We use it pretty much how Joe is... to create a unique composite key. It > seems as though that shouldn't be a difficult functionality to add. > Possibly, you could flip your current dynamic key/value properties. Make the > key the name of the attribute you want to create, and the value is the > attribute / attributes (newline delimited) that you want to include in the > hash. This does mean you can't use "${algorithm.name > <http://algorithm.name/>}" in the name of the created hash attribute, but I > don't know if you'd consider that a big loss. In any case, I'm sure there > are other solutions, this is just a thought. > > Brandon > > On Wed, Sep 5, 2018 at 10:27 AM Joe Percivall <jperciv...@apache.org > <mailto:jperciv...@apache.org>> wrote: > Hey Andy, > > We're currently using the HashAttribute processor. The use-case is that we > have various events that come in but sometimes those events are just updates > of previous ones. We store everything in ElasticSearch. So for certain > events, we'll calculate a hash based on a couple of attributes in order to > have a composite unique key to upsert as the ES _id. This allows us to easily > just insert/update events that are the same (as determined by the hashed > composite key). > > As for the configuration of the processors, we're essentially just specifying > exact attributes as dynamic properties of HashAttribute. Then passing that FF > to PutElasticSearchHttp with the resulting attribute from HashAttribute as > the "Identifier Attribute". > > Joe > > On Mon, Sep 3, 2018 at 9:52 PM Andy LoPresto <alopre...@apache.org > <mailto:alopre...@apache.org>> wrote: > I opened PRs for 2980 [1] and 2983 [2] which add more performant, consistent, > and full-featured processors to calculate cryptographic hashes of flowfile > content and flowfile attributes. I would like to deprecate and drop support > for HashAttribute, as it performs a convoluted calculation that was probably > useful in an old scenario, but doesn’t “hash attributes” like the name > implies. As it blocks the new implementation from using that name and > following our naming convention, I am hoping to find anyone still using the > old implementation and understand their use case. Thanks for your help. > > [1] https://github.com/apache/nifi/pull/2980 > <https://github.com/apache/nifi/pull/2980> > [2] https://github.com/apache/nifi/pull/2983 > <https://github.com/apache/nifi/pull/2983> > > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > -- > Joe Percivall > linkedin.com/in/Percivall <http://linkedin.com/in/Percivall> > e: jperciv...@apache.com <mailto:jperciv...@apache.com>
signature.asc
Description: Message signed with OpenPGP using GPGMail
