I think the only services that won't break if you merge the variables are the ones that use only mandatory keys. Ones like MongoDBLookupService will simply take the entire lookup coordinate map and turn it into a query. I don't think there's an elegant solution to allow the service to pick and choose in that case.
On Tue, Jun 12, 2018 at 4:43 AM Koji Kawamura <ijokaruma...@gmail.com> wrote: > Hi Mike, > > I'm still not sure which is better, separating the variables or > containing all values. > I wrote a comment so that we can keep the discussion there. > https://github.com/apache/nifi/pull/2777#issuecomment-396512384 > > Thanks, > Koji > > On Tue, Jun 12, 2018 at 1:56 AM, Mike Thomsen <mikerthom...@gmail.com> > wrote: > > Koji, > > > > After reading Mark's comments on GitHub, it occurred to me that the > MongoDB > > lookup service and the ES one I have as a PR would be screwed up if we > take > > the original approach because they blindly build a query from the total > > coordinates set. So they'd add flowfile attributes as criteria by > default. > > I'll update the PR accordingly and make the new method default to the > > existing one in all of the lookup services that are already there. > > > > On Sat, Jun 9, 2018 at 8:44 AM Mike Thomsen <mikerthom...@gmail.com> > wrote: > > > >> https://issues.apache.org/jira/browse/NIFI-5287 > >> > >> On Sat, Jun 9, 2018 at 1:20 AM Koji Kawamura <ijokaruma...@gmail.com> > >> wrote: > >> > >>> Thanks Mike for starting the discussion. > >>> > >>> Yes, I believe that will make LookupService and Schema access strategy > >>> much easier, reusable, and useful. > >>> > >>> What I was imagined is not adding new method signature, but simply > >>> copy certain FlowFile attributes into the coordinates map. > >>> We can add that at LookupRecord. > >>> Currently LookupAttribute only uses one coordinate value and can be > >>> left as it is. > >>> > >>> Specifically, by adding new processor property, 'Copy FlowFile > >>> Attributes into Coordinates', where user can define RegularExpression > >>> to select which attributes to copy. > >>> I think it's fine to mix FlowFile attributes and values defined as > >>> dynamic properties into the same coordinates map. > >>> The put oder should be FlowFile attributes, then dynamic properties, > >>> so that user can overwrite attribute values when necessary. > >>> > >>> Koji > >>> > >>> > >>> On Sat, Jun 9, 2018 at 1:06 AM, Mike Thomsen <mikerthom...@gmail.com> > >>> wrote: > >>> > On the RestLookupService PR I think Koji mentioned the idea of > expanding > >>> > the lookup capability to include flowfile attributes. That sort of > thing > >>> > would be immensely useful on two PRs I have already open for lookup > >>> service > >>> > changes for ES and Mongo. Koji, add your thoughts, but what I'm > thinking > >>> > would be a new PR that adds: > >>> > > >>> > T lookup(Map<String, String> flowfileAttributes, Map<String, Object> > >>> > coordinates); > >>> > > >>> > to the LookupService interface and has the related processors pass in > >>> the > >>> > flowfile attribute map. Specifically, it would help make the schema > >>> access > >>> > capabilities really usable with lookup services (see > >>> MongoDBLookupService > >>> > PR for example; I added a new SchemaRegistryService type for JSON > >>> sources) > >>> > >> >
