I should qualify what my security concern on the map cache was. Given a shared cache server any other flow not related to mine could read my keys.
> On 5 Jan 2018, at 08:32, Andy LoPresto <[email protected]> wrote: > > UpdateAttribute doesn’t pull from a database, it uses static or dynamic > attribute values and supports NiFi Expression Language. In your original > message, you didn’t mention any database interaction, so I thought you were > just trying to accomplish "I wanted to add some attributes to a FlowFile > while not altering the contents of that FlowFile”, which is indeed what > UpdateAttribute does. > > If you need to retrieve those values from a database, as Joey mentions, the > LookupService is the right tool. > > With your prior setup, the distributed map cache is as secure as the NiFi > configuration — if using secured NiFi, the communication between that node > and any other is over TLS, and within the node it’s a memory access. > > A big part of the NiFi philosophy is the same as the Unix philosophy — each > tool should do one job very well, and to perform complicated tasks, chain > those tools together. This helps with provenance reporting, usage reporting, > debugging, flow development lifecycle, maintenance, etc. A processor which > retrieves attributes from a database and updates the incoming flowfile with > them is certainly useful in the use case you describe, but is not a generic > pattern. There’s no intent to discourage custom development, and whatever > makes your flow work is fine. Just explaining why you likely won’t see a > solution like that in the NiFi bundled components. > > > Andy LoPresto > [email protected] > [email protected] > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jan 4, 2018, at 4:07 PM, Brett Ryan <[email protected]> wrote: >> >> Thanks Andy, how would update attribute be able to get the value from sql? >> >> Consider a flow where a piece of information needs to be obtained from a DB >> but i do not want the contents of the current FF to be altered, using >> ExecuteSQL anywhere prior would not be possible due to replacing the FF >> contents. >> >> What I had was two seperate flows, one that updates an oauth key in a db >> keeping it fresh, the main flow would then read the db just before an >> invokehttp. >> >> I was originally using a distributed map cache but had concerns that it >> might not be secure and was also advised the cache server has been known to >> go down. >> >>> On 5 Jan 2018, at 04:01, Joey Frazee <[email protected]> wrote: >>> >>> Andy, Brett, >>> >>> Taking a quick glance at the code it looks like it's enriching attributes >>> from a database according to a query. >>> >>> If that's correct, there's a LookupAttribute processor that delegates >>> lookups to a "LookupService" and adds attributes without altering content. >>> There are a variety of these LookupServices included. I think what you >>> implemented would make sense as a LookupService and then you could just >>> configure the processor to use that. It could also be used with >>> LookupRecord then too so there'd be a double payoff. >>> >>> -joey >>> >>>> On Jan 4, 2018, 8:44 AM -0800, Andy LoPresto <[email protected]>, wrote: >>>> Hi Brett, >>>> >>>> It’s great that you found it easy to write a new processor for Apache >>>> NiFi. It is probably an indicator that we need to improve >>>> education/evangelism/documentation, however, that you did not find >>>> UpdateAttribute [1], which should do exactly what you were looking for. >>>> >>>> [1] >>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-update-attribute-nar/1.4.0/org.apache.nifi.processors.attributes.UpdateAttribute/index.html >>>> >>>> Andy LoPresto >>>> [email protected] >>>> [email protected] >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>>> On Jan 4, 2018, at 7:03 AM, Brett Ryan <[email protected]> wrote: >>>>> >>>>> Hi all, having used NiFi for a couple days I wanted to add some >>>>> attributes to a FlowFile while not altering the contents of that FlowFile. >>>>> >>>>> I had suggestions to use a script processor but that just sounded like a >>>>> hack which could become a nuisance to replicate. >>>>> >>>>> Anyway, I figured I'd write a processor to do this, anyone interested you >>>>> can find it here. >>>>> >>>>> https://github.com/brettryan/nifi-drunken-bundle >>>>> >>>>> Feel free to do with it what you please. >>>>> >>>>> I've published to maven central but it will take a day to appear in the >>>>> search. >>>>> >>>>> <dependency> >>>>> <groupId>com.drunkendev</groupId> >>>>> <artifactId>nifi-drunken-nar</artifactId> >>>>> <version>1.0.0</version> >>>>> <type>nar</type> >>>>> </dependency> >>>>> <dependency> >>>>> <groupId>com.drunkendev</groupId> >>>>> <artifactId>nifi-drunken-processors</artifactId> >>>>> <version>1.0.0</version> >>>>> </dependency> >>>>> <dependency> >>>>> <groupId>com.drunkendev</groupId> >>>>> <artifactId>nifi-drunken-bundle</artifactId> >>>>> <version>1.0.0</version> >>>>> <type>pom</type> >>>>> </dependency> >>>>> >>>> >
