The main processors we have for interacting with JSON are EvaluateJsonPath and SplitJson.
Regarding a lookup file you probably want to take a look at ScanAttribute and ScanContent: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ScanAttribute/index.html https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ScanContent/index.html You could potentially use ScanContent directly against the json, or you could first use EvaluateJsonPath to extract a field of interest from the json into a FlowFile attribute, and then use ScanAttribute to check that attribute against the lookup values. -Bryan On Thu, Nov 5, 2015 at 5:13 PM, Madhire, Naveen < [email protected]> wrote: > The data format is JSON coming from Kafka. Is there any processor for > actually matching the contents of the flow files against a lookup file ? > > > From: Bryan Bende <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Thursday, November 5, 2015 at 4:07 PM > To: "[email protected]" <[email protected]> > Subject: Re: Processor to use > > Hello, > > Can you share the data format coming from Kafka? json, avro, something > else? > > The reason I ask is because there are many different processors that can > modify FlowFiles in different ways, but a lot depends on the type of data > involved. > > For general text-based operations we have... > - ReplaceText to modify the content of the FlowFile > - ExtractText to extract parts of the content into FlowFile attributes > - UpdateAttribute to add/update a FlowFile attribute, can use expression > language here to do dynamic stuff > > There are processors that interact with json, xml, and avro as well. > > If you have a need that is outside all of the existing processors, it is > also possible to develop your own custom processor. Let us know and we can > point you in the right direction. > > -Bryan > > > On Thu, Nov 5, 2015 at 4:55 PM, Madhire, Naveen < > [email protected]> wrote: > >> Hi All, >> >> Just need few suggestions on which Processor to use. I have the below >> need and I need to figure out which processor to use in the 2nd point as I >> don’t see any processor to write customize logic. >> >> >> 1. Read the data from Kafka Topic ( I will use the getKafka processor) >> 2. *Apply business logic to the data fetched from Kafka like few >> calculations (Here I would need which processor to use to modify the flow >> files on the flow)* >> 3. Write the data into HDFS. (putHDFS processor) >> >> Is there any processor which I can use to make the changes to the flow >> files? >> >> Thanks. >> >> ------------------------------ >> >> The information contained in this e-mail is confidential and/or >> proprietary to Capital One and/or its affiliates and may only be used >> solely in performance of work or services for Capital One. The information >> transmitted herewith is intended only for use by the individual or entity >> to which it is addressed. If the reader of this message is not the intended >> recipient, you are hereby notified that any review, retransmission, >> dissemination, distribution, copying or other use of, or taking of any >> action in reliance upon this information is strictly prohibited. If you >> have received this communication in error, please contact the sender and >> delete the material from your computer. >> > > > ------------------------------ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. >
