The data format is JSON coming from Kafka. Is there any processor for actually matching the contents of the flow files against a lookup file ?
From: Bryan Bende <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, November 5, 2015 at 4:07 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Processor to use Hello, Can you share the data format coming from Kafka? json, avro, something else? The reason I ask is because there are many different processors that can modify FlowFiles in different ways, but a lot depends on the type of data involved. For general text-based operations we have... - ReplaceText to modify the content of the FlowFile - ExtractText to extract parts of the content into FlowFile attributes - UpdateAttribute to add/update a FlowFile attribute, can use expression language here to do dynamic stuff There are processors that interact with json, xml, and avro as well. If you have a need that is outside all of the existing processors, it is also possible to develop your own custom processor. Let us know and we can point you in the right direction. -Bryan On Thu, Nov 5, 2015 at 4:55 PM, Madhire, Naveen <[email protected]<mailto:[email protected]>> wrote: Hi All, Just need few suggestions on which Processor to use. I have the below need and I need to figure out which processor to use in the 2nd point as I don’t see any processor to write customize logic. 1. Read the data from Kafka Topic ( I will use the getKafka processor) 2. Apply business logic to the data fetched from Kafka like few calculations (Here I would need which processor to use to modify the flow files on the flow) 3. Write the data into HDFS. (putHDFS processor) Is there any processor which I can use to make the changes to the flow files? Thanks. ________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer. ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
