Prabhu, I don’t think NiFi is the right tool to do the complete task (fine for CSV parsing & splitting), but you should look at writing a script or using Spark, etc. to do the data operations. Any NiFi-native solution will be brittle and won’t scale well.
That said, I did answer how you could do this in NiFi only on your StackOverflow question [1]. However, you could certainly use a combination of SplitText processors to get the proper data into individual flowfiles (i.e. all Mahi rows in one, all Andy rows in another). Once you have a record that looks like: Andy,1,85 Andy,1,95 you can use ExtractText with regular expressions to get 85 and 95 into attributes marks.1 and marks.2 (a good example of where scaling will break down -- doing this with 2 rows is easy; doing this with 100k is ridiculous). You can then use UpdateAttribute with the Expression Language to calculate the average of those two attributes (convert toNumber() first) and populate a third attribute marks.average (either through chaining plus() and divide() functions or with the math advanced operation <https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#math> (uses Java Reflection)). Once you have the desired result in an attribute, use ReplaceText to update the flowfile content, and MergeContent to merge the individual flowfiles back into a single instance. [1] http://stackoverflow.com/a/42381999/70465 Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 21, 2017, at 5:20 AM, prabhu Mahendran <[email protected]> wrote: > > i need to find average of two values in seperate lines. > > My Csv file loo like this., > > Name,ID,Marks > Mahi,1,90 > Mahi,1,90 > > > Andy,2,100 > Andy,2,100 > Now i need to store that average of 2 marks in database. "Average" column > should add two marks and divide with 2 and store that result in SQL query > > Table: > > Name,ID,Average > Mahi,2,90 > Andy,2,100 > Is this possible to find average of two value's in seperate rows using NiFi? >
signature.asc
Description: Message signed with OpenPGP using GPGMail
