Awesome! Thanks Mark, I didn't think multiple records flow file, it makes a lot of sense your suggestions.
Thanks, Juan On Tue, 21 Nov 2017 at 11:29 Mark Payne <[email protected]> wrote: > Juan, > > Working with attributes is a little bit trickier in this case because the > record-oriented processors > are generally intended to work on 'streams' of records. I.e., each > FlowFile can have 1 record or > it can be made up of thousands or more records. So if you have many > records in a FlowFile, it's > a little more difficult to extract a field value into an attribute. > > So what we do is a little bit different here. We need to group together > 'like records' into separate > FlowFiles. For example, if we have 5 records in a FlowFile, and > /person/name is 'Juan' for the first 2 > and is 'Mark' for the last 3, then we can use PartitionRecord to separate > our FlowFile into two separate > FlowFiles, the first containing those records where /person/name is 'Juan' > and the second FlowFile > containing those records where /person/name is 'Mark'. Once we have done, > it now makes more sense > to extract the name 'Juan' and the name 'Mark' into FlowFile attributes. > And that's just what PartitionRecord > does. Each outbound FlowFile will have an attribute that is equal to the > value of the field specified. > > So for example, if you add a single property to PartitionRecord named > 'person' with a value of /person/name > and then send in that example FlowFile mentioned above, then you'd get out > 2 FlowFiles. The first would > have an attribute 'person' (the name of the property you added is the name > of the attribute) with a value of > 'Juan' and the second would have an attribute 'person' with a value of > 'Mark'. > > Also of note - the PartitionRecord processor takes a Record Reader and > Writer, so this allows you to read > the data in as JSON and then write it out as Avro. Essentially, it allows > for an implicit record conversion, so > you will no longer need your ConvertRecord processor in your flow. > > The documentation for PartitionRecord can be found here [1]. If you click > the 'Additional Details...' link at the > end of the first paragraph, it will provide quite a bit more documentation > with examples. Hopefully this all > makes sense, but if you have further questions, I am happy to elaborate if > there is something that's not clear. > > Thanks! > -Mark > > [1] > http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.PartitionRecord/index.html > > > > > On Nov 21, 2017, at 7:10 AM, Juan Pablo Gardella < > [email protected]> wrote: > > > > Hello all, > > > > I'm working with Nifi records. Currently I have a json converted to Avro > object using ConvertRecord processor. I would like to know if it is > possible to use something similar to UpdateAttribute to add an attribute > which evaluates a record expression. Something similar to: > > > > attribute1 -> ${/person/name} > > > > Juan > >
