FYI, in case someone wants to work it, the ticket for extracting from Avro is: https://issues.apache.org/jira/browse/NIFI-962
On Tue, Oct 13, 2015 at 9:29 AM, Andrew Grande <[email protected]> wrote: > A typical pattern/workaround for this situation was to copy e.g. the json > _in full_ into an attribute, leaving the payload in a binary format. But, > as you can imagine, it's not ideal as FlowFile memory and disk pressure > will be raised significantly and duplicate that of an existing content repo. > > Andrew > > > > > On 10/13/15, 9:21 AM, "Joe Witt" <[email protected]> wrote: > > >Hello > > > >Is the only reason for converting from AVRO or whatever to JSON so > >that you can extract attributes? > > > >I recommend not converting the data simply so that you can do that. I > >recommend building processes to extract attributes from the raw. I > >believe we have JIRA's targeted for the next release to do this for > >AVRO just like JSON. If you have other custom formats in mind i > >recommend building 'ExtractXYZAttributes'. > > > >There is no mechanism in play today where we convert from format A to > >B and then in the resulting B we keep the original A hanging around > >that object. You can do this of course by making archive/container > >formats to hold both but this is also not recommended. > > > >Does this make sense? > > > >Thanks > >Joe > > > >On Tue, Oct 13, 2015 at 9:06 AM, Oleg Zhurakousky > ><[email protected]> wrote: > >> Sorry, I meant to say that you have to enrich the original file with a > correlation attribute, otherwise there is nothing to correlate on. > >> I am not sure if NiFi has any implementation of ContentEnricher (EIP), > perhaps UpdateAttribute will do the trick. > >> > >> Oleg > >> > >>> On Oct 13, 2015, at 8:21 AM, yejug <[email protected]> wrote: > >>> > >>> Hi Oleg > >>> > >>> THanks for response, may be I missing something (I cannot find you > image > >>> =)), but you suggestion doesn;t appropriate. > >>> > >>> There into MergeContent processor brings two types of flowFiles : > >>> 1) one is flow file with original content (AVRO) but without populated > >>> "correlation" attribute, directly from GetKafka > >>> 2) and second type of flow file with parsed content (JSON) and > populated > >>> "correlation" attribute > >>> > >>> > >>> > >>> > >>> -- > >>> View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/Ingest-Original-data-from-External-system-by-data-s-dependent-condition-tp3093p3096.html > >>> Sent from the Apache NiFi Developer List mailing list archive at > Nabble.com. > >>> > >> > > >
