A typical pattern/workaround for this situation was to copy e.g. the json _in full_ into an attribute, leaving the payload in a binary format. But, as you can imagine, it's not ideal as FlowFile memory and disk pressure will be raised significantly and duplicate that of an existing content repo.
Andrew On 10/13/15, 9:21 AM, "Joe Witt" <[email protected]> wrote: >Hello > >Is the only reason for converting from AVRO or whatever to JSON so >that you can extract attributes? > >I recommend not converting the data simply so that you can do that. I >recommend building processes to extract attributes from the raw. I >believe we have JIRA's targeted for the next release to do this for >AVRO just like JSON. If you have other custom formats in mind i >recommend building 'ExtractXYZAttributes'. > >There is no mechanism in play today where we convert from format A to >B and then in the resulting B we keep the original A hanging around >that object. You can do this of course by making archive/container >formats to hold both but this is also not recommended. > >Does this make sense? > >Thanks >Joe > >On Tue, Oct 13, 2015 at 9:06 AM, Oleg Zhurakousky ><[email protected]> wrote: >> Sorry, I meant to say that you have to enrich the original file with a >> correlation attribute, otherwise there is nothing to correlate on. >> I am not sure if NiFi has any implementation of ContentEnricher (EIP), >> perhaps UpdateAttribute will do the trick. >> >> Oleg >> >>> On Oct 13, 2015, at 8:21 AM, yejug <[email protected]> wrote: >>> >>> Hi Oleg >>> >>> THanks for response, may be I missing something (I cannot find you image >>> =)), but you suggestion doesn;t appropriate. >>> >>> There into MergeContent processor brings two types of flowFiles : >>> 1) one is flow file with original content (AVRO) but without populated >>> "correlation" attribute, directly from GetKafka >>> 2) and second type of flow file with parsed content (JSON) and populated >>> "correlation" attribute >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-nifi-developer-list.39713.n7.nabble.com/Ingest-Original-data-from-External-system-by-data-s-dependent-condition-tp3093p3096.html >>> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com. >>> >> >
