One thing about its limitations had to do with timing, the record-aware stuff happened after QDT. Would be great to have QDT use a record writer, then depending on the writer you could choose your schema output strategy as Koji outlined.
I'm not sure if there is a JIRA for this or not (or any of the other earlier processors like ExecuteSQL), but we can certainly go after record-aware versions of them to get all the goodness therein. Regards, Matt Sent from my iPhone > On Sep 11, 2017, at 9:05 PM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > > Hi Uwe, > > I had a similar expectation when I was using QueryDatabaseTable or any > other processor creating Avro FlowFile which has its schema embedded, > combining new record reader/writer controllers. > > Now, NiFi has "Inherit Record Schema" option as "Schema Access > Strategy" of RecordWriter, already merged in master branch. > https://issues.apache.org/jira/browse/NIFI-3921 > > I was able to reuse the Avro schema at subsequent flow using "Inherit > Record Schema", it's really useful. You can construct a flow like > below: > > - QueryDatabaseTable > - outputs FlowFile with Avro schema embedded > - ConvertRecord > - AvroReader: > - "Schema Access Strategy" = "Use Embedded Avro Schema" > - CSVRecordSetWriter: > - "Schema Access Strategy" = "Inherit Record Schema" > - "Schema Write Strategy" = "Set 'avro.schema' Attribute" > > This way, you don't have to have the schema in registry, and result > CSV FlowFile has 'avro.schema' attribute inheriting the one created by > QueryDatabaseTable. > > Hope this helps. > > Thanks, > Koji > >> On Tue, Sep 12, 2017 at 5:02 AM, Uwe Geercken <uwe.geerc...@web.de> wrote: >> Hello, >> >> I was wondering why if the QueryDatabaseTable processor creates internally >> an Avro schema, why is this schema not available as an attribute or saved to >> the registry? >> >> If it would, then one could reuse the schema. E.g. if I use the >> ConvertRecord processor and I specify an AvroReader as RecordReader, then >> this Reader will take the schema from the flowfile the QueryDatabaseTable >> processor creates. But the RecordWriter in the ConvertRecord - in my example >> a CSVRecordSetWriter requires the schema as an attribute or as a reference >> to the schema registry. >> >> I can see there is an ExtractAvroSchema processor but I don'see there is a >> way of combining the metadata into e.g. the ConvertRecord processor. >> >> Any help or ideas? >> >> Rgds, >> >> Uwe