Hi Matt Thanks for your reply, I will certainly take on board everything you and Andy advise and I will look at classes you mentioned and I will also read the links provided. I ran the TestXMLReader as a junit in Eclipse, a sample of the the console output is : 20:15:01.675 [pool-1-thread-1] DEBUG org.apache.nifi.schema.access.AvroSchemaTextStrategy - For {path=target, filename=253762304418.mockFlowFile, xml.stream.is.array=true, uuid=34fb0980-8fc3-4c41-b4f5-3078d26b6f67} found schema text { "namespace": "nifi", "name": "test", "type": "record", "fields": [ { "name": "ID", "type": "string" }, { "name": "NAME", "type": "string" }, { "name": "AGE", "type": "int" }, { "name": "COUNTRY", "type": "string" } ] }
Anyway, thanks again I have something to go on now. Dave On Tuesday, 21 April 2020, 17:47:21 BST, Andy LoPresto <alopre...@apache.org> wrote: Hi Dave, The underlying internal “record format” is not JSON. Avro [1] is used to describe schemas across all record formats, but the internal data storage is NiFi specific. You may be interested in these articles by Mark Payne and Bryan Bende [2][3][4] and the potential use of the ScriptedReader [5] or ScriptedRecordSetWriter [6] to prototype your needed conversions. [1] https://avro.apache.org/ <https://avro.apache.org/> [2] https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi <https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi> [3] https://blogs.apache.org/nifi/entry/real-time-sql-on-event <https://blogs.apache.org/nifi/entry/real-time-sql-on-event> [4] https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries <https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries> [5] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedReader/index.html [6] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Apr 21, 2020, at 6:01 AM, DAVID SMITH <davidrsm...@btinternet.com.INVALID> > wrote: > > Hi > I want to use the ConvertRecord Processor with it's underlying Record Readers > and Writers to convert files from XML or JSON to a bespoke format and > probably vice versa.I have looked at the Readers/Writers currently provided > and decided that I can use the XML/JSON ones provided but I will need to > write something for the bespoke format. So I started looking at the current > source code for the Readers/Writers to see how they work and what I would > need to do. When running the unit tests on the XMLReader I notice on the > console that the output is in JSON format.My question is, is JSON the common > format that all records are converted to and from? > Also is there any specific documentation on writing Reader/Writers, I have > only found the developers guide? > Many thanksDave >