I've looked into source code and I see that I do not have to use a single character but can use any string which then is encoded as a byte array by NiFi. I can generate UUID and use it as a safe demarcator.
On Wed, Dec 26, 2018 at 4:47 PM Boris Tyukin <[email protected]> wrote: > Hello, > > I would like to batch Kafka messages using demarcator property but I am > not sure which one to use. In my case I get JSON messages and some message > values can contain special characters and even newline delimiters in both > Windows and Unix flavors (data is coming from GoldenGate, which streams > data from an Oracle database, that is populated by a third-party > application, that can run on various platforms). > > I understand I can use ctrl-enter or shift-enter but not sure it will be > totally bulletproof in my case and I cannot test all the possible values > since I do not know what I might get. > > Is there 100% safe demarcator? > > ConsumeKafkaRecord is not an option for me before you propose it as I > cannot define a schema for messages - I need to work with any json message > and cannot create a schema for it upfront. > > Thanks! > Boris >
