Marcus, Try this as a potential work around:
https://www.elastic.co/guide/en/beats/filebeat/master/kafka-output.html You should be able to aggregate your log data into a Kafka topic and then consume that with NiFi. On Fri, Mar 15, 2019 at 10:37 AM Joe Witt <[email protected]> wrote: > Hello > > I am not very familiar with beats or this processor specifically but it > does appear to have some beats specific framing that it does by looking at > the processor code. Json based framing appears supported as well but it > isn't clear that it would be or is intended to be valid JSON in a combined > sense. With that in mind it isn't clear you can expect to use the JSON > based processors/readers after this. > > Hopefully someone with more experience in the processor and its intended > use can comment soon. > > Thanks > > On Fri, Mar 15, 2019 at 10:07 AM Ling, Marcus <[email protected]> > wrote: > >> Hi, >> >> >> >> Anybody using the ListenBeats processor to pull auditing info from centos >> (audit/file beats) -> NIFI? >> >> >> >> Some of the flowfiles have corrupted JSON. (Noticed when using the >> EvaluateJsonpath processor.) >> >> >> >> I thought beats-> listenbeats was already json format. >> >> >> >> The corruption is where >1 audit event seems to get munged into 1 record, >> this no longer json format! help! >> >> >> >> Thanks >> >> >> >> Marcus >> >> >> >> BAE Systems will collect and process information about you that may be >> subject to data protection laws. For more information about how we use and >> disclose your personal information, how we protect your information, our >> legal basis to use your information, your rights and who you can contact, >> please refer to the relevant sections of our Privacy note at >> www.baesystems.com/en/cybersecurity/privacy >> >> >> Please consider the environment before printing this email. This message >> should be regarded as confidential. If you have received this email in >> error please notify the sender and destroy it immediately. Statements of >> intent shall only become binding when confirmed in hard copy by an >> authorised signatory. The contents of this email may relate to dealings >> with other companies under the control of BAE Systems PLC, details of which >> can be found at http://www.baesystems.com/Businesses/index.htm. >> >
