From comments from Scott Deboy I believe Chainsaw also supports Log4j 2 but not 
in an official release.

Ralph

> On Jul 17, 2017, at 12:13 PM, Mikael Ståldal <mi...@apache.org> wrote:
> 
> Hi.
> 
> (Moving this discussion to logging dev mailing list.)
> 
> Have you tried to use:
> <JsonLayout properties="true" eventEol="true" compact="true"/>
> 
> Then each log event will be terminated by end-of-line (\r\n).
> 
> I think it would be easy to implement 0-byte terminated log events in 
> JsonLayout, and that would make sense since we have implemented support for 
> that in GelfLayout. Created a JIRA issue for it: 
> https://issues.apache.org/jira/browse/LOG4J2-1981
> 
> As for other tools, the only receivers for Log4j 2 SerializedLayout we know 
> are Log4j's own SocketServer and Lilith. Chainsaw currently only support 
> Log4j 1 SerializedLayout. Log4j's own SocketServer support JsonLayout and 
> XmlLayout as well, and we are changing it to use JsonLayout by default.
> 
> 
> On 2017-07-17 15:01, Joern Huxhorn wrote:
>> Hi Mikael,
>> I've just taken a look at the JsonLayout and see no way to parse its
>> output event by event.
>> It's semi-fine for files because then it will result in a well-formed
>> array at the root level that can be read in one big swoop. It isn't
>> really ideal for that use case either since it means that the whole
>> array containing all events needs to be read into memory in one piece,
>> causing OOM in case of huge files.
>> But in Lilith, I really need to process events one by one. This is
>> absolutely crucial.
>> The only way to achieve this would essentially require me to
>> re-implement a JSON parser that can cope with data like this.
>> I implemented a similar kludge to support reading of log4j xml files
>> (see
>> https://github.com/huxi/lilith/blob/master/log4j/log4j-xml/src/main/java/de/huxhorn/lilith/log4j/xml/Log4jImportCallable.java
>> ) but this was way easier since i could simply search for
>> "</log4j:event>" to find the position at which I could split the stream
>> of events into a valid XML document for every single event . This was
>> still kind of nasty but doable, especially since processing an "offline"
>> XML file doesn't have the same performance restriction as handling of
>> live events.
>> I wouldn't have the luxury of such a unique split signal in JSON, though.
>> I would need to detect the "}" at the level of the root array which
>> would not simply involve counting of opening and closing brackets but
>> also ignoring of brackets inside of strings. This would boil down to
>> implementing a custom JSON reader and I won't be doing this.
>> This isn't just me being lazy either. Such an implementation wouldn't be
>> resource (CPU/memory) friendly anyway.
>> In my own JSON receiver using my own JSON format there are two ways to
>> send those events:
>> - write an int containing the amount of bytes representing the event,
>> then the bytes of the event. This type of events also supports compression.
>> - write the bytes of the event, followed by a 0-byte. This works fine
>> and JavaScript/ActionScript is able to produce events like that while
>> they are unable (or were unable 7 years ago) to count the bytes on their
>> own. This type of events only supports plain text like JSON or XML and
>> no compression since compressed events could contain the 0-byte while
>> XML and JSON both won't allow it.
>> Both are essentially "binary" formats because of either the raw ints or
>> the 0-bytes.
>> I'm not sure why you deprecate SerializedLayout since the security
>> issues arise only while deserializing the events, not while serializing
>> them. Is this just meant to educate the user about the issue?
>> I fixed the remote code execution exploit by implementing
>> https://github.com/huxi/lilith/blob/master/lilith-engine/src/main/java/de/huxhorn/lilith/engine/impl/eventproducer/WhitelistObjectInputStream.java
>> This still doesn't fix DoS scenarios (endless loops or OOM in case of
>> malicious data) but is close enough for me, mainly because fixing the
>> DoS issues would really have to be handled by Java, not by user code.
>> Manually re-implementing deserialization would likely be fragile and
>> error prone, possibly even introducing other security issues on its own.
>> I'd be interested how other tools are tackling this issue. Is Chainsaw
>> officially dead or are they implementing receivers for event streams
>> like this?
>> I'm totally fine with implementing a new receiver as a replacement for
>> log4j2 SerializedLayout (even though I won't be able to get rid of the
>> Serializable receiver anyway) but the current JsonLayout just isn't up
>> to the job at the moment.
>> Feel free to share this mail with your fellow developers. I'm open for
>> suggestions.
>> I hope this doesn't come across in a harsh way. It isn't meant to be.
>> Cheers,
>> Jörn.
> 
> 


Reply via email to