Hi All (I'm back from vacation today): For 3.0, my thoughts are that the 'core' should only include infrastructure used to implement actual appenders and layouts (and filters and so on.)
My experience with the products that use Log4j 1 and 2 at work is: - Use the Console appender by default. - No consistent usage of at least one kind of File appender (where the layout is always a pattern.) - One or more JDBC Appender - One or more JMS Appender - A few _Custom_ appenders The JDBC and JMS Appenders clearly (to me) should live in their own module. Since there is no 'always use this kind of file appender', file appenders should live in one or more of their own modules. Maybe Matt's new configuration code is flexible enough to be in a separate module, I do not know yet. Certainly each kind of layout should live in it's own module which is already almost in place for 3,0. Gary On Tue, Mar 3, 2020 at 4:12 AM Volkan Yazıcı <[email protected]> wrote: > Hello, > > Given I am done with my JsonTemplateLayout PR and waiting for the > review (I am looking at you Ralph), I have started working on removing > Jackson dependency. This would allow the layout to be included in the > core. A major obstacle I encounter is the implementation of the > following two configuration knobs: maxStringLength (the maximum > allowed length for a string field in JSON) and maxLength (the maximum > allowed length of the resultant JSON string length). It is pretty > difficult to implement them without getting a notable performance hit. > I am considering leaving these two features out. This opens up the > following set of questions: Who is responsible for > truncating/discarding excessive (malicious?) log payloads? Is it > possible to handle such payloads earlier in the chain by means of a > built-in filter? > > Note that filtering out excessive payloads after layout kicks in is > not a good idea. Consider the following: An excessive LogEvent makes > its way up to the layout and causes the internal StringBuilder to grow > excessively. Later on, the output of the layout is discarded by, say, > the appender or some other filter. Now... Who is gonna trim the > internal char[] used by the StringBuilder? The filtering should happen > before the explosive reaches to the layout. It is not impossible to > implement all these measures in the layout, but pretty misplaced and > technically challenging to get it right efficiently. > > How one would determine excessive LogEvents is also a challenge on its > own. The notion of excessive very well depends on the layout. That is, > for a particular LogEvent, the output of a compressed binary layout > can be considered okay while its JSON representation might appear > excessive. But I've just aforementioned filtering should kick in > before layout. Sort of a chicken and egg problem. > > For the records, I've discussed this issue with the colleagues at > work, where we use LogstashLayout with its excessive payload > prevention mechanisms in tens of thousands of services, they really > would like to have either it or a replacement, FYI. > > I am pretty much lost at this stage. Any thoughts? > > Kinds regards. >
