The new config code is all extremely modular. I put it all in the plugins module for now, but it can certainly end up as a stand-alone module. FWIW, I’d love to go with this idea as I’ve wanted something like it since I started working on OSGi integration long ago.
On Tue, Mar 3, 2020 at 07:52 Gary Gregory <garydgreg...@gmail.com> wrote: > Hi All (I'm back from vacation today): > > For 3.0, my thoughts are that the 'core' should only include infrastructure > used to implement actual appenders and layouts (and filters and so on.) > > My experience with the products that use Log4j 1 and 2 at work is: > - Use the Console appender by default. > - No consistent usage of at least one kind of File appender (where the > layout is always a pattern.) > - One or more JDBC Appender > - One or more JMS Appender > - A few _Custom_ appenders > > The JDBC and JMS Appenders clearly (to me) should live in their own module. > Since there is no 'always use this kind of file appender', file appenders > should live in one or more of their own modules. > > Maybe Matt's new configuration code is flexible enough to be in a separate > module, I do not know yet. > > Certainly each kind of layout should live in it's own module which is > already almost in place for 3,0. > > Gary > > On Tue, Mar 3, 2020 at 4:12 AM Volkan Yazıcı <volkan.yaz...@gmail.com> > wrote: > > > Hello, > > > > Given I am done with my JsonTemplateLayout PR and waiting for the > > review (I am looking at you Ralph), I have started working on removing > > Jackson dependency. This would allow the layout to be included in the > > core. A major obstacle I encounter is the implementation of the > > following two configuration knobs: maxStringLength (the maximum > > allowed length for a string field in JSON) and maxLength (the maximum > > allowed length of the resultant JSON string length). It is pretty > > difficult to implement them without getting a notable performance hit. > > I am considering leaving these two features out. This opens up the > > following set of questions: Who is responsible for > > truncating/discarding excessive (malicious?) log payloads? Is it > > possible to handle such payloads earlier in the chain by means of a > > built-in filter? > > > > Note that filtering out excessive payloads after layout kicks in is > > not a good idea. Consider the following: An excessive LogEvent makes > > its way up to the layout and causes the internal StringBuilder to grow > > excessively. Later on, the output of the layout is discarded by, say, > > the appender or some other filter. Now... Who is gonna trim the > > internal char[] used by the StringBuilder? The filtering should happen > > before the explosive reaches to the layout. It is not impossible to > > implement all these measures in the layout, but pretty misplaced and > > technically challenging to get it right efficiently. > > > > How one would determine excessive LogEvents is also a challenge on its > > own. The notion of excessive very well depends on the layout. That is, > > for a particular LogEvent, the output of a compressed binary layout > > can be considered okay while its JSON representation might appear > > excessive. But I've just aforementioned filtering should kick in > > before layout. Sort of a chicken and egg problem. > > > > For the records, I've discussed this issue with the colleagues at > > work, where we use LogstashLayout with its excessive payload > > prevention mechanisms in tens of thousands of services, they really > > would like to have either it or a replacement, FYI. > > > > I am pretty much lost at this stage. Any thoughts? > > > > Kinds regards. > > > -- Matt Sicker <boa...@gmail.com>
