As we are already doing dynamic log level changes and are talking about dynamically adding appenders, does it make sense and technically feasible to apply a delta log4j configuration dynamically (which can include log levels + appeneders + ?) on top of the static configuration provided by the system.
On Mon, Apr 10, 2017 at 2:18 PM, Sergey Golovko <ser...@datatorrent.com> wrote: > Ram, > > Really the new appender class must extend the abstract class > AppenderSkeleton. And in order to add a new appender programmatically in > Java, some code in Apex should call the following log4j method: > > org.apache.log4j.Logger.getRootLogger().addAppender(Appender newAppender) > > The general idea of my proposal is "*based on some runtime parameter(s) to > provide ability to create an appender instance via reflection and add it to > the list of active log4j appenders*". > > Thanks, > Sergey > > > On Mon, Apr 10, 2017 at 2:04 PM, Vlad Rozov <v.ro...@datatorrent.com> > wrote: > > > It will require application recompilation and repackaging. The proposed > > functionality is for dev-ops to be able to route application logging to a > > preferred destination without recompiling applications. It is run-time > > configuration vs compile time hardcoded appender. > > > > Thank you, > > > > Vlad > > > > On 4/10/17 11:23, Munagala Ramanath wrote: > > > >> You can do it in a trivial derived class without changing the base > class. > >> > >> Ram > >> > >> On Mon, Apr 10, 2017 at 11:19 AM, Vlad Rozov <v.ro...@datatorrent.com> > >> wrote: > >> > >> Does not the proposal to use Logger.addAppender() requires modifications > >>> to used operators code? > >>> > >>> Thank you, > >>> > >>> Vlad > >>> > >>> On 4/10/17 10:58, Munagala Ramanath wrote: > >>> > >>> People can currently do this by simply implementing the Appender > >>>> interface > >>>> and adding it > >>>> with Logger.addAppender() in the setup method. Why do we need > something > >>>> more elaborate ? > >>>> > >>>> Ram > >>>> > >>>> On Mon, Apr 10, 2017 at 10:30 AM, Sergey Golovko < > >>>> ser...@datatorrent.com> > >>>> wrote: > >>>> > >>>> The configuration of a log4j appender via log4j configuration file is > a > >>>> > >>>>> static configuration that cannot be disabled/enabled and managed > >>>>> dynamically by an application designer. The programmatic approach > will > >>>>> allow an application designer to specify which of the available > log4j > >>>>> appenders should be used for the specific application. > >>>>> > >>>>> It is not necessary Apex should use the predefined log4j appenders > >>>>> only. > >>>>> The log4j events contain useful but the very limited number of > >>>>> properties > >>>>> which values can be printed into output log4j sources. But based on > the > >>>>> knowledge of the software product workflow, the custom defined log4j > >>>>> appender can extend a list of predefined output log events properties > >>>>> and, > >>>>> for instance for Apex, return: node, user name, application name, > >>>>> application id, container id, operator name, etc. > >>>>> > >>>>> Also the output log events that are generated by a custom defined > log4j > >>>>> appender can be stored and indexed by any type of a full text search > >>>>> database. It will allow the customers and developers to simplify > >>>>> collection > >>>>> of log events statistics and searching/filtering of specific events > for > >>>>> debugging and investigation. > >>>>> > >>>>> Thanks, > >>>>> Sergey > >>>>> > >>>>> > >>>>> On Mon, Apr 10, 2017 at 6:34 AM, Vlad Rozov <v.ro...@datatorrent.com > > > >>>>> wrote: > >>>>> > >>>>> +1 Apex engine does not own log4j config file - it is provided either > >>>>> by > >>>>> > >>>>>> Hadoop or an application. Hadoop log4j config does not necessarily > >>>>>> meet > >>>>>> application logging requirements, but if log4j is provided by an > >>>>>> application designer, who can only specify what to log, it may not > >>>>>> meet > >>>>>> operations requirements. Dev-ops should have an ability to specify > >>>>>> where > >>>>>> > >>>>>> to > >>>>> > >>>>> log depending on the available infrastructure at run-time. > >>>>>> > >>>>>> It will be good to have an ability not only specify extra log4j > >>>>>> appenders > >>>>>> at lunch time, but also at run-time, the same way how log4j logger > >>>>>> levels > >>>>>> may be changed. > >>>>>> > >>>>>> Thank you, > >>>>>> > >>>>>> Vlad > >>>>>> > >>>>>> On 4/9/17 23:14, Priyanka Gugale wrote: > >>>>>> > >>>>>> We can always write a custom appender and add it by changing root > >>>>>> appender > >>>>>> in log4j config file. Can you explain how adding appender > >>>>>> grammatically > >>>>>> > >>>>>>> would help? > >>>>>>> > >>>>>>> -Priyanka > >>>>>>> > >>>>>>> On Sun, Apr 9, 2017 at 11:50 AM, Sanjay Pujare < > >>>>>>> san...@datatorrent.com > >>>>>>> wrote: > >>>>>>> > >>>>>>> Please give some examples and/or use cases of this programmatic > log4j > >>>>>>> > >>>>>>> appender. > >>>>>>>> > >>>>>>>> On Fri, Apr 7, 2017 at 8:40 PM, Sergey Golovko < > >>>>>>>> ser...@datatorrent.com > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi All, > >>>>>>>> > >>>>>>>> I'd like to add supporting of a custom defined log4j appender that > >>>>>>>>> can > >>>>>>>>> be > >>>>>>>>> added to Apex Application Master and Containers and be > configurable > >>>>>>>>> programmatically. > >>>>>>>>> > >>>>>>>>> Sometimes it is not trivial to control log4j configuration via > >>>>>>>>> log4j > >>>>>>>>> properties. And I think the having of the approach to add a log4j > >>>>>>>>> > >>>>>>>>> appender > >>>>>>>>> > >>>>>>>> programmatically will allow the customers and developers to plugin > >>>>>>>> their > >>>>>>>> > >>>>>>> own custom defined log4j appenders and be much flexible for > streaming > >>>>>> > >>>>>>> and > >>>>>>>>> collection of Apex log events. > >>>>>>>>> > >>>>>>>>> I assume to provide generic approach for definition of the > >>>>>>>>> > >>>>>>>>> programmatic > >>>>>>>> > >>>>>>> log4j appender and to pass all configuration parameters including a > >>>>>> > >>>>>>> name > >>>>>>>> > >>>>>>> of > >>>>>> > >>>>>>> the Java class with implementation of the log4j appender via system > >>>>>>>> > >>>>>>>>> and/or > >>>>>>>>> > >>>>>>>> command line properties. > >>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Sergey > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>> > >> > > >
