Yes, I understand, I was just wondering if individual operators could define the appenders they potentially need at compile time and then the operator callbacks could simply check the desired runtime condition and add the appropriate appender.
Or are we saying there are scenarios where we absolutely cannot create the appender beforehand ? So broadly speaking, my question is whether the combination of providing predefined appenders and the PropertyConfigurator capabilities meets the need. Ram On Mon, Apr 10, 2017 at 2:18 PM, Sergey Golovko <[email protected]> wrote: > Ram, > > Really the new appender class must extend the abstract class > AppenderSkeleton. And in order to add a new appender programmatically in > Java, some code in Apex should call the following log4j method: > > org.apache.log4j.Logger.getRootLogger().addAppender(Appender newAppender) > > The general idea of my proposal is "*based on some runtime parameter(s) to > provide ability to create an appender instance via reflection and add it to > the list of active log4j appenders*". > > Thanks, > Sergey > > > On Mon, Apr 10, 2017 at 2:04 PM, Vlad Rozov <[email protected]> > wrote: > > > It will require application recompilation and repackaging. The proposed > > functionality is for dev-ops to be able to route application logging to a > > preferred destination without recompiling applications. It is run-time > > configuration vs compile time hardcoded appender. > > > > Thank you, > > > > Vlad > > > > On 4/10/17 11:23, Munagala Ramanath wrote: > > > >> You can do it in a trivial derived class without changing the base > class. > >> > >> Ram > >> > >> On Mon, Apr 10, 2017 at 11:19 AM, Vlad Rozov <[email protected]> > >> wrote: > >> > >> Does not the proposal to use Logger.addAppender() requires modifications > >>> to used operators code? > >>> > >>> Thank you, > >>> > >>> Vlad > >>> > >>> On 4/10/17 10:58, Munagala Ramanath wrote: > >>> > >>> People can currently do this by simply implementing the Appender > >>>> interface > >>>> and adding it > >>>> with Logger.addAppender() in the setup method. Why do we need > something > >>>> more elaborate ? > >>>> > >>>> Ram > >>>> > >>>> On Mon, Apr 10, 2017 at 10:30 AM, Sergey Golovko < > >>>> [email protected]> > >>>> wrote: > >>>> > >>>> The configuration of a log4j appender via log4j configuration file is > a > >>>> > >>>>> static configuration that cannot be disabled/enabled and managed > >>>>> dynamically by an application designer. The programmatic approach > will > >>>>> allow an application designer to specify which of the available > log4j > >>>>> appenders should be used for the specific application. > >>>>> > >>>>> It is not necessary Apex should use the predefined log4j appenders > >>>>> only. > >>>>> The log4j events contain useful but the very limited number of > >>>>> properties > >>>>> which values can be printed into output log4j sources. But based on > the > >>>>> knowledge of the software product workflow, the custom defined log4j > >>>>> appender can extend a list of predefined output log events properties > >>>>> and, > >>>>> for instance for Apex, return: node, user name, application name, > >>>>> application id, container id, operator name, etc. > >>>>> > >>>>> Also the output log events that are generated by a custom defined > log4j > >>>>> appender can be stored and indexed by any type of a full text search > >>>>> database. It will allow the customers and developers to simplify > >>>>> collection > >>>>> of log events statistics and searching/filtering of specific events > for > >>>>> debugging and investigation. > >>>>> > >>>>> Thanks, > >>>>> Sergey > >>>>> > >>>>> > >>>>> On Mon, Apr 10, 2017 at 6:34 AM, Vlad Rozov <[email protected] > > > >>>>> wrote: > >>>>> > >>>>> +1 Apex engine does not own log4j config file - it is provided either > >>>>> by > >>>>> > >>>>>> Hadoop or an application. Hadoop log4j config does not necessarily > >>>>>> meet > >>>>>> application logging requirements, but if log4j is provided by an > >>>>>> application designer, who can only specify what to log, it may not > >>>>>> meet > >>>>>> operations requirements. Dev-ops should have an ability to specify > >>>>>> where > >>>>>> > >>>>>> to > >>>>> > >>>>> log depending on the available infrastructure at run-time. > >>>>>> > >>>>>> It will be good to have an ability not only specify extra log4j > >>>>>> appenders > >>>>>> at lunch time, but also at run-time, the same way how log4j logger > >>>>>> levels > >>>>>> may be changed. > >>>>>> > >>>>>> Thank you, > >>>>>> > >>>>>> Vlad > >>>>>> > >>>>>> On 4/9/17 23:14, Priyanka Gugale wrote: > >>>>>> > >>>>>> We can always write a custom appender and add it by changing root > >>>>>> appender > >>>>>> in log4j config file. Can you explain how adding appender > >>>>>> grammatically > >>>>>> > >>>>>>> would help? > >>>>>>> > >>>>>>> -Priyanka > >>>>>>> > >>>>>>> On Sun, Apr 9, 2017 at 11:50 AM, Sanjay Pujare < > >>>>>>> [email protected] > >>>>>>> wrote: > >>>>>>> > >>>>>>> Please give some examples and/or use cases of this programmatic > log4j > >>>>>>> > >>>>>>> appender. > >>>>>>>> > >>>>>>>> On Fri, Apr 7, 2017 at 8:40 PM, Sergey Golovko < > >>>>>>>> [email protected] > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi All, > >>>>>>>> > >>>>>>>> I'd like to add supporting of a custom defined log4j appender that > >>>>>>>>> can > >>>>>>>>> be > >>>>>>>>> added to Apex Application Master and Containers and be > configurable > >>>>>>>>> programmatically. > >>>>>>>>> > >>>>>>>>> Sometimes it is not trivial to control log4j configuration via > >>>>>>>>> log4j > >>>>>>>>> properties. And I think the having of the approach to add a log4j > >>>>>>>>> > >>>>>>>>> appender > >>>>>>>>> > >>>>>>>> programmatically will allow the customers and developers to plugin > >>>>>>>> their > >>>>>>>> > >>>>>>> own custom defined log4j appenders and be much flexible for > streaming > >>>>>> > >>>>>>> and > >>>>>>>>> collection of Apex log events. > >>>>>>>>> > >>>>>>>>> I assume to provide generic approach for definition of the > >>>>>>>>> > >>>>>>>>> programmatic > >>>>>>>> > >>>>>>> log4j appender and to pass all configuration parameters including a > >>>>>> > >>>>>>> name > >>>>>>>> > >>>>>>> of > >>>>>> > >>>>>>> the Java class with implementation of the log4j appender via system > >>>>>>>> > >>>>>>>>> and/or > >>>>>>>>> > >>>>>>>> command line properties. > >>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Sergey > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>> > >> > > > -- _______________________________________________________ Munagala V. Ramanath Software Engineer E: [email protected] | M: (408) 331-5034 | Twitter: @UnknownRam www.datatorrent.com | apex.apache.org
