I agree it would be very nice to use only slf4j interfaces for the implementation. But unfortunately the interface Appender belongs to org.apache.log4j package.
"SLF4J is only a facade, meaning that it does not provide a complete logging solution. Operations such as configuring appenders or setting logging levels cannot be performed with SLF4J. Thus, at some point in time, any non-trivial application will need to directly invoke the underlying logging system. In other words, complete independence from the API underlying logging system is not possible for a stand-alone application. Nevertheless, SLF4J reduces the impact of this dependence to near-painless levels." https://www.slf4j.org/faq.html#when Thanks, Sergey On Thu, Apr 13, 2017 at 7:56 AM, Thomas Weise <t...@apache.org> wrote: > +1 > > Also the proposed feature would need to be implemented in a way that avoids > a hard dependency on log4j. The interface for logging is slf4j and it > should be possible to use other logger backends. > > > On Mon, Apr 10, 2017 at 9:21 PM, Sergey Golovko <ser...@datatorrent.com> > wrote: > > > I don't think an operator needs a specific appender. An appender can be > > dynamically assigned to an application designer, application master and > > container. > > > > Thanks, > > Sergey > > > > > > On Mon, Apr 10, 2017 at 6:26 PM, Munagala Ramanath <r...@datatorrent.com> > > wrote: > > > > > I don't have one, I thought that was what the intent of the proposal > was, > > > but looks like > > > I misunderstood. After re-reading some of the earlier responses, I > > > understand the > > > proposal better. > > > > > > Ram > > > > > > > > > > > > On Mon, Apr 10, 2017 at 5:39 PM, Vlad Rozov <v.ro...@datatorrent.com> > > > wrote: > > > > > > > I don't see a use case where an individual operators need to define a > > > > specific appender, can you provide one? > > > > > > > > Thank you, > > > > > > > > Vlad > > > > > > > > On 4/10/17 16:53, Munagala Ramanath wrote: > > > > > > > >> Yes, totally agree, it would be helpful to have a detailed use case > > > and/or > > > >> a detailed spec > > > >> of the desired capabilities -- not necessarily a complete spec but > > with > > > >> enough detail to > > > >> understand why existing capabilities are inadequate. > > > >> > > > >> Ram > > > >> > > > >> On Mon, Apr 10, 2017 at 4:43 PM, Vlad Rozov < > v.ro...@datatorrent.com> > > > >> wrote: > > > >> > > > >> It will be good to understand a use case where an operator needs a > > > >>> specific appender. > > > >>> > > > >>> IMO, an operator designer defines *what* should be logged and > dev-ops > > > >>> team > > > >>> defines *where* to log. > > > >>> > > > >>> Thank you, > > > >>> > > > >>> Vlad > > > >>> On 4/10/17 16:27, Munagala Ramanath wrote: > > > >>> > > > >>> Yes, I understand, I was just wondering if individual operators > could > > > >>>> define the appenders > > > >>>> they potentially need at compile time and then the operator > > callbacks > > > >>>> could > > > >>>> simply > > > >>>> check the desired runtime condition and add the appropriate > > appender. > > > >>>> > > > >>>> Or are we saying there are scenarios where we absolutely cannot > > create > > > >>>> the > > > >>>> appender beforehand ? > > > >>>> > > > >>>> So broadly speaking, my question is whether the combination of > > > providing > > > >>>> predefined appenders > > > >>>> and the PropertyConfigurator capabilities meets the need. > > > >>>> > > > >>>> Ram > > > >>>> > > > >>>> On Mon, Apr 10, 2017 at 2:18 PM, Sergey Golovko < > > > ser...@datatorrent.com > > > >>>> > > > > >>>> wrote: > > > >>>> > > > >>>> Ram, > > > >>>> > > > >>>>> Really the new appender class must extend the abstract class > > > >>>>> AppenderSkeleton. And in order to add a new appender > > programmatically > > > >>>>> in > > > >>>>> Java, some code in Apex should call the following log4j method: > > > >>>>> > > > >>>>> org.apache.log4j.Logger.getRootLogger().addAppender(Appender > > > >>>>> newAppender) > > > >>>>> > > > >>>>> The general idea of my proposal is "*based on some runtime > > > parameter(s) > > > >>>>> to > > > >>>>> provide ability to create an appender instance via reflection and > > add > > > >>>>> it > > > >>>>> to > > > >>>>> the list of active log4j appenders*". > > > >>>>> > > > >>>>> Thanks, > > > >>>>> Sergey > > > >>>>> > > > >>>>> > > > >>>>> On Mon, Apr 10, 2017 at 2:04 PM, Vlad Rozov < > > v.ro...@datatorrent.com > > > > > > > >>>>> wrote: > > > >>>>> > > > >>>>> It will require application recompilation and repackaging. The > > > proposed > > > >>>>> > > > >>>>>> functionality is for dev-ops to be able to route application > > logging > > > >>>>>> to > > > >>>>>> a > > > >>>>>> preferred destination without recompiling applications. It is > > > run-time > > > >>>>>> configuration vs compile time hardcoded appender. > > > >>>>>> > > > >>>>>> Thank you, > > > >>>>>> > > > >>>>>> Vlad > > > >>>>>> > > > >>>>>> On 4/10/17 11:23, Munagala Ramanath wrote: > > > >>>>>> > > > >>>>>> You can do it in a trivial derived class without changing the > base > > > >>>>>> class. > > > >>>>>> Ram > > > >>>>>> > > > >>>>>>> On Mon, Apr 10, 2017 at 11:19 AM, Vlad Rozov < > > > >>>>>>> v.ro...@datatorrent.com> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>> Does not the proposal to use Logger.addAppender() requires > > > >>>>>>> modifications > > > >>>>>>> > > > >>>>>>> to used operators code? > > > >>>>>>>> > > > >>>>>>>> Thank you, > > > >>>>>>>> > > > >>>>>>>> Vlad > > > >>>>>>>> > > > >>>>>>>> On 4/10/17 10:58, Munagala Ramanath wrote: > > > >>>>>>>> > > > >>>>>>>> People can currently do this by simply implementing the > Appender > > > >>>>>>>> > > > >>>>>>>> interface > > > >>>>>>>>> and adding it > > > >>>>>>>>> with Logger.addAppender() in the setup method. Why do we need > > > >>>>>>>>> > > > >>>>>>>>> something > > > >>>>>>>> > > > >>>>>>> more elaborate ? > > > >>>>>> > > > >>>>>>> Ram > > > >>>>>>>>> > > > >>>>>>>>> On Mon, Apr 10, 2017 at 10:30 AM, Sergey Golovko < > > > >>>>>>>>> ser...@datatorrent.com> > > > >>>>>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>> The configuration of a log4j appender via log4j configuration > > > file > > > >>>>>>>>> is > > > >>>>>>>>> > > > >>>>>>>>> a > > > >>>>>>>> > > > >>>>>>> static configuration that cannot be disabled/enabled and > managed > > > >>>>>> > > > >>>>>>> dynamically by an application designer. The programmatic > approach > > > >>>>>>>>>> > > > >>>>>>>>>> will > > > >>>>>>>>> > > > >>>>>>>> allow an application designer to specify which of the > available > > > >>>>>> > > > >>>>>>> log4j > > > >>>>>>>>> > > > >>>>>>>> appenders should be used for the specific application. > > > >>>>>> > > > >>>>>>> It is not necessary Apex should use the predefined log4j > > appenders > > > >>>>>>>>>> only. > > > >>>>>>>>>> The log4j events contain useful but the very limited number > of > > > >>>>>>>>>> properties > > > >>>>>>>>>> which values can be printed into output log4j sources. But > > based > > > >>>>>>>>>> on > > > >>>>>>>>>> > > > >>>>>>>>>> the > > > >>>>>>>>> > > > >>>>>>>> knowledge of the software product workflow, the custom defined > > > log4j > > > >>>>>> > > > >>>>>>> appender can extend a list of predefined output log events > > > >>>>>>>>>> properties > > > >>>>>>>>>> and, > > > >>>>>>>>>> for instance for Apex, return: node, user name, application > > > name, > > > >>>>>>>>>> application id, container id, operator name, etc. > > > >>>>>>>>>> > > > >>>>>>>>>> Also the output log events that are generated by a custom > > > defined > > > >>>>>>>>>> > > > >>>>>>>>>> log4j > > > >>>>>>>>> > > > >>>>>>>> appender can be stored and indexed by any type of a full text > > > search > > > >>>>>> > > > >>>>>>> database. It will allow the customers and developers to > simplify > > > >>>>>>>>>> collection > > > >>>>>>>>>> of log events statistics and searching/filtering of specific > > > >>>>>>>>>> events > > > >>>>>>>>>> > > > >>>>>>>>>> for > > > >>>>>>>>> > > > >>>>>>>> debugging and investigation. > > > >>>>>> > > > >>>>>>> Thanks, > > > >>>>>>>>>> Sergey > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> On Mon, Apr 10, 2017 at 6:34 AM, Vlad Rozov < > > > >>>>>>>>>> v.ro...@datatorrent.com > > > >>>>>>>>>> wrote: > > > >>>>>>>>>> > > > >>>>>>>>>> +1 Apex engine does not own log4j config file - it is > provided > > > >>>>>>>>>> either > > > >>>>>>>>>> by > > > >>>>>>>>>> > > > >>>>>>>>>> Hadoop or an application. Hadoop log4j config does not > > > necessarily > > > >>>>>>>>>> > > > >>>>>>>>>>> meet > > > >>>>>>>>>>> application logging requirements, but if log4j is provided > by > > > an > > > >>>>>>>>>>> application designer, who can only specify what to log, it > > may > > > >>>>>>>>>>> not > > > >>>>>>>>>>> meet > > > >>>>>>>>>>> operations requirements. Dev-ops should have an ability to > > > >>>>>>>>>>> specify > > > >>>>>>>>>>> where > > > >>>>>>>>>>> > > > >>>>>>>>>>> to > > > >>>>>>>>>>> > > > >>>>>>>>>>> log depending on the available infrastructure at run-time. > > > >>>>>>>>>> > > > >>>>>>>>>> It will be good to have an ability not only specify extra > > log4j > > > >>>>>>>>>>> appenders > > > >>>>>>>>>>> at lunch time, but also at run-time, the same way how log4j > > > >>>>>>>>>>> logger > > > >>>>>>>>>>> levels > > > >>>>>>>>>>> may be changed. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Thank you, > > > >>>>>>>>>>> > > > >>>>>>>>>>> Vlad > > > >>>>>>>>>>> > > > >>>>>>>>>>> On 4/9/17 23:14, Priyanka Gugale wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>> We can always write a custom appender and add it by > changing > > > root > > > >>>>>>>>>>> appender > > > >>>>>>>>>>> in log4j config file. Can you explain how adding appender > > > >>>>>>>>>>> grammatically > > > >>>>>>>>>>> > > > >>>>>>>>>>> would help? > > > >>>>>>>>>>> > > > >>>>>>>>>>>> -Priyanka > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On Sun, Apr 9, 2017 at 11:50 AM, Sanjay Pujare < > > > >>>>>>>>>>>> san...@datatorrent.com > > > >>>>>>>>>>>> wrote: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Please give some examples and/or use cases of this > > > programmatic > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> log4j > > > >>>>>>>>>>> > > > >>>>>>>>>> appender. > > > >>>>>> > > > >>>>>>> On Fri, Apr 7, 2017 at 8:40 PM, Sergey Golovko < > > > >>>>>>>>>>>>> ser...@datatorrent.com > > > >>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Hi All, > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> I'd like to add supporting of a custom defined log4j > > appender > > > >>>>>>>>>>>>> that > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> can > > > >>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>> added to Apex Application Master and Containers and be > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> configurable > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> programmatically. > > > >>>>>> > > > >>>>>>> Sometimes it is not trivial to control log4j configuration via > > > >>>>>>>>>>>>>> log4j > > > >>>>>>>>>>>>>> properties. And I think the having of the approach to > add > > a > > > >>>>>>>>>>>>>> log4j > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> appender > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> programmatically will allow the customers and developers > > to > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> plugin > > > >>>>>>>>>>>>> their > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> own custom defined log4j appenders and be much flexible > for > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> streaming > > > >>>>>>>>>>> > > > >>>>>>>>>> and > > > >>>>>> > > > >>>>>>> collection of Apex log events. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> I assume to provide generic approach for definition of > the > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> programmatic > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> log4j appender and to pass all configuration parameters > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>> including a > > > >>>>>>>>>>>> name > > > >>>>>>>>>>>> of > > > >>>>>>>>>>>> the Java class with implementation of the log4j appender > via > > > >>>>>>>>>>>> system > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> and/or > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> command line properties. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Sergey > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>> > > > >> > > > > > > > > > > > > > -- > > > > > > _______________________________________________________ > > > > > > Munagala V. Ramanath > > > > > > Software Engineer > > > > > > E: r...@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam > > > > > > www.datatorrent.com | apex.apache.org > > > > > >
