I don't have one, I thought that was what the intent of the proposal was, but looks like I misunderstood. After re-reading some of the earlier responses, I understand the proposal better.
Ram On Mon, Apr 10, 2017 at 5:39 PM, Vlad Rozov <v.ro...@datatorrent.com> wrote: > I don't see a use case where an individual operators need to define a > specific appender, can you provide one? > > Thank you, > > Vlad > > On 4/10/17 16:53, Munagala Ramanath wrote: > >> Yes, totally agree, it would be helpful to have a detailed use case and/or >> a detailed spec >> of the desired capabilities -- not necessarily a complete spec but with >> enough detail to >> understand why existing capabilities are inadequate. >> >> Ram >> >> On Mon, Apr 10, 2017 at 4:43 PM, Vlad Rozov <v.ro...@datatorrent.com> >> wrote: >> >> It will be good to understand a use case where an operator needs a >>> specific appender. >>> >>> IMO, an operator designer defines *what* should be logged and dev-ops >>> team >>> defines *where* to log. >>> >>> Thank you, >>> >>> Vlad >>> On 4/10/17 16:27, Munagala Ramanath wrote: >>> >>> Yes, I understand, I was just wondering if individual operators could >>>> define the appenders >>>> they potentially need at compile time and then the operator callbacks >>>> could >>>> simply >>>> check the desired runtime condition and add the appropriate appender. >>>> >>>> Or are we saying there are scenarios where we absolutely cannot create >>>> the >>>> appender beforehand ? >>>> >>>> So broadly speaking, my question is whether the combination of providing >>>> predefined appenders >>>> and the PropertyConfigurator capabilities meets the need. >>>> >>>> Ram >>>> >>>> On Mon, Apr 10, 2017 at 2:18 PM, Sergey Golovko <ser...@datatorrent.com >>>> > >>>> wrote: >>>> >>>> Ram, >>>> >>>>> Really the new appender class must extend the abstract class >>>>> AppenderSkeleton. And in order to add a new appender programmatically >>>>> in >>>>> Java, some code in Apex should call the following log4j method: >>>>> >>>>> org.apache.log4j.Logger.getRootLogger().addAppender(Appender >>>>> newAppender) >>>>> >>>>> The general idea of my proposal is "*based on some runtime parameter(s) >>>>> to >>>>> provide ability to create an appender instance via reflection and add >>>>> it >>>>> to >>>>> the list of active log4j appenders*". >>>>> >>>>> Thanks, >>>>> Sergey >>>>> >>>>> >>>>> On Mon, Apr 10, 2017 at 2:04 PM, Vlad Rozov <v.ro...@datatorrent.com> >>>>> wrote: >>>>> >>>>> It will require application recompilation and repackaging. The proposed >>>>> >>>>>> functionality is for dev-ops to be able to route application logging >>>>>> to >>>>>> a >>>>>> preferred destination without recompiling applications. It is run-time >>>>>> configuration vs compile time hardcoded appender. >>>>>> >>>>>> Thank you, >>>>>> >>>>>> Vlad >>>>>> >>>>>> On 4/10/17 11:23, Munagala Ramanath wrote: >>>>>> >>>>>> You can do it in a trivial derived class without changing the base >>>>>> class. >>>>>> Ram >>>>>> >>>>>>> On Mon, Apr 10, 2017 at 11:19 AM, Vlad Rozov < >>>>>>> v.ro...@datatorrent.com> >>>>>>> wrote: >>>>>>> >>>>>>> Does not the proposal to use Logger.addAppender() requires >>>>>>> modifications >>>>>>> >>>>>>> to used operators code? >>>>>>>> >>>>>>>> Thank you, >>>>>>>> >>>>>>>> Vlad >>>>>>>> >>>>>>>> On 4/10/17 10:58, Munagala Ramanath wrote: >>>>>>>> >>>>>>>> People can currently do this by simply implementing the Appender >>>>>>>> >>>>>>>> interface >>>>>>>>> and adding it >>>>>>>>> with Logger.addAppender() in the setup method. Why do we need >>>>>>>>> >>>>>>>>> something >>>>>>>> >>>>>>> more elaborate ? >>>>>> >>>>>>> Ram >>>>>>>>> >>>>>>>>> On Mon, Apr 10, 2017 at 10:30 AM, Sergey Golovko < >>>>>>>>> ser...@datatorrent.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> The configuration of a log4j appender via log4j configuration file >>>>>>>>> is >>>>>>>>> >>>>>>>>> a >>>>>>>> >>>>>>> static configuration that cannot be disabled/enabled and managed >>>>>> >>>>>>> dynamically by an application designer. The programmatic approach >>>>>>>>>> >>>>>>>>>> will >>>>>>>>> >>>>>>>> allow an application designer to specify which of the available >>>>>> >>>>>>> log4j >>>>>>>>> >>>>>>>> appenders should be used for the specific application. >>>>>> >>>>>>> It is not necessary Apex should use the predefined log4j appenders >>>>>>>>>> only. >>>>>>>>>> The log4j events contain useful but the very limited number of >>>>>>>>>> properties >>>>>>>>>> which values can be printed into output log4j sources. But based >>>>>>>>>> on >>>>>>>>>> >>>>>>>>>> the >>>>>>>>> >>>>>>>> knowledge of the software product workflow, the custom defined log4j >>>>>> >>>>>>> appender can extend a list of predefined output log events >>>>>>>>>> properties >>>>>>>>>> and, >>>>>>>>>> for instance for Apex, return: node, user name, application name, >>>>>>>>>> application id, container id, operator name, etc. >>>>>>>>>> >>>>>>>>>> Also the output log events that are generated by a custom defined >>>>>>>>>> >>>>>>>>>> log4j >>>>>>>>> >>>>>>>> appender can be stored and indexed by any type of a full text search >>>>>> >>>>>>> database. It will allow the customers and developers to simplify >>>>>>>>>> collection >>>>>>>>>> of log events statistics and searching/filtering of specific >>>>>>>>>> events >>>>>>>>>> >>>>>>>>>> for >>>>>>>>> >>>>>>>> debugging and investigation. >>>>>> >>>>>>> Thanks, >>>>>>>>>> Sergey >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Apr 10, 2017 at 6:34 AM, Vlad Rozov < >>>>>>>>>> v.ro...@datatorrent.com >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> +1 Apex engine does not own log4j config file - it is provided >>>>>>>>>> either >>>>>>>>>> by >>>>>>>>>> >>>>>>>>>> Hadoop or an application. Hadoop log4j config does not necessarily >>>>>>>>>> >>>>>>>>>>> meet >>>>>>>>>>> application logging requirements, but if log4j is provided by an >>>>>>>>>>> application designer, who can only specify what to log, it may >>>>>>>>>>> not >>>>>>>>>>> meet >>>>>>>>>>> operations requirements. Dev-ops should have an ability to >>>>>>>>>>> specify >>>>>>>>>>> where >>>>>>>>>>> >>>>>>>>>>> to >>>>>>>>>>> >>>>>>>>>>> log depending on the available infrastructure at run-time. >>>>>>>>>> >>>>>>>>>> It will be good to have an ability not only specify extra log4j >>>>>>>>>>> appenders >>>>>>>>>>> at lunch time, but also at run-time, the same way how log4j >>>>>>>>>>> logger >>>>>>>>>>> levels >>>>>>>>>>> may be changed. >>>>>>>>>>> >>>>>>>>>>> Thank you, >>>>>>>>>>> >>>>>>>>>>> Vlad >>>>>>>>>>> >>>>>>>>>>> On 4/9/17 23:14, Priyanka Gugale wrote: >>>>>>>>>>> >>>>>>>>>>> We can always write a custom appender and add it by changing root >>>>>>>>>>> appender >>>>>>>>>>> in log4j config file. Can you explain how adding appender >>>>>>>>>>> grammatically >>>>>>>>>>> >>>>>>>>>>> would help? >>>>>>>>>>> >>>>>>>>>>>> -Priyanka >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Apr 9, 2017 at 11:50 AM, Sanjay Pujare < >>>>>>>>>>>> san...@datatorrent.com >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Please give some examples and/or use cases of this programmatic >>>>>>>>>>>> >>>>>>>>>>>> log4j >>>>>>>>>>> >>>>>>>>>> appender. >>>>>> >>>>>>> On Fri, Apr 7, 2017 at 8:40 PM, Sergey Golovko < >>>>>>>>>>>>> ser...@datatorrent.com >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> I'd like to add supporting of a custom defined log4j appender >>>>>>>>>>>>> that >>>>>>>>>>>>> >>>>>>>>>>>>> can >>>>>>>>>>>>>> be >>>>>>>>>>>>>> added to Apex Application Master and Containers and be >>>>>>>>>>>>>> >>>>>>>>>>>>>> configurable >>>>>>>>>>>>> >>>>>>>>>>>> programmatically. >>>>>> >>>>>>> Sometimes it is not trivial to control log4j configuration via >>>>>>>>>>>>>> log4j >>>>>>>>>>>>>> properties. And I think the having of the approach to add a >>>>>>>>>>>>>> log4j >>>>>>>>>>>>>> >>>>>>>>>>>>>> appender >>>>>>>>>>>>>> >>>>>>>>>>>>>> programmatically will allow the customers and developers to >>>>>>>>>>>>>> >>>>>>>>>>>>> plugin >>>>>>>>>>>>> their >>>>>>>>>>>>> >>>>>>>>>>>>> own custom defined log4j appenders and be much flexible for >>>>>>>>>>>>> >>>>>>>>>>>> streaming >>>>>>>>>>> >>>>>>>>>> and >>>>>> >>>>>>> collection of Apex log events. >>>>>>>>>>>>> >>>>>>>>>>>>>> I assume to provide generic approach for definition of the >>>>>>>>>>>>>> >>>>>>>>>>>>>> programmatic >>>>>>>>>>>>>> >>>>>>>>>>>>>> log4j appender and to pass all configuration parameters >>>>>>>>>>>>> >>>>>>>>>>>> including a >>>>>>>>>>>> name >>>>>>>>>>>> of >>>>>>>>>>>> the Java class with implementation of the log4j appender via >>>>>>>>>>>> system >>>>>>>>>>>> >>>>>>>>>>>> and/or >>>>>>>>>>>>> >>>>>>>>>>>>>> command line properties. >>>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>>> Sergey >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>> >> > -- _______________________________________________________ Munagala V. Ramanath Software Engineer E: r...@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam www.datatorrent.com | apex.apache.org
