+1
> On Apr 3, 2017, at 11:48 AM, Aviem Zur <aviem...@gmail.com> wrote: > > Upon further inspection there seems to be an issue we may have overlooked: > In cluster mode, some of the runners will have dependencies added directly > to the classpath by the cluster, and since SLF4J can only work with one > binding, the first one in the classpath will be used. > > So while what we suggested would work in local mode, the user's chosen > binding and configuration might be ignored in cluster mode, which is > detrimental to what we wanted to accomplish. > > So I believe what we should do instead is: > > 1. Add better documentation regarding logging in each runner, which > binding is used, perhaps examples of how to configure logging for that > runner. > 2. Have direct runner use the most common binding among runners (this > appears to be log4j which is used by Spark runner, Flink runner and Apex > runner). > > >> On Mon, Apr 3, 2017 at 7:02 PM Aljoscha Krettek <aljos...@apache.org> wrote: >> >> Yes, I think we can exclude log4j from the Flink dependencies. It’s >> somewhat annoying that they are there in the first place. >> >> The Flink doc has this to say about the topic: >> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/logging.html >>>> On 3. Apr 2017, at 17:56, Aviem Zur <aviem...@gmail.com> wrote: >>>> >>>> * java.util.logging could be a good choice for the Direct Runner >>> Yes, this will be great for users (Instead of having no logging when >> using >>> direct runner). >>> >>>> * Logging backend could be runner-specific, particularly if it needs to >>>> integrate into some other experience >>> Good point, let's take a look at the current state of runners: >>> Direct runner - will use JUL as suggested. >>> Dataflow runner - looks like there is already no binding (There is a >>> binding in tests only). >>> Spark runner - currently uses slf4j-log4j12. does not require any >> specific >>> logger, we can change this to no binding. >>> Flink runner - uses slf4j-log4j12 transitively from Flink dependencies. >> I'm >>> assuming this is not a must and we can default to no binding here. >>> @aljoscha please confirm. >>> Apex runner - uses slf4j-log4j12 transitively from Apex dependencies. I'm >>> assuming this is not a must and we can default to no binding here. @thw >>> please confirm. >>> >>> It might be a good idea to use a consistent binding in tests (Since we'll >>> use JUL for direct runner, let this be JUL). >>> >>> On Wed, Mar 29, 2017 at 7:23 PM Davor Bonaci <da...@apache.org> wrote: >>> >>> +1 on consistency across Beam modules on the logging facade >>> +1 on enforcing consistency >>> +1 on clearly documenting how to do logging >>> >>> Mixed feelings: >>> * Logging backend could be runner-specific, particularly if it needs to >>> integrate into some other experience >>> * java.util.logging could be a good choice for the Direct Runner >>> >>> On Tue, Mar 28, 2017 at 6:50 PM, Ahmet Altay <al...@google.com.invalid> >>> wrote: >>> >>>> On Wed, Mar 22, 2017 at 10:38 AM, Tibor Kiss <tk...@hortonworks.com> >>>> wrote: >>>> >>>>> This is a great idea! >>>>> >>>>> I believe Python-SDK's logging could also be enhanced (a bit >>>> differently): >>>>> Currently we are not instantiating the logger, just using the class >> what >>>>> logging package provides. >>>>> Shortcoming of this approach is that the user cannot set the log level >>> on >>>>> a per module basis as all log messages >>>>> end up in the root level. >>>> >>>> +1 to this. Python SDK needs to expands its logging capabilities. Filed >>> [1] >>>> for this. >>>> >>>> Ahmet >>>> >>>> [1] https://issues.apache.org/jira/browse/BEAM-1825 >>>> >>>> >>>>> >>>>> On 3/22/17, 5:46 AM, "Aviem Zur" <aviem...@gmail.com> wrote: >>>>> >>>>> +1 to what JB said. >>>>> >>>>> Will just have to be documented well as if we provide no binding >>>> there >>>>> will >>>>> be no logging out of the box unless the user adds a binding. >>>>> >>>>> On Wed, Mar 22, 2017 at 6:24 AM Jean-Baptiste Onofré < >>>> j...@nanthrax.net> >>>>> wrote: >>>>> >>>>>> Hi Aviem, >>>>>> >>>>>> Good point. >>>>>> >>>>>> I think, in our dependencies set, we should just depend to >>>> slf4j-api >>>>> and >>>>>> let the >>>>>> user provides the binding he wants (slf4j-log4j12, slf4j-simple, >>>>> whatever). >>>>>> >>>>>> We define a binding only with test scope in our modules. >>>>>> >>>>>> Regards >>>>>> JB >>>>>> >>>>>>> On 03/22/2017 04:58 AM, Aviem Zur wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> There have been a few reports lately (On JIRA [1] and on Slack) >>>>> from >>>>>> users >>>>>>> regarding inconsistent loggers used across Beam's modules. >>>>>>> >>>>>>> While we use SLF4J, different modules use a different logger >>>>> behind it >>>>>>> (JUL, log4j, etc) >>>>>>> So when people add a log4j.properties file to their classpath >>> for >>>>>> instance, >>>>>>> they expect this to affect all of their dependencies on Beam >>>>> modules, but >>>>>>> it doesn’t and they miss out on some logs they thought they >>> would >>>>> see. >>>>>>> >>>>>>> I think we should strive for consistency in which logger is used >>>>> behind >>>>>>> SLF4J, and try to enforce this in our modules. >>>>>>> I for one think it should be slf4j-log4j. However, if >>> performance >>>>> of >>>>>>> logging is critical we might want to consider logback. >>>>>>> >>>>>>> Note: SLF4J will still be the facade for logging across the >>>>> project. The >>>>>>> only change would be the logger SLF4J delegates to. >>>>>>> >>>>>>> Once we have something like this it would also be useful to add >>>>>>> documentation on logging in Beam to the website. >>>>>>> >>>>>>> [1] https://issues.apache.org/jira/browse/BEAM-1757 >>>>>> >>>>>> -- >>>>>> Jean-Baptiste Onofré >>>>>> jbono...@apache.org >>>>>> http://blog.nanthrax.net >>>>>> Talend - http://www.talend.com >> >>
