Fair enough. +1 especially for the documentation.
Regards
JB
On 04/03/2017 08:48 PM, Aviem Zur wrote:
Upon further inspection there seems to be an issue we may have overlooked:
In cluster mode, some of the runners will have dependencies added directly
to the classpath by the cluster, and since SLF4J can only work with one
binding, the first one in the classpath will be used.
So while what we suggested would work in local mode, the user's chosen
binding and configuration might be ignored in cluster mode, which is
detrimental to what we wanted to accomplish.
So I believe what we should do instead is:
1. Add better documentation regarding logging in each runner, which
binding is used, perhaps examples of how to configure logging for that
runner.
2. Have direct runner use the most common binding among runners (this
appears to be log4j which is used by Spark runner, Flink runner and Apex
runner).
On Mon, Apr 3, 2017 at 7:02 PM Aljoscha Krettek <aljos...@apache.org> wrote:
Yes, I think we can exclude log4j from the Flink dependencies. It’s
somewhat annoying that they are there in the first place.
The Flink doc has this to say about the topic:
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/logging.html
On 3. Apr 2017, at 17:56, Aviem Zur <aviem...@gmail.com> wrote:
* java.util.logging could be a good choice for the Direct Runner
Yes, this will be great for users (Instead of having no logging when
using
direct runner).
* Logging backend could be runner-specific, particularly if it needs to
integrate into some other experience
Good point, let's take a look at the current state of runners:
Direct runner - will use JUL as suggested.
Dataflow runner - looks like there is already no binding (There is a
binding in tests only).
Spark runner - currently uses slf4j-log4j12. does not require any
specific
logger, we can change this to no binding.
Flink runner - uses slf4j-log4j12 transitively from Flink dependencies.
I'm
assuming this is not a must and we can default to no binding here.
@aljoscha please confirm.
Apex runner - uses slf4j-log4j12 transitively from Apex dependencies. I'm
assuming this is not a must and we can default to no binding here. @thw
please confirm.
It might be a good idea to use a consistent binding in tests (Since we'll
use JUL for direct runner, let this be JUL).
On Wed, Mar 29, 2017 at 7:23 PM Davor Bonaci <da...@apache.org> wrote:
+1 on consistency across Beam modules on the logging facade
+1 on enforcing consistency
+1 on clearly documenting how to do logging
Mixed feelings:
* Logging backend could be runner-specific, particularly if it needs to
integrate into some other experience
* java.util.logging could be a good choice for the Direct Runner
On Tue, Mar 28, 2017 at 6:50 PM, Ahmet Altay <al...@google.com.invalid>
wrote:
On Wed, Mar 22, 2017 at 10:38 AM, Tibor Kiss <tk...@hortonworks.com>
wrote:
This is a great idea!
I believe Python-SDK's logging could also be enhanced (a bit
differently):
Currently we are not instantiating the logger, just using the class
what
logging package provides.
Shortcoming of this approach is that the user cannot set the log level
on
a per module basis as all log messages
end up in the root level.
+1 to this. Python SDK needs to expands its logging capabilities. Filed
[1]
for this.
Ahmet
[1] https://issues.apache.org/jira/browse/BEAM-1825
On 3/22/17, 5:46 AM, "Aviem Zur" <aviem...@gmail.com> wrote:
+1 to what JB said.
Will just have to be documented well as if we provide no binding
there
will
be no logging out of the box unless the user adds a binding.
On Wed, Mar 22, 2017 at 6:24 AM Jean-Baptiste Onofré <
j...@nanthrax.net>
wrote:
Hi Aviem,
Good point.
I think, in our dependencies set, we should just depend to
slf4j-api
and
let the
user provides the binding he wants (slf4j-log4j12, slf4j-simple,
whatever).
We define a binding only with test scope in our modules.
Regards
JB
On 03/22/2017 04:58 AM, Aviem Zur wrote:
Hi all,
There have been a few reports lately (On JIRA [1] and on Slack)
from
users
regarding inconsistent loggers used across Beam's modules.
While we use SLF4J, different modules use a different logger
behind it
(JUL, log4j, etc)
So when people add a log4j.properties file to their classpath
for
instance,
they expect this to affect all of their dependencies on Beam
modules, but
it doesn’t and they miss out on some logs they thought they
would
see.
I think we should strive for consistency in which logger is used
behind
SLF4J, and try to enforce this in our modules.
I for one think it should be slf4j-log4j. However, if
performance
of
logging is critical we might want to consider logback.
Note: SLF4J will still be the facade for logging across the
project. The
only change would be the logger SLF4J delegates to.
Once we have something like this it would also be useful to add
documentation on logging in Beam to the website.
[1] https://issues.apache.org/jira/browse/BEAM-1757
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
