Chris Riccomini created SAMZA-350:
-------------------------------------
Summary: Allow dynamic log level toggling
Key: SAMZA-350
URL: https://issues.apache.org/jira/browse/SAMZA-350
Project: Samza
Issue Type: Bug
Components: container
Affects Versions: 0.7.0
Reporter: Chris Riccomini
Samza uses SLF4J for logging. We do not expose any mechanism to dynamically
change the log level at runtime. I'm not even sure if SLF4J can do this because
the log level settings are usually implementation-specific.
Log4J does allow this via the setLevel API. By default in Log4J 2, JMX is
already enabled. The hello-samza app shows how to use Log4J 1.2.*, and we use
Log4J 1.2.* at LinkedIn.
We should figure out how to implement this. Ideally, we'd do it at the SLF4J
level so we don't break the log-implementation-independence that we get with
SLF4J. Unfortunately, it doesn't seem that SLF4J supports this. This leaves us
with log-implementation-specific support.
I propose in this ticket that we just write Log4J 1.2.* specific implementation
that calls:
{code}
Logger.getRootLogger().setLevel(...);
{code}
The second question is how to implement the RPC. There are many ways. We could
use the ConfigLog (SAMZA-348), some other control stream, an HTTP endpoint in
the containers, or JMX.
My preference is to use JMX for this. It's nice and simple. The ConfigLog
approach seems appealing, but it would either require containers to restart
(not ideal if you're trying to debug an issue as it's happening) or would
require all containers to listen to the ConfigLog topic (seems a big commitment
if we want to keep containers light-weight).
The third question is where this code should live. This code should be run in
all containers and the YARN AM. I am favor having this in its own module
(samza-log4j) just so we don't have to introduce a Log4J dependency to
samza-core, which we've thus far been able to avoid. A reasonable location for
this seems to be in a TaskLifecycleListener, which could setup the MBean in the
beforeInit method. Unfortunately, the YARN AM does not use
TaskLifecycleListener. We could write a SamzaAppMasterLifecycleListener
interface, and implement the Log4J MBean for both.
There might also be a better place for this to live. An argument could be made
that this should be part of Samza-proper, and that we should just switch to
Log4J. Another argument could be made that we need a better lifecycle
interface, or some generic plugin thingy to let us run this kind of logic.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
