[
https://issues.apache.org/jira/browse/PIRK-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514313#comment-15514313
]
Walter Ray-Dulany commented on PIRK-65:
---------------------------------------
I've been digging around on this.
First I thought that perhaps we weren't setting the right property file
variables to ensure the log level we wanted. I did a bunch of things, but began
to suspect this wasn't the problem when, after eliminating the entire contents
of log4j2.xml, the distributed tests produced exactly the same logs as before.
Then I thought the problem was some incompatibility between hadoop (which only
knows log4j1) and use (using log4j2), and so I rolled my local branch back to
log4j1, and tried a bunch of different options suggested by various internet
search hits in a log4j.properties file. This, frustratingly, also did nothing.
Finally, I found some posts that explained things. [this
post|http://stackoverflow.com/a/29117078] describes how the log4j levels are
determined in a hadoop/yarn system: for the driver, by a system-wide
log4j.properties file, and for yarn, by either a data node-by-data node config
file container-log4j.properties or, if that doesn't exist, by the values
hard-coded into the hadoop-yarn-server-nodemanager jar file. The author of that
post assures the reader that we cannot change these no matter how many
log4j.properties files we add because the classpath used by hadoop will
pre-empt them. I did some reading to know why/how hadoop was influencing the
classpath: I found [this explanation|http://stackoverflow.com/a/28260868]
sufficiently illuminating, and essentially confirmed what the first-linked post
above was saying.
I haven't yet tested this in a system where I was anything but a tenant, which
means I can't test changing the global values, or using log4j2 (for us) and...I
guess a log4j.properties file (to affect hadoop?).
TL;DR
Unless I'm seriously misunderstanding things,
- In a tenant system without the ability to change global configuration files
for hadoop and yarn, there is no way to change the logging level.
- In a system where you have the ability to change the global configuration
files, you can make global configuration changes to change log levels.
- I think this ticket should be closed as "Won't Fix" or something similar.
I greatly desire the community's thoughts, and would love nothing more than to
be wrong, b/c right now I suspect all we can do to debug the distributed
portions is to, as needed, change logger.debug statements to logger.info
statements.
> Debug Logging Not Working within Distributed Applications
> ---------------------------------------------------------
>
> Key: PIRK-65
> URL: https://issues.apache.org/jira/browse/PIRK-65
> Project: PIRK
> Issue Type: Bug
> Reporter: Ellison Anne Williams
> Assignee: Walter Ray-Dulany
>
> It appears that debug logging is not working within distributed applications
> - only within standalone components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
