Alexander Semenov wrote:
Ok, thanks. Unfortunately ant is currently not installed on machine
running hadoop. What if I use slf4j + logback just in job's jar?
depends on the classpath. You can do it in your own code by getting
whatever classloader you use then call getResource("log4.properties") to
get the URL, which you can print to the console, something like:
System.err.println("Log4J properties at
"+this.getClass().getClassloader().getResource("log4.properties"));
BTW, is hadoop planning to migrate on this stack from the deprecated
Apache commons and log4j?
Deprecated? That's very much a point of view.
1. commons-log is a thin front end to anything; I have my own that can
be switched in when I desire to get the logs from >1 machine in one
place. Where it is great for is in libraries which can be embedded in
other things -Hadoop does get used this way- as it stops the library
saying "here is the logging tool you must use". you get to choose.
2. Log4J is a fantastic logging API with some really good back end
implementations. In Hadoop, I'd recommend the rolling Logs. There is
good support in Hadoop for changing logging levels as you go along.
3. slf4j was meant to be a log api that avoided all the problems of
layering, but it actually introduces a new one: the risk of >1 SLF4J on
the classpath. And, as it's an extra JAR that Jetty requires, you have
to make sure your job's SLF4j doesn't clash with the one used for Jetty.
And its back-ends aren't on a par with Log4J.
I'm happy with commons-logging and log4j, just wish that jetty had
stayed with commons-logging. The alternative would be the
java.util.logging APIs, which are themselves a dog to configure. You
need to point to the right config file using JVM system properties which
really need to be set on the command line to get picked up early enough,
and as the docs say:
"By default, the LogManager reads its initial configuration from a
properties file "lib/logging.properties" in the JRE directory. "
This isn't as bad as having a commons-logging or log4.properties in the
JAR of a third party library, but it means that by default, one logging
setup per JVM unless you deliberately configure each app differently.
Which is a PITA, and its probably one of the reasons that the java
logging API never took off, and doesn't have as good back end loggers as
Log4J.
Summary: try and find the properties file, it may just be classpath/JVM
quirks, learn to use the rolling/daily log4j logs, don't keep the logs
on your root drive either.
-Steve
