I'm running on a cluster. I'm trying to write to the log files on the cluster machines, the ones that are visible through the jobtracker web interface.
The log4j file I gave excerpts from is a central one for the cluster. On Wed, Dec 15, 2010 at 1:38 PM, Aaron Kimball <[email protected]> wrote: > W. P., > > How are you running your Reducer? Is everything running in standalone mode > (all mappers/reducers in the same process as the launching application)? Or > are you running this in pseudo-distributed mode or on a remote cluster? > > Depending on the application's configuration, log4j configuration could be > read from one of many different places. > > Furthermore, where are you expecting your output? If you're running in > pseudo-distributed (or fully distributed) mode, mapper / reducer tasks will > not emit output back to the console of the launching application. That > only > happens in local mode. In the distributed flavors, you'll see a different > file for each task attempt containing its log output, on the machine where > the task executed. These files can be accessed through the web UI at > http://jobtracker:50030/ -- click on the job, then the task, then the task > attempt, then "syslog" in the right-most column. > > - Aaron > > On Mon, Dec 13, 2010 at 10:05 AM, W.P. McNeill <[email protected]> wrote: > > > I would like to use Hadoop's Log4j infrastructure to do logging from my > > map/reduce application. I think I've got everything set up correctly, > but > > I > > am still unable to specify the logging level I want. > > > > By default Hadoop is set up to log at level INFO. The first line of its > > log4j.properties file looks like this: > > > > hadoop.root.logger=INFO,console > > > > > > I have an application whose reducer looks like this: > > > > package com.me; > > > > public class MyReducer<...> extends Reducer<...> { > > private static Logger logger = > > Logger.getLogger(MyReducer.class.getName()); > > > > ... > > protected void reduce(...) { > > logger.debug("My message"); > > ... > > } > > } > > > > > > I've added the following line to the Hadoop log4j.properties file: > > > > log4j.logger.com.me.MyReducer=DEBUG > > > > > > I expect the Hadoop system to log at level INFO, but my application to > log > > at level DEBUG, so that I see "My message" in the logs for the reducer > > task. > > However, my application does not produce any log4j output. If I change > > the > > line in my reducer to read logger.info("My message") the message does > get > > logged, so somehow I'm failing to specify that log level for this class. > > > > I've also tried changing the log4j line for my app to > > read log4j.logger.com.me.MyReducer=DEBUG,console and get the same result. > > > > I've been through the Hadoop and log4j documentation and I can't figure > out > > what I'm doing wrong. Any suggestions? > > > > Thanks. > > >
