Hi Tobias, Ok, write down the job id of the failing job and try the following: Open a terminal shell to one of your hadoop nodes and issue the following command: yarn logs -applicationId <job id> substitute 'job id' with the id of the MR job. This will print out all log information that is available. (App Master logs, mapper logs, reducers logs...etc)
Oussama Chougna > From: [email protected] > To: [email protected] > Subject: AW: AW: Logging using Log4J in MapReduce action > Date: Wed, 9 Sep 2015 12:24:13 +0000 > > Hi Oussama, > > this is exactly my problem: when I do run the MR job inside a MR in Oozie, > the logs from the Mappers and Reducers do not appear in the logs on the Job > History Server. > > Best, > > Tobias > > -----Ursprüngliche Nachricht----- > Von: Oussama Chougna [mailto:[email protected]] > Gesendet: Mittwoch, 9. September 2015 14:17 > An: [email protected] > Betreff: RE: AW: Logging using Log4J in MapReduce action > > Hi Tobias, > Oozie will submit your MR job to YARN.Check the job history server for the > job logs, it is accessible via a browser on http://<your host>:19888. (19888 > is the default port).In this UI click on the failed job, then you can consult > the logs of the mappers and reducers. > Good luck, > > Oussama Chougna > > > From: [email protected] > > To: [email protected] > > Subject: AW: Logging using Log4J in MapReduce action > > Date: Wed, 9 Sep 2015 12:01:44 +0000 > > > > Hi Oussama, > > > > we are using YARN. And we do have a kerberized cluster. I do submit jobs > > via the Command Line like > > > > oozie job -config jobtr1vi jo.properties -run > > > > In my job properties I do set the following properties: > > > > oozie.use.system.libpath=true > > security_enabled=true > > nameNode=hdfs://<mynamenodedns>:8020 > > jobTracker=<jobtrackerdns>:8032 > > oozie.wf.application.path=hdfs://<path-to-workflowxml>/workflow.xml > > > > Best, > > > > Tobias > > > > -----Ursprüngliche Nachricht----- > > Von: Oussama Chougna [mailto:[email protected]] > > Gesendet: Mittwoch, 9. September 2015 13:57 > > An: [email protected] > > Betreff: RE: Logging using Log4J in MapReduce action > > > > Hi Tobias, > > Are you using MRv1 or YARN? How do you submit the jobs? Command line? > > > > Cheers, > > > > Oussama Chougna > > > > > From: [email protected] > > > To: [email protected] > > > Subject: Logging using Log4J in MapReduce action > > > Date: Wed, 9 Sep 2015 08:12:04 +0000 > > > > > > Hello everyone, > > > > > > we did implement a couple of custom MapReduce jobs which we plan to > > > schedule using Oozie workflows in MapReduce actions. > > > > > > All MR jobs are running fine outside of Oozie, but one of them is not. > > > > > > The problem we have is that if those MR jobs are run inside Oozie nothing > > > is get logged using the Log4J loggers that we are registering within the > > > Mapper and Reducer classes using e.g. "private static final Logger logger > > > = Logger.getLogger(TransformMapper.class);" > > > The logging works fine if the MR jobs are run outside of Oozie and > > > logging messages appear in the task attempt logs, but not when they are > > > run within Oozie. > > > > > > Does anyone face similar issues? > > > > > > Does anyone know a solution to this problem? > > > > > > Maybe it is just a matter of configuring something somewhere > > > appropriately. > > > > > > Thanks in advance for your answers, > > > > > > Tobias > > > > > > -- > > > Tobias B. > > > > > >
