So given that it's a fairly unusual logging error, and that you're using slf4j in your reducer, I'm wondering if your logging configuration is somehow interfering with the logging config the child process is using. Could you humor me and pull out your logging code/dependencies and give the job another try?
On Thu, Jun 30, 2011 at 10:07 PM, Paolo Castagna <castagna.li...@googlemail.com> wrote: > Aaron Baff wrote: >> >> Are you submitting from a Windows machine? > > No, I was submitting my jobs from Linux (Ubuntu). > But, thanks for sharing your experience. > > Paolo > >> I remember seeing at one point in time something like this dealing with >> trying to find the user & group on MR submission (if you look, it has to do >> with javax.security.auth and >> org.apache.hadoop.security.UserGroupInformation), and ran into this when >> submitting from a Windows machine. >> >> --Aaron >> >> -----Original Message----- >> From: Paolo Castagna [mailto:castagna.li...@googlemail.com] >> Sent: Wednesday, June 29, 2011 11:54 PM >> To: common-user@hadoop.apache.org >> Subject: NullPointerException when running multiple reducers with Hadoop >> 0.22.0-SNAPSHOT >> >> Hi, >> I am using Apache Whirr to setup an Hadoop cluster on EC2 using Hadoop >> 0.22.0 SNAPSHOTs (nightly) builds from Jenkins. For details, see [1,2]. >> (Is there a better place where I can get nightly builds for Hadoop?) >> >> I have a Reducer which does not emit any (key,value) pairs (it generates >> only side effect files). When I run using only one reducer everything >> seems >> fine. If I setup multiple reducers, each one generating different side >> effect >> files, I get a NullPointerException: >> >> WARN Exception running child : java.lang.NullPointerException >> at >> org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:96) >> at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:239) >> at org.apache.hadoop.mapred.Child$4.run(Child.java:225) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153) >> at org.apache.hadoop.mapred.Child.main(Child.java:217) >> >> Have you ever seen this exception/stacktrace? >> >> My driver is here [3] and the reducer is here [4]. >> >> Regards, >> Paolo >> >> [1] >> https://github.com/castagna/tdbloader3/blob/master/hadoop-ec2.properties >> [2] >> >> https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-22-Build/lastSuccessfulBuild/artifact/hadoop-0.22.0-SNAPSHOT.tar.gz >> [3] >> >> https://github.com/castagna/tdbloader3/blob/master/src/main/java/com/talis/labs/tdb/tdbloader3/ThirdDriver.java >> [4] >> >> https://github.com/castagna/tdbloader3/blob/master/src/main/java/com/talis/labs/tdb/tdbloader3/ThirdReducer.java >> > >
