Hi All,
I am trying to follow first steps on getting a simple C++ program to work using
Hadoop Pipes. And I get the following error while running it. Can anybody help
me understand what could I be doing wrong?
Used the following code.
http://wiki.apache.org/hadoop/C%2B%2BWordCount
# hadoop pipes -D hadoop.pipes.java.recordreader=true -D
hadoop.pipes.java.recordwriter=true -input cppinput -output coutput -program
bin/wordc
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.
11/03/02 10:47:23 INFO security.Groups: Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
11/03/02 10:47:23 WARN conf.Configuration: mapred.task.id is deprecated.
Instead, use mapreduce.task.attempt.id
11/03/02 10:47:23 WARN mapreduce.JobSubmitter: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
11/03/02 10:47:23 WARN mapreduce.JobSubmitter: No job jar file set. User
classes may not be found. See Job or Job#setJar(String).
11/03/02 10:47:23 INFO mapred.FileInputFormat: Total input paths to process : 1
11/03/02 10:47:24 WARN conf.Configuration: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
11/03/02 10:47:24 INFO mapreduce.JobSubmitter: number of splits:2
11/03/02 10:47:24 INFO mapreduce.JobSubmitter: adding the following namenodes'
delegation tokens:null
11/03/02 10:47:24 INFO mapreduce.Job: Running job: job_201102161124_0012
11/03/02 10:47:25 INFO mapreduce.Job: map 0% reduce 0%
11/03/02 10:47:40 INFO mapreduce.Job: Task Id :
attempt_201102161124_0012_m_000000_0, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:160)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:102)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:86)
... 7 more
attempt_201102161124_0012_m_000000_0: Hadoop Pipes Exception: Aborted by driver
11/03/02 10:47:40 INFO mapreduce.Job: Task Id :
attempt_201102161124_0012_m_000001_0, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:160)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:102)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:86)
... 7 more
attempt_201102161124_0012_m_000001_0: Hadoop Pipes Exception: Aborted by driver
11/03/02 10:47:52 INFO mapreduce.Job: Task Id :
attempt_201102161124_0012_m_000000_1, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:160)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:102)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:86)
... 7 more
attempt_201102161124_0012_m_000000_1: Hadoop Pipes Exception: Aborted by driver
11/03/02 10:47:52 INFO mapreduce.Job: Task Id :
attempt_201102161124_0012_m_000001_1, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:160)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:102)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:86)
... 7 more
attempt_201102161124_0012_m_000001_1: Hadoop Pipes Exception: Aborted by driver
11/03/02 10:48:04 INFO mapreduce.Job: Task Id :
attempt_201102161124_0012_m_000000_2, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:160)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:102)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:86)
... 7 more
attempt_201102161124_0012_m_000000_2: Hadoop Pipes Exception: Aborted by driver
11/03/02 10:48:04 INFO mapreduce.Job: Task Id :
attempt_201102161124_0012_m_000001_2, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:160)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:102)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:86)
... 7 more
attempt_201102161124_0012_m_000001_2: Hadoop Pipes Exception: Aborted by driver
11/03/02 10:48:20 INFO mapreduce.Job: Job complete: job_201102161124_0012
11/03/02 10:48:20 INFO mapreduce.Job: Counters: 8
Job Counters
Data-local map tasks=1
Total time spent by all maps waiting after reserving slots
(ms)=0
Total time spent by all reduces waiting after reserving slots
(ms)=0
Failed map tasks=1
Rack-local map tasks=7
SLOTS_MILLIS_MAPS=89957
SLOTS_MILLIS_REDUCES=0
Launched map tasks=8
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:782)
at org.apache.hadoop.mapred.pipes.Submitter.runJob(Submitter.java:264)
at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:504)
at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:519)
Any help greatly appreciated,
Thanks,
Amit
