Hi Pierre, As per the below information, we see Job is running in local mode and trying to use the local file system for staging dir. Could you please configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html Thanks Devaraj k From: Pierre-Francois Laquerre [mailto:[email protected]] Sent: 01 August 2013 22:57 To: [email protected] Subject: incorrect staging area path in 2.0.5 I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs have been failing to launch due to what seems like an incorrect staging path: # /opt/hadoop2/bin/hadoop jar /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar -input foo -output bar -mapper baz -reducer foobar 13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/myuser1544460269/.staging/job_local1544460269_0001<file:///\\user\myuser1544460269\.staging\job_local1544460269_0001> 13/08/01 10:43:50 ERROR security.UserGroupInformation: PriviledgedActionException as:myuser (auth:SIMPLE) cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory 13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job : chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory Streaming Command Failed! This is for a job launched as "myuser". Given that mapreduce.jobtracker.staging.root.dir is set to /user, I would expect the staging area to be in /user/myuser/.staging/job_local$jobid. Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which fails since /user/myuser$jobid/ doesn't exist. Has the way staging.root.dir is used changed in 2.x? Thank you, Pierre
