Hi All, Really appreciate any input on this issue.
Thanks and Regards, Inosh On Tue, Jan 20, 2015 at 10:29 PM, Inosh Goonewardena <[email protected]> wrote: > Hi All, > > We have setup a Hadoop cluster using hadoop 1.0.4 and we use Hive to > submit map/reduce jobs. But we have noticed that these map/reduce job > submission failing time to time due to a permission issue occurring in > Hadoop. When this happens, we have to use the hadoop hdfs command and > change the file permission of mapreduce staging directory. Following is the > error that is getting logged. > > [2015-01-16 05:30:13,314] ERROR {org.apache.hadoop.hive.ql.exec.Task} - > Job Submission failed with exception 'java.io.IOException(The > ownership/permissions on the staging directory hdfs:// > hadoop0.test.com:9000/mnt/hadoop_tmp/mapred/staging/inosh/.staging is not > as expected. It is owned by inosh and permissions are rwxr-xr-x. The > directory must be owned by the submitter inosh or by inosh and permissions > must be rwx------)' > java.io.IOException: The ownership/permissions on the staging directory > hdfs://hadoop0.test.com:9000/mnt/hadoop_tmp/mapred/staging/inosh/.staging > is not as expected. It is owned by inosh and permissions are rwxr-xr-x. The > directory must be owned by the submitter inosh or by inosh and permissions > must be rwx------ > at > org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:108) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:798) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:792) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1123) > at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:792) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:766) > at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:460) > at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:129) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:62) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1351) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1126) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:934) > at > org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:201) > at > org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:187) > ... > > > When we get this error we go to hadoop name node and execute following > command to set proper permission to the folder mentioned in the exception. > ./hadoop dfs -chmod 700 /mnt/hadoop_tmp/mapred/staging/inosh/.staging > > > We would like know why this happens and how to fix this permanently. > > > Thanks and Regards, > Inosh >
