try running with sudo hive. On Mon, Apr 6, 2015 at 1:40 PM, Inosh Goonewardena <[email protected]> wrote:
> Hi All, > > Really appreciate any input on this issue. > > Thanks and Regards, > Inosh > > On Tue, Jan 20, 2015 at 10:29 PM, Inosh Goonewardena <[email protected]> > wrote: > >> Hi All, >> >> We have setup a Hadoop cluster using hadoop 1.0.4 and we use Hive to >> submit map/reduce jobs. But we have noticed that these map/reduce job >> submission failing time to time due to a permission issue occurring in >> Hadoop. When this happens, we have to use the hadoop hdfs command and >> change the file permission of mapreduce staging directory. Following is the >> error that is getting logged. >> >> [2015-01-16 05:30:13,314] ERROR {org.apache.hadoop.hive.ql.exec.Task} - >> Job Submission failed with exception 'java.io.IOException(The >> ownership/permissions on the staging directory hdfs:// >> hadoop0.test.com:9000/mnt/hadoop_tmp/mapred/staging/inosh/.staging is >> not as expected. It is owned by inosh and permissions are rwxr-xr-x. The >> directory must be owned by the submitter inosh or by inosh and permissions >> must be rwx------)' >> java.io.IOException: The ownership/permissions on the staging directory >> hdfs://hadoop0.test.com:9000/mnt/hadoop_tmp/mapred/staging/inosh/.staging >> is not as expected. It is owned by inosh and permissions are rwxr-xr-x. The >> directory must be owned by the submitter inosh or by inosh and permissions >> must be rwx------ >> at >> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:108) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:798) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:792) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1123) >> at >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:792) >> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:766) >> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:460) >> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) >> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:129) >> at >> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:62) >> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1351) >> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1126) >> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:934) >> at >> org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:201) >> at >> org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:187) >> ... >> >> >> When we get this error we go to hadoop name node and execute following >> command to set proper permission to the folder mentioned in the exception. >> ./hadoop dfs -chmod 700 /mnt/hadoop_tmp/mapred/staging/inosh/.staging >> >> >> We would like know why this happens and how to fix this permanently. >> >> >> Thanks and Regards, >> Inosh >> > > -- Harshit Mathur
