[
https://issues.apache.org/jira/browse/IGNITE-218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364776#comment-14364776
]
Ivan Veselovsky edited comment on IGNITE-218 at 3/17/15 9:14 AM:
-----------------------------------------------------------------
Konstantin, there is a problem with the image: no hadoop daemon starts on it.
The reason is that all the daemons are intended to run on bahalf of specific
user, and for some reason , which is not quite clear for me, "su" command fails
with message "could not open session". This can be easily reproduced even with
command "su yarn": the command fails with the same symptoms. Same happens for
"hdfs" user. However, for example, "su jenkins" work perfectly fine. Home
directories for yarn and hdfs do exist and are readable/executable.
(I ran your docker image under RedHat 6.4 in VirtualBox.)
was (Author: iveselovskiy):
Konstantin, there is a problem with the image: no hadoop daemon starts on it.
The reason is that all the daemons are intended to run on bahalf of specific
user, and for some reason , which is not quite clear for me, "su" command fails
with message "could not open session". This can be easily reproduced even with
command "su yarn": the command fails with the same symptoms. Same happens for
"hdfs" user. However, for example, "su jenkins" work perfectly fine. Home
directories for yarn and hdfs do exist and are readable/executable.
> Wrong staging permissions while running MR job under hadoop accelerator
> -----------------------------------------------------------------------
>
> Key: IGNITE-218
> URL: https://issues.apache.org/jira/browse/IGNITE-218
> Project: Ignite
> Issue Type: Bug
> Components: hadoop
> Affects Versions: sprint-1
> Environment: Hadoop 2.4.1
> gridgain-hadoop-6.5.6 (Bigtop trunk) no later than
> d90b89dcc4ac7895d902fed034793c5fcb332e8b
> Reporter: Konstantin Boudnik
> Assignee: Ivan Veselovsky
> Fix For: sprint-3
>
>
> I am trying to run an MR example on gridgain-hadoop setup in Bigtop. The ggfs
> is configured to be backed by HDFS with
> {code}
> <property name="secondaryFileSystem">
> <bean
> class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
> <constructor-arg
> value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
> <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
> </bean>
> </property>
> {code}
> set for {{ggfsConfiguration}}
> While all usual filesystem operations work properly, running an MR job
> produces the following exception in the beginning of the run
> {noformat}
> HADOOP_CONF_DIR=/etc/hadoop/gridgain.client.conf hadoop jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 2 2
> Number of Maps = 2
> Samples per Map = 2
> Wrote input for Map #0
> Wrote input for Map #1
> Starting Job
> java.io.IOException: The ownership on the staging directory
> /tmp/hadoop-yarn/staging/root/.staging is not as expected. It is owned by
> gridgain. The directory must be owned by the submitter root or by root
> at
> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:112)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
> at
> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}
> However the permissions seem to be correct.
> {noformat}
> [root@vmhost05-hbase2 gridgain.client.conf]# sudo -u hdfs hadoop fs -ls
> /tmp/hadoop-yarn/staging/root
> Found 1 items
> drwx------ - root root 0 2015-02-10 19:31
> /tmp/hadoop-yarn/staging/root/.staging
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
