Hi,
> 1 - I don't understand why this is happening. I should have been able
> to copy data. Why I can't copy data between hdfs?
I think you should check permissions not only for root ("/"), you need
to make sure that whole path is accessible for root. You can do this
using following command:
> /root/Programs/hadoop/bin/hadoop fs -cat
> webhdfs://WorkGroup0000:50070/wiki/wiki/wv92.txt
Make sure that folders /wiki and /wiki/wiki can be listed by root.
Actually I've confused by your use case, I don't understand why you use
webhdfs instead of hdfs. Native hdfs protocol is more faster than webhdfs.
> 2 - Is it possible to turn off the privilege rules?
Yes, it is possible, you should add flowing configuration to
core-site.xml and restart mapreduce & hdfs daemons
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>-- Thanks, Sergey On 12/04/14 20:52, xeon wrote: > Hi, > > I want to copy data between hdfs in different hosts using webhdfs in > hadoop MRv2. The destination host has the user and group as root and > supergroup. > > WorkGroup0010:~/# hdfs dfs -ls / > 14/04/12 16:42:18 WARN util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes > where applicable > Found 3 items > drwxrwx--- - root supergroup 0 2014-04-12 16:25 /root > drwxr-xr-x - root supergroup 0 2014-04-12 16:25 /tmp > drwxrwxrwx - root supergroup 0 2014-04-12 16:39 /wiki > > I defined in the core-site.xml these params for the user and the group. > > WorkGroup0010:~/# cat ../hadoop/etc/hadoop/core-site.xml > <?xml version="1.0" encoding="UTF-8"?> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put site-specific property overrides in this file. --> > <configuration> > <property> <name>fs.default.name</name> > <value>hdfs://172.16.100.5:9000</value> </property> > <property> <name>hadoop.tmp.dir</name> > <value>/tmp/hadoop-temp</value> </property> > <property><name>hadoop.proxyuser.root.hosts</name><value>*</value></property> > > <property><name>hadoop.proxyuser.supergroup.groups</name><value>*</value></property> > <-- I am trying to use with thisgroup definition > <!-- > property><name>hadoop.proxyuser.root.groups</name><value>*</value></property > --> <-- or with this group > </configuration> > > When I copy data > > WorkGroup0000:~/# /root/Programs/hadoop/bin/hadoop distcp > webhdfs://WorkGroup0000:50070/wiki webhdfs://WorkGroup0010:50070/wiki > > I get the error on the destination as: > 2014-04-12 16:37:06,720 ERROR > org.apache.hadoop.security.UserGroupInformation: > PriviledgedActionException as:root (auth:SIMPLE) > cause:java.io.FileNotFoundException: File does not exist: > /wiki/wiki/wv92.txt > > 1 - I don't understand why this is happening. I should have been able > to copy data. Why I can't copy data between hdfs? > 2 - Is it possible to turn off the privilege rules? >
signature.asc
Description: OpenPGP digital signature
