Need some good insight on User Impersonation

2017-05-19 Thread Niranjan Subramanian
Hi All, I’m trying to access HDFS on behalf of another user. I’m trying this with the following application import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.security.UserGroupInformation; import

Yarn not picking up the Resource manager configurations

2017-05-19 Thread Dhanushka Parakrama
Hi All I have setup the multi node hadoop cluster . but when is run the simple map reduce job it gives me the following error the in the data nodes container log . Im running the hadoop 2.7.3 version In namenode which contain resource manager as well . i ran below job hadoop jar

Re: Yarn not picking up the Resource manager configurations

2017-05-19 Thread Miklos Szegedi
Hi, I usually get this error when the yarn-site.xml is not on the class path and it picks up a default 0.0.0.0. I would try: 1. bin/yarn jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi 1 1000 2. explicitly adding $HADOOP_HOME/etc/hadoop to your client classpath 3. make sure

Re: How to make a queue cannot be preempted under FairScheduler?

2017-05-19 Thread Sasaki Kai
Hi I believe there is no nice way to prevent preemption from specific queue. Only thing we can do is disabling preemption completely. yarn.scheduler.fair.preemption true From Hadoop 2.9.0 or 3.0.0-alpha1 we will be able to use allowPreemptionFrom property to forbid preemption from