Hi, I installed the apache hadoop with just unzip the tar ball. The configuration should be under $HADOOP_HOME/etc/hadoop/, right? I have added $HADOOP_HOME/etc/hadoop/* in yarn.application.classpath
On Wed, Oct 29, 2014 at 11:24 AM, Sumit Mohanty <[email protected]> wrote: > Add the hadoop conf dir as is to yarn classpath variable. > > E.g. > <property> > <name>yarn.application.classpath</name> > <value>*/etc/hadoop/conf*,/usr/hdp/current/hadoop-client/* > > On Wed, Oct 29, 2014 at 11:18 AM, [email protected] <[email protected]> > wrote: > > > @Pushkar > > > > Thanks, but it doesn't work for me. > > My slider-client.xml setting is > > > > <property> > > <name>yarn.application.classpath</name > > > > > <value>/home/siyuan/hadoop/current/etc/hadoop/*,/home/siyuan/hadoop/current/share/hadoop/tools/lib/*,/home/siyuan/hadoop/current/share/hadoop/mapreduce/lib/*,/home/siyuan/hadoop/current/share/hadoop/common/*,/home/siyuan/hadoop/current/share/hadoop/common/lib/*,/home/siyuan/hadoop/current/share/hadoop/hdfs/lib/*,/home/siyuan/hadoop/current/share/hadoop/yarn/*,/home/siyuan/hadoop/current/share/hadoop/yarn/lib/*</value> > > </property> > > > > And > > > > echo $HADOOP_CONF_DIR > > /home/siyuan/hadoop/current/etc/hadoop > > > > echo $YARN_CONF_DIR > > /home/siyuan/hadoop/current/etc/hadoop > > > > > > > > Best, > > Siyuan > > > > On Wed, Oct 29, 2014 at 8:48 AM, Pushkar Raste <[email protected]> > > wrote: > > > > > @hsy541 - > > > This could be problem with your hadoop setup. I too had this problem > and > > > thought it was problem with Slider. Turned out that I hadn't > setfollowing > > > variable > > > HADOOP_CONF_DIR > > > YARN_CONF_DIR > > > > > > > > > One way to verify if your 'core-site.xml' and 'yar-site.xml' ... are > > being > > > read in Configuration, It to dump it using Configuration object from > > > SliderAppMaster class. Here is what I added to getClusterFS() function > in > > > SliderAppMaster.java > > > > > > public SliderFileSystem getClusterFS() throws IOException { > > > log.info("Config is {}", conf.toString()); > > > return new SliderFileSystem(conf); > > > > > > } > > > > > > > > > Oputput clearly indicated me that my core-site.xml yarn-site.xml and > > > hdfs-site.xml were not being read. On further investigation I realized > > that > > > my HADOOP_CONF_DIR and YARN_CONF_DIR were not set. > > > > > > On Wed, Oct 29, 2014 at 8:32 AM, Steve Loughran < > [email protected]> > > > wrote: > > > > > > > On 27 October 2014 14:54, Pushkar Raste <[email protected]> > > wrote: > > > > > > > > > I am trying to to deploy memcached package using slider (0.40.0 > > > > version). I > > > > > keep getting "Wrong FS: > > > > > hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: > > > file:///" > > > > > error > > > > > > > > > > > > > > > > > That's the warning hadoop gives when the filesystem is different from > > the > > > > path URL handed in. Here something thinks the filesystem is file://, > > but > > > > the path to the cluster is > > > > hdfs://localhost:9000/user/root/.slider/cluster/c100 > > > > > > > > Now, file:// is the default filesystem of Hadoop —its presence is > > warning > > > > sign that that the configuration that the application has loaded is > not > > > > picking up the core-site.xml of that cluster. > > > > > > > > We've been changing bin/slider & slider.py to pick up the local > Hadoop > > > > config dirs if present, but I suspect this message may actually be > > coming > > > > from the deployed slider application master. If that's the case, the > > > > probable cause is that whatever classpath it is using to pick up the > > > hadoop > > > > JARs, this path does not include the Hadoop configuration directory. > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > entity > > > to > > > > which it is addressed and may contain information that is > confidential, > > > > privileged and exempt from disclosure under applicable law. If the > > reader > > > > of this message is not the intended recipient, you are hereby > notified > > > that > > > > any printing, copying, dissemination, distribution, disclosure or > > > > forwarding of this communication is strictly prohibited. If you have > > > > received this communication in error, please contact the sender > > > immediately > > > > and delete it from your system. Thank You. > > > > > > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
