Add the hadoop conf dir as is to yarn classpath variable.
E.g.
<property>
<name>yarn.application.classpath</name>
<value>*/etc/hadoop/conf*,/usr/hdp/current/hadoop-client/*
On Wed, Oct 29, 2014 at 11:18 AM, [email protected] <[email protected]> wrote:
> @Pushkar
>
> Thanks, but it doesn't work for me.
> My slider-client.xml setting is
>
> <property>
> <name>yarn.application.classpath</name
>
> <value>/home/siyuan/hadoop/current/etc/hadoop/*,/home/siyuan/hadoop/current/share/hadoop/tools/lib/*,/home/siyuan/hadoop/current/share/hadoop/mapreduce/lib/*,/home/siyuan/hadoop/current/share/hadoop/common/*,/home/siyuan/hadoop/current/share/hadoop/common/lib/*,/home/siyuan/hadoop/current/share/hadoop/hdfs/lib/*,/home/siyuan/hadoop/current/share/hadoop/yarn/*,/home/siyuan/hadoop/current/share/hadoop/yarn/lib/*</value>
> </property>
>
> And
>
> echo $HADOOP_CONF_DIR
> /home/siyuan/hadoop/current/etc/hadoop
>
> echo $YARN_CONF_DIR
> /home/siyuan/hadoop/current/etc/hadoop
>
>
>
> Best,
> Siyuan
>
> On Wed, Oct 29, 2014 at 8:48 AM, Pushkar Raste <[email protected]>
> wrote:
>
> > @hsy541 -
> > This could be problem with your hadoop setup. I too had this problem and
> > thought it was problem with Slider. Turned out that I hadn't setfollowing
> > variable
> > HADOOP_CONF_DIR
> > YARN_CONF_DIR
> >
> >
> > One way to verify if your 'core-site.xml' and 'yar-site.xml' ... are
> being
> > read in Configuration, It to dump it using Configuration object from
> > SliderAppMaster class. Here is what I added to getClusterFS() function in
> > SliderAppMaster.java
> >
> > public SliderFileSystem getClusterFS() throws IOException {
> > log.info("Config is {}", conf.toString());
> > return new SliderFileSystem(conf);
> >
> > }
> >
> >
> > Oputput clearly indicated me that my core-site.xml yarn-site.xml and
> > hdfs-site.xml were not being read. On further investigation I realized
> that
> > my HADOOP_CONF_DIR and YARN_CONF_DIR were not set.
> >
> > On Wed, Oct 29, 2014 at 8:32 AM, Steve Loughran <[email protected]>
> > wrote:
> >
> > > On 27 October 2014 14:54, Pushkar Raste <[email protected]>
> wrote:
> > >
> > > > I am trying to to deploy memcached package using slider (0.40.0
> > > version). I
> > > > keep getting "Wrong FS:
> > > > hdfs://localhost:9000/user/root/.slider/cluster/c100, expected:
> > file:///"
> > > > error
> > > >
> > >
> > >
> > > That's the warning hadoop gives when the filesystem is different from
> the
> > > path URL handed in. Here something thinks the filesystem is file://,
> but
> > > the path to the cluster is
> > > hdfs://localhost:9000/user/root/.slider/cluster/c100
> > >
> > > Now, file:// is the default filesystem of Hadoop —its presence is
> warning
> > > sign that that the configuration that the application has loaded is not
> > > picking up the core-site.xml of that cluster.
> > >
> > > We've been changing bin/slider & slider.py to pick up the local Hadoop
> > > config dirs if present, but I suspect this message may actually be
> coming
> > > from the deployed slider application master. If that's the case, the
> > > probable cause is that whatever classpath it is using to pick up the
> > hadoop
> > > JARs, this path does not include the Hadoop configuration directory.
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
