And all I have in slider-err.txt is log4j:WARN No appenders could be found for logger (org.apache.slider.core.main.ServiceLauncher). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Exception: Wrong FS: hdfs://localhost:9000/user/siyuan/.slider/cluster/cl15, expected: file:///
Looks like there is no log4j log at all. How do I properly setup log4j? Best, Siyuan On Wed, Oct 29, 2014 at 11:47 AM, [email protected] <[email protected]> wrote: > Hi, > > I installed the apache hadoop with just unzip the tar ball. > The configuration should be under $HADOOP_HOME/etc/hadoop/, right? > I have added $HADOOP_HOME/etc/hadoop/* in yarn.application.classpath > > On Wed, Oct 29, 2014 at 11:24 AM, Sumit Mohanty <[email protected]> > wrote: > >> Add the hadoop conf dir as is to yarn classpath variable. >> >> E.g. >> <property> >> <name>yarn.application.classpath</name> >> <value>*/etc/hadoop/conf*,/usr/hdp/current/hadoop-client/* >> >> On Wed, Oct 29, 2014 at 11:18 AM, [email protected] <[email protected]> >> wrote: >> >> > @Pushkar >> > >> > Thanks, but it doesn't work for me. >> > My slider-client.xml setting is >> > >> > <property> >> > <name>yarn.application.classpath</name >> > >> > >> <value>/home/siyuan/hadoop/current/etc/hadoop/*,/home/siyuan/hadoop/current/share/hadoop/tools/lib/*,/home/siyuan/hadoop/current/share/hadoop/mapreduce/lib/*,/home/siyuan/hadoop/current/share/hadoop/common/*,/home/siyuan/hadoop/current/share/hadoop/common/lib/*,/home/siyuan/hadoop/current/share/hadoop/hdfs/lib/*,/home/siyuan/hadoop/current/share/hadoop/yarn/*,/home/siyuan/hadoop/current/share/hadoop/yarn/lib/*</value> >> > </property> >> > >> > And >> > >> > echo $HADOOP_CONF_DIR >> > /home/siyuan/hadoop/current/etc/hadoop >> > >> > echo $YARN_CONF_DIR >> > /home/siyuan/hadoop/current/etc/hadoop >> > >> > >> > >> > Best, >> > Siyuan >> > >> > On Wed, Oct 29, 2014 at 8:48 AM, Pushkar Raste <[email protected] >> > >> > wrote: >> > >> > > @hsy541 - >> > > This could be problem with your hadoop setup. I too had this problem >> and >> > > thought it was problem with Slider. Turned out that I hadn't >> setfollowing >> > > variable >> > > HADOOP_CONF_DIR >> > > YARN_CONF_DIR >> > > >> > > >> > > One way to verify if your 'core-site.xml' and 'yar-site.xml' ... are >> > being >> > > read in Configuration, It to dump it using Configuration object from >> > > SliderAppMaster class. Here is what I added to getClusterFS() >> function in >> > > SliderAppMaster.java >> > > >> > > public SliderFileSystem getClusterFS() throws IOException { >> > > log.info("Config is {}", conf.toString()); >> > > return new SliderFileSystem(conf); >> > > >> > > } >> > > >> > > >> > > Oputput clearly indicated me that my core-site.xml yarn-site.xml and >> > > hdfs-site.xml were not being read. On further investigation I realized >> > that >> > > my HADOOP_CONF_DIR and YARN_CONF_DIR were not set. >> > > >> > > On Wed, Oct 29, 2014 at 8:32 AM, Steve Loughran < >> [email protected]> >> > > wrote: >> > > >> > > > On 27 October 2014 14:54, Pushkar Raste <[email protected]> >> > wrote: >> > > > >> > > > > I am trying to to deploy memcached package using slider (0.40.0 >> > > > version). I >> > > > > keep getting "Wrong FS: >> > > > > hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: >> > > file:///" >> > > > > error >> > > > > >> > > > >> > > > >> > > > That's the warning hadoop gives when the filesystem is different >> from >> > the >> > > > path URL handed in. Here something thinks the filesystem is file://, >> > but >> > > > the path to the cluster is >> > > > hdfs://localhost:9000/user/root/.slider/cluster/c100 >> > > > >> > > > Now, file:// is the default filesystem of Hadoop —its presence is >> > warning >> > > > sign that that the configuration that the application has loaded is >> not >> > > > picking up the core-site.xml of that cluster. >> > > > >> > > > We've been changing bin/slider & slider.py to pick up the local >> Hadoop >> > > > config dirs if present, but I suspect this message may actually be >> > coming >> > > > from the deployed slider application master. If that's the case, the >> > > > probable cause is that whatever classpath it is using to pick up the >> > > hadoop >> > > > JARs, this path does not include the Hadoop configuration directory. >> > > > >> > > > -- >> > > > CONFIDENTIALITY NOTICE >> > > > NOTICE: This message is intended for the use of the individual or >> > entity >> > > to >> > > > which it is addressed and may contain information that is >> confidential, >> > > > privileged and exempt from disclosure under applicable law. If the >> > reader >> > > > of this message is not the intended recipient, you are hereby >> notified >> > > that >> > > > any printing, copying, dissemination, distribution, disclosure or >> > > > forwarding of this communication is strictly prohibited. If you have >> > > > received this communication in error, please contact the sender >> > > immediately >> > > > and delete it from your system. Thank You. >> > > > >> > > >> > >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. >> > >
