Sorry, my classpath should be $HADOOP_HOME/etc/hadoop/, thanks for you guys' help!
On Wed, Oct 29, 2014 at 12:06 PM, [email protected] <[email protected]> wrote: > > > And all I have in slider-err.txt is > > log4j:WARN No appenders could be found for logger > (org.apache.slider.core.main.ServiceLauncher). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > Exception: Wrong FS: hdfs://localhost:9000/user/siyuan/.slider/cluster/cl15, > expected: file:/// > > Looks like there is no log4j log at all. > > How do I properly setup log4j? > > > Best, > > Siyuan > > > > On Wed, Oct 29, 2014 at 11:47 AM, [email protected] <[email protected]> > wrote: > >> Hi, >> >> I installed the apache hadoop with just unzip the tar ball. >> The configuration should be under $HADOOP_HOME/etc/hadoop/, right? >> I have added $HADOOP_HOME/etc/hadoop/* in yarn.application.classpath >> >> On Wed, Oct 29, 2014 at 11:24 AM, Sumit Mohanty <[email protected] >> > wrote: >> >>> Add the hadoop conf dir as is to yarn classpath variable. >>> >>> E.g. >>> <property> >>> <name>yarn.application.classpath</name> >>> <value>*/etc/hadoop/conf*,/usr/hdp/current/hadoop-client/* >>> >>> On Wed, Oct 29, 2014 at 11:18 AM, [email protected] <[email protected]> >>> wrote: >>> >>> > @Pushkar >>> > >>> > Thanks, but it doesn't work for me. >>> > My slider-client.xml setting is >>> > >>> > <property> >>> > <name>yarn.application.classpath</name >>> > >>> > >>> <value>/home/siyuan/hadoop/current/etc/hadoop/*,/home/siyuan/hadoop/current/share/hadoop/tools/lib/*,/home/siyuan/hadoop/current/share/hadoop/mapreduce/lib/*,/home/siyuan/hadoop/current/share/hadoop/common/*,/home/siyuan/hadoop/current/share/hadoop/common/lib/*,/home/siyuan/hadoop/current/share/hadoop/hdfs/lib/*,/home/siyuan/hadoop/current/share/hadoop/yarn/*,/home/siyuan/hadoop/current/share/hadoop/yarn/lib/*</value> >>> > </property> >>> > >>> > And >>> > >>> > echo $HADOOP_CONF_DIR >>> > /home/siyuan/hadoop/current/etc/hadoop >>> > >>> > echo $YARN_CONF_DIR >>> > /home/siyuan/hadoop/current/etc/hadoop >>> > >>> > >>> > >>> > Best, >>> > Siyuan >>> > >>> > On Wed, Oct 29, 2014 at 8:48 AM, Pushkar Raste < >>> [email protected]> >>> > wrote: >>> > >>> > > @hsy541 - >>> > > This could be problem with your hadoop setup. I too had this problem >>> and >>> > > thought it was problem with Slider. Turned out that I hadn't >>> setfollowing >>> > > variable >>> > > HADOOP_CONF_DIR >>> > > YARN_CONF_DIR >>> > > >>> > > >>> > > One way to verify if your 'core-site.xml' and 'yar-site.xml' ... are >>> > being >>> > > read in Configuration, It to dump it using Configuration object from >>> > > SliderAppMaster class. Here is what I added to getClusterFS() >>> function in >>> > > SliderAppMaster.java >>> > > >>> > > public SliderFileSystem getClusterFS() throws IOException { >>> > > log.info("Config is {}", conf.toString()); >>> > > return new SliderFileSystem(conf); >>> > > >>> > > } >>> > > >>> > > >>> > > Oputput clearly indicated me that my core-site.xml yarn-site.xml and >>> > > hdfs-site.xml were not being read. On further investigation I >>> realized >>> > that >>> > > my HADOOP_CONF_DIR and YARN_CONF_DIR were not set. >>> > > >>> > > On Wed, Oct 29, 2014 at 8:32 AM, Steve Loughran < >>> [email protected]> >>> > > wrote: >>> > > >>> > > > On 27 October 2014 14:54, Pushkar Raste <[email protected]> >>> > wrote: >>> > > > >>> > > > > I am trying to to deploy memcached package using slider (0.40.0 >>> > > > version). I >>> > > > > keep getting "Wrong FS: >>> > > > > hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: >>> > > file:///" >>> > > > > error >>> > > > > >>> > > > >>> > > > >>> > > > That's the warning hadoop gives when the filesystem is different >>> from >>> > the >>> > > > path URL handed in. Here something thinks the filesystem is >>> file://, >>> > but >>> > > > the path to the cluster is >>> > > > hdfs://localhost:9000/user/root/.slider/cluster/c100 >>> > > > >>> > > > Now, file:// is the default filesystem of Hadoop —its presence is >>> > warning >>> > > > sign that that the configuration that the application has loaded >>> is not >>> > > > picking up the core-site.xml of that cluster. >>> > > > >>> > > > We've been changing bin/slider & slider.py to pick up the local >>> Hadoop >>> > > > config dirs if present, but I suspect this message may actually be >>> > coming >>> > > > from the deployed slider application master. If that's the case, >>> the >>> > > > probable cause is that whatever classpath it is using to pick up >>> the >>> > > hadoop >>> > > > JARs, this path does not include the Hadoop configuration >>> directory. >>> > > > >>> > > > -- >>> > > > CONFIDENTIALITY NOTICE >>> > > > NOTICE: This message is intended for the use of the individual or >>> > entity >>> > > to >>> > > > which it is addressed and may contain information that is >>> confidential, >>> > > > privileged and exempt from disclosure under applicable law. If the >>> > reader >>> > > > of this message is not the intended recipient, you are hereby >>> notified >>> > > that >>> > > > any printing, copying, dissemination, distribution, disclosure or >>> > > > forwarding of this communication is strictly prohibited. If you >>> have >>> > > > received this communication in error, please contact the sender >>> > > immediately >>> > > > and delete it from your system. Thank You. >>> > > > >>> > > >>> > >>> >>> -- >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to >>> which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified >>> that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender >>> immediately >>> and delete it from your system. Thank You. >>> >> >> >
