Are you using the develop branch or release 0.40? On Tue, Oct 28, 2014 at 7:02 PM, [email protected] <[email protected]> wrote:
> I try to run kafka as slider application > > > On Tue, Oct 28, 2014 at 7:01 PM, [email protected] <[email protected]> > wrote: > > > I had the same problem > > > > This is my appConfig.json > > > > { > > "schema": "http://example.org/specification/v2.0.0";, > > "metadata": { > > }, > > "global": { > > "application.def": > > "hdfs://localhost:9000/user/siyuan/slider_kafka.zip", > > "java_home": "/usr/lib/jvm/java-7-oracle/", > > "package_list": "files/kafka_2.10-0.8.1.1.tgz", > > > > "site.global.app_user": "siyuan", > > "site.global.app_root": > "${AGENT_WORK_ROOT}/app/install/slider_kafka", > > "site.global.pid_file": "${AGENT_WORK_ROOT}/app/run/component.pid", > > > > "site.global.additional_cp": "/usr/lib/hadoop/lib/*", > > "site.global.xmx_val": "256m", > > "site.global.xms_val": "128m" > > }, > > "components": { > > "slider-appmaster": { > > "jvm.heapsize": "256M" > > } > > } > > } > > > > On Mon, Oct 27, 2014 at 8:06 AM, Jon Maron <[email protected]> > wrote: > > > >> I think the problem is probably in your appConfig file (e.g. referencing > >> an application package from the the local filesystem as opposed to > HDFS). > >> Can you attach that? > >> > >> — Jon > >> > >> On Oct 27, 2014, at 10:54 AM, Pushkar Raste <[email protected]> > >> wrote: > >> > >> > Hi, > >> > I am trying to to deploy memcached package using slider (0.40.0 > >> version). I > >> > keep getting "Wrong FS: > >> > hdfs://localhost:9000/user/root/.slider/cluster/c100, expected: > >> file:///" > >> > error > >> > > >> > I saw that 'Rui Zhang' had similar issue, but she got around by > >> > setting SLIDER_CLASSPATH_EXTRA. > >> > Unfortunately that solution did not work for me. Here is my setup > >> > 1. slider-client.xml > >> > <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" > >> > href="configuration.xsl"?> <!-- Licensed to the Apache Software > >> Foundation > >> > (ASF) under one or more contributor license agreements. See the NOTICE > >> file > >> > distributed with this work for additional information regarding > >> copyright > >> > ownership. The ASF licenses this file to You under the Apache License: > >> > Version 2.0 (the "License"); you may not use this file except in > >> compliance > >> > with the License. You may obtain a copy of the License at > >> > http://www.apache.org/licenses/LICENSE-2.0 Unless required by > >> applicable > >> > law or agreed to in writing: software distributed under the License is > >> > distributed on an "AS IS" BASIS: WITHOUT WARRANTIES OR CONDITIONS OF > ANY > >> > KIND: either express or implied. See the License for the specific > >> language > >> > governing permissions and limitations under the License. --> <!-- > >> > Properties set here are picked up in the client. --> <configuration> > >> > <property> <name>slider.client.resource.origin</name> > >> > <value>conf/slider-client.xml</value> <description>This is just for > >> > diagnostics</description> </property> <property> > >> > <name>yarn.log-aggregation-enable</name> <value>true</value> > </property> > >> > <property> <name>slider.yarn.queue</name> <value>default</value> > >> > <description>YARN queue for the Application Master</description> > >> > </property> <!-- <property> <name>yarn.resourcemanager.address</name> > >> > <value>master:8032</value> </property> <property> > >> <name>fs.defaultFS</name> > >> > <value>hdfs://master:9090</value> </property> <property> > >> > <name>yarn.resourcemanager.principal</name> > >> > <value>yarn/master@MINICLUSTER</value> > >> > </property> <property> <name>slider.security.enabled</name> > >> > <value>true</value> </property> <property> > >> > <name>dfs.namenode.kerberos.principal</name> > >> > <value>hdfs/master@MINICLUSTER</value> > >> > </property> --> <property> <name>yarn.application.classpath</name> > >> > > >> > <value>/usr/local/hadoop/etc/hadoop/*:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/*</value> > >> > </property> <property> <name>slider.zookeeper.quorum</name> > >> > <value>localhost:2181</value> </property> <property> > >> > <name>yarn.resourcemanager.address</name> > <value>localhost:8032</value> > >> > </property> <property> > >> <name>yarn.resourcemanager.scheduler.address</name> > >> > <value>localhost:8030</value> </property> <property> > >> > <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> > >> </property> > >> > </configuration> > >> > > >> > 2. Environment variables in my ~/.bashrc > >> > export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export > >> > HADOOP_INSTALL=/usr/local/hadoop export PATH=$PATH:$HADOOP_INSTALL/bin > >> > export PATH=$PATH:$HADOOP_INSTALL/sbin export > >> > HADOOP_MAPRED_HOME=$HADOOP_INSTALL export > >> > HADOOP_COMMON_HOME=$HADOOP_INSTALL export > >> HADOOP_HDFS_HOME=$HADOOP_INSTALL > >> > export YARN_HOME=$HADOOP_INSTALL export > >> > HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native export > >> > HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib" export > >> > HADOOP_CONF_DIR=$HADOOP_INSTALL/etc/hadoop export > >> > > >> > SLIDER_CLASSPATH_EXTRA=$HADOOP_CONF_DIR/*:/usr/local/hadoop/etc/hadoop/*:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/* > >> > export YARN_CONF_DIR=$HADOOP_INSTALL/etc/hadoop > >> > > >> > > >> > > >> > 3. Output when I run 'python slider.py version' slider_home = > >> > "/usr/local/slider/slider-0.40" slider_jvm_opts = > >> > "-Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Xmx256m > >> > -Djava.confdir=/usr/local/slider/slider-0.40/conf" slider_classpath = > >> > > >> > "/usr/local/slider/slider-0.40/lib/*:/usr/local/slider/slider-0.40/conf:/usr/local/hadoop/etc/hadoop/*:/usr/local/hadoop/etc/hadoop/*:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/*" > >> > ready to exec : ['java', '-Djava.net.preferIPv4Stack=true', > >> > '-Djava.awt.headless=true', '-Xmx256m', > >> > '-Djava.confdir=/usr/local/slider/slider-0.40/conf', '-classpath', > >> > > >> > '/usr/local/slider/slider-0.40/lib/*:/usr/local/slider/slider-0.40/conf:/usr/local/hadoop/etc/hadoop/*:/usr/local/hadoop/etc/hadoop/*:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/*', > >> > 'org.apache.slider.Slider', 'version'] SLF4J: Class path contains > >> multiple > >> > SLF4J bindings. 2014-10-27 14:52:17,995 [main] INFO client.RMProxy - > >> > Connecting to ResourceManager at localhost/127.0.0.1:8032 SLF4J: > Found > >> > binding in > >> > > >> > [jar:file:/usr/local/slider/slider-0.40/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] > >> > 2014-10-27 14:52:18,121 [main] INFO client.SliderClient - Slider > >> Core-0.40 > >> > Built against commit# ddf7338759 on Java 1.7.0_67 by praste SLF4J: > Found > >> > binding in > >> > > >> > [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > >> > 2014-10-27 14:52:18,125 [main] INFO client.SliderClient - Compiled > >> against > >> > Hadoop 2.4.0 SLF4J: See > >> http://www.slf4j.org/codes.html#multiple_bindings > >> > for an explanation. SLF4J: Actual binding is of type > >> > [org.slf4j.impl.Log4jLoggerFactory] 2014-10-27 14:52:18,129 [main] > INFO > >> > client.SliderClient - Hadoop runtime version branch-2.4.0 with source > >> > checksum 375b2832a6641759c6eaf6e3e998147 and build date > >> 2014-03-31T08:31Z > >> > 2014-10-27 14:52:18,133 [main] INFO util.ExitUtil - Exiting with > status > >> 0 > >> > >> > >> -- > >> CONFIDENTIALITY NOTICE > >> NOTICE: This message is intended for the use of the individual or entity > >> to > >> which it is addressed and may contain information that is confidential, > >> privileged and exempt from disclosure under applicable law. If the > reader > >> of this message is not the intended recipient, you are hereby notified > >> that > >> any printing, copying, dissemination, distribution, disclosure or > >> forwarding of this communication is strictly prohibited. If you have > >> received this communication in error, please contact the sender > >> immediately > >> and delete it from your system. Thank You. > >> > > > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
