Thanks Sumit.
I use the complete path and it can submit the application to YARN now.
But the AM container failed very soon and I put the complete error info at
the bottom.
I use the appConfig.json from the created application package and replace
some tags. (e.g., the application.def, java_home, agent.conf and
zookeeper.quorum).
The complete file is shown below. Do I miss something?
*appConfig.json:*
{
"schema": "http://example.org/specification/v2.0.0";,
"metadata": {
},
"global": {
"application.def":
"/user/hustnn/slider-hbase-app-package-0.98.5-hadoop2.zip",
"create.default.zookeeper.node": "true",
"java_home": "/users/staff/hustnn/hadoop-0.23.6/java/jdk1.6.0_39",
"system_configs": "core-site",
"agent.conf": "/user/hustnn/agent/conf/agent.ini",
"site.global.app_user": "yarn",
"site.global.app_root":
"${AGENT_WORK_ROOT}/app/install/hbase-0.98.5-hadoop2",
"site.global.ganglia_server_host": "${NN_HOST}",
"site.global.ganglia_server_port": "8667",
"site.global.ganglia_server_id": "Application1",
"site.global.ganglia_enabled":"true",
"site.global.hbase_instance_name": "instancename",
"site.global.hbase_root_password": "secret",
"site.global.user_group": "hadoop",
"site.global.security_enabled": "false",
"site.global.monitor_protocol": "http",
"site.global.hbase_thrift_port": "${HBASE_THRIFT.ALLOCATED_PORT}",
"site.global.hbase_thrift2_port": "${HBASE_THRIFT2.ALLOCATED_PORT}",
"site.global.hbase_rest_port": "${HBASE_REST.ALLOCATED_PORT}",
"site.hbase-env.hbase_master_heapsize": "1024m",
"site.hbase-env.hbase_regionserver_heapsize": "1024m",
"site.hbase-site.hbase.rootdir": "${DEFAULT_DATA_DIR}",
"site.hbase-site.hbase.superuser": "yarn",
"site.hbase-site.hbase.tmp.dir": "${AGENT_WORK_ROOT}/work/app/tmp",
"site.hbase-site.hbase.local.dir": "${hbase.tmp.dir}/local",
"site.hbase-site.hbase.zookeeper.quorum": "155.69.148.21:2181",
"site.hbase-site.zookeeper.znode.parent": "${DEF_ZK_PATH}",
"site.hbase-site.hbase.regionserver.info.port": "0",
"site.hbase-site.hbase.master.info.port":
"${HBASE_MASTER.ALLOCATED_PORT}",
"site.hbase-site.hbase.regionserver.port": "0",
"site.hbase-site.hbase.master.port": "0"
},
"components": {
"slider-appmaster": {
"jvm.heapsize": "256M"
}
}
}
*The error log of AM container:*
14/09/11 21:07:24 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/09/11 21:07:25 INFO appmaster.SliderAppMaster: Login user is hustnn
(auth:SIMPLE)
14/09/11 21:07:25 WARN appmaster.SliderAppMaster:
14/09/11 21:07:25 INFO appmaster.SliderAppMaster:
14/09/11 21:07:25 INFO appmaster.SliderAppMaster: OpenSSL 0.9.8e-fips-rhel5
01 Jul 2008
14/09/11 21:07:25 WARN appmaster.SliderAppMaster:
14/09/11 21:07:25 WARN appmaster.SliderAppMaster: Unknown option: --
14/09/11 21:07:25 WARN appmaster.SliderAppMaster: usage: python [option]
... [-c cmd | -m mod | file | -] [arg] ...
14/09/11 21:07:25 WARN appmaster.SliderAppMaster: Try `python -h' for more
information.
14/09/11 21:07:25 INFO appmaster.SliderAppMaster:
14/09/11 21:07:25 INFO service.AbstractService: Service python failed in
state STARTED; cause: org.apache.slider.core.main.ServiceLaunchException:
python failed with code 2
org.apache.slider.core.main.ServiceLaunchException: python failed with code
2
at
org.apache.slider.server.services.workflow.ForkedProcessService.reportFailure(ForkedProcessService.java:202)
at
org.apache.slider.server.services.workflow.ForkedProcessService.onProcessExited(ForkedProcessService.java:192)
at
org.apache.slider.server.services.workflow.LongLivedProcess.run(LongLivedProcess.java:345)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
14/09/11 21:07:25 WARN tools.SliderUtils: Expected exit code={0}, actual
exit code={2}
14/09/11 21:07:25 INFO tools.SliderUtils: [ERR]
14/09/11 21:07:25 INFO tools.SliderUtils: [ERR] Unknown option: --
14/09/11 21:07:25 INFO tools.SliderUtils: [ERR] usage: python [option] ...
[-c cmd | -m mod | file | -] [arg] ...
14/09/11 21:07:25 INFO tools.SliderUtils: [ERR] Try `python -h' for more
information.
14/09/11 21:07:25 INFO tools.SliderUtils: [OUT]
14/09/11 21:07:25 INFO service.AbstractService: Service SliderAppMaster
failed in state INITED; cause:
org.apache.slider.core.exceptions.SliderException: Process python failed:
Expected exit code={0}, actual exit code={2}
org.apache.slider.core.exceptions.SliderException: Process python failed:
Expected exit code={0}, actual exit code={2}
at
org.apache.slider.common.tools.SliderUtils.execCommand(SliderUtils.java:1744)
at
org.apache.slider.common.tools.SliderUtils.validateSliderServerEnvironment(SliderUtils.java:1777)
at
org.apache.slider.server.appmaster.SliderAppMaster.serviceInit(SliderAppMaster.java:405)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:180)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:471)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:401)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:626)
at
org.apache.slider.server.appmaster.SliderAppMaster.main(SliderAppMaster.java:1897)
Exception: org.apache.slider.core.exceptions.SliderException: Process
python failed: Expected exit code={0}, actual exit code={2}
14/09/11 21:07:25 ERROR main.ServiceLauncher: Exception:
org.apache.slider.core.exceptions.SliderException: Process python failed:
Expected exit code={0}, actual exit code={2}
org.apache.hadoop.service.ServiceStateException:
org.apache.slider.core.exceptions.SliderException: Process python failed:
Expected exit code={0}, actual exit code={2}
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:180)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:471)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:401)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:626)
at
org.apache.slider.server.appmaster.SliderAppMaster.main(SliderAppMaster.java:1897)
Caused by: org.apache.slider.core.exceptions.SliderException: Process
python failed: Expected exit code={0}, actual exit code={2}
at
org.apache.slider.common.tools.SliderUtils.execCommand(SliderUtils.java:1744)
at
org.apache.slider.common.tools.SliderUtils.validateSliderServerEnvironment(SliderUtils.java:1777)
at
org.apache.slider.server.appmaster.SliderAppMaster.serviceInit(SliderAppMaster.java:405)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
... 5 more
14/09/11 21:07:25 INFO util.ExitUtil: Exiting with status 32
2014-09-11 21:33 GMT+08:00 Sumit Mohanty <[email protected]>:
> Think of the slider-agent as a bridge between AppMaster and the application
> workload (HBase Master, HBase RegionServer). *The need to explicitly pass
> its path is being removed in the upcoming release as its required by
> default.*
>
> The json files are application configurations - appConfig.json is the
> configuration required for the application (as well as some details needed
> by Slider itself) and the resources.json file is the application
> requirement from YARN - e.g. how much memory.
>
> So in your case, you will need to use the files for HBase. However, you
> need to use the ones from the created application package (not from the
> repo). There are some tokens that get replaced when you invoke the mvn
> command to create the package.
>
> On Thu, Sep 11, 2014 at 2:14 AM, 牛兆捷 <[email protected]> wrote:
>
> > @Shivaji
> > Please try and give the complete path to your files. It should work.
> >
> > Do you mean specify the complete path of the two files in
> > app-packages\hbase path? But I think those files are specific for Hbase
> not
> > for slider-agent.tar.gz.
> >
> > What is the relationship between Hbase and the agent?
> >
> > 2014-09-10 22:02 GMT+08:00 Shivaji <[email protected]>:
> >
> > > Please try and give the complete path to your files. It should work.
> > >
> > > Sent from my iPhone
> > >
> > > > On Sep 10, 2014, at 6:23 AM, 牛兆捷 <[email protected]> wrote:
> > > >
> > > > In "http://slider.incubator.apache.org/docs/getting_started.html";,
> > > >
> > > > cd ${slider-install-dir}/slider-0.40.0/bin
> > > >
> > > > ./slider create cl1 --image
> > > > hdfs://yourNameNodeHost:8020/user/yarn/agent/slider-agent.tar.gz
> > > > --template appConfig.json --resources resources.json
> > > >
> > > > The appConfig.json and resources.json are not found in the bin path.
> > > When I
> > > > execute it, it does show that the file cannot be found.
> > > >
> > > >
> > > >
> > > > --
> > > > *Regards,*
> > > > *Zhaojie*
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> >
> >
> > --
> > *Regards,*
> > *Zhaojie*
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
*Regards,*
*Zhaojie*
