Thanks Billie for the quick and thoughtful reply.
You are correct, I'm testing in a single node docker container and have slider
built from source and installed during the docker build action.
Regarding the file systems (in the docker container) I want to be precise and
make sure that I understand what needs to be in the hdfs and what needs to be
on the local linux (to the docker container) file system.
I have not had much luck in getting slider to pickup code or configuration data
from HDFS, is there a setting I need to apply to do this?
Also does slider write any logs? Where would they be?
Regarding the protobuf-java-2.5.0.jar, I see it in the local (not HDFS)
directory: /usr/lib/slider/lib/
# ls /usr/lib/slider/lib/protobuf-java-2.5.0.jar
/usr/lib/slider/lib/protobuf-java-2.5.0.jar
Is this the correct location? Should I install these jars in HDFS, if so what
is the appropriate hdfs path?
Regarding jmemcached-1.0.0.zip, you are correct, I forgot to update the
jmemcached file name in the script (I updated to 1.0.1 since I built memcached
for source to create an uber jar using the maven assembly plugin), I should
have caught that.
The revised test script was updated to reflect this the install-package command
now has the correct version number in the zip name and now reads:
So now the script reads (the commented out lines using hdfs did not work, not
sure why, even when I change appConfig to use hdfs url paths):
#!/bin/bash
# this does not work when run during docker-build, we need to do this once we
are logged in
echo "setting up the enviornment variables"
env
export JAVA_HOME="/usr/lib/jvm/java"
export HADOOP_CONF="/etc/hadoop/conf"
env
echo "memcached built, registering memcached with slider and then running it"
/usr/lib/slider/bin/slider install-package --package
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
--name jmemcached --debug
#echo "using hdfs version, posting artifacts to hdfs"
#hadoop fs -put ./incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
./incubator-slider/app-packages/memcached/*.json /user/root/artifacts/memcached
#/usr/lib/slider/bin/slider install-package --package
hdfs://quickstart.cloudera/user/root/artifacts/memcached/jmemcached-1.0.1.zip
--name jmemcached --debug
echo "Listing applications - before starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
# startup memcached properly, yarn task name will be mymemcached
/usr/lib/slider/bin/slider create mymemcached --template
/tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources
/tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
--manager localhost:8032 --debug
#/usr/lib/slider/bin/slider create mymemcached --template
hdfs://quickstart.cloudera/user/root/artifacts/memcached/appConfig.json
--resources
hdfs://quickstart.cloudera/user/root/artifacts/memcached/resources-default.json
--manager localhost:8032 --debug
echo "Listing applications - after starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
echo "Finished $0"
appConfig.json reads:
{
"schema": "http://example.org/specification/v2.0.0";,
"metadata": {
},
"global": {
"agent.conf": "/tmp/mybuild/agent/conf/agent.ini",
"application.def":
"/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip",
"java_home": "/usr/lib/jvm/java/",
"site.global.additional_cp":
"/tmp/mybuild/incubator-slider/slider-assembly/target/lib/",
"site.global.xmx_val": "256m",
"site.global.xms_val": "128m",
"site.global.memory_val": "200M",
"site.global.listen_port": "${MEMCACHED.ALLOCATED_PORT}{PER_CONTAINER}"
},
"components": {
"slider-appmaster": {
"jvm.heapsize": "256M"
}
}
}
/tmp/mybuild/agent/conf/agent.ini reads:
app_task_dir=.
app_log_dir=.
app_tmp_dir=app/tmp
log_dir=.
run_dir=infra/run
version_file=infra/version
;log_level=INFO
log_level=DEBUG
[python]
[command]
max_retries=2
sleep_between_retries=1
auto_restart=5,5
[security]
[heartbeat]
state_interval=6
log_lines_count=300
And then running the corrected script I see:
+ bash -x ./test_slider.sh
+ echo 'setting up the enviornment variables'
setting up the enviornment variables
+ env
HOSTNAME=quickstart.cloudera
TERM=xterm
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/tmp/mybuild
LANG=en_US.UTF-8
SHLVL=3
HOME=/root
no_proxy=*.local, 169.254/16
LESSOPEN=||/usr/bin/lesspipe.sh %s
container=docker
_=/usr/bin/env
+ export JAVA_HOME=/usr/lib/jvm/java
+ JAVA_HOME=/usr/lib/jvm/java
+ export HADOOP_CONF=/etc/hadoop/conf
+ HADOOP_CONF=/etc/hadoop/conf
+ env
HOSTNAME=quickstart.cloudera
TERM=xterm
HADOOP_CONF=/etc/hadoop/conf
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/tmp/mybuild
JAVA_HOME=/usr/lib/jvm/java
LANG=en_US.UTF-8
SHLVL=3
HOME=/root
no_proxy=*.local, 169.254/16
LESSOPEN=||/usr/bin/lesspipe.sh %s
container=docker
_=/usr/bin/env
+ echo 'memcached built, registering memcached with slider and then running it'
memcached built, registering memcached with slider and then running it
+ /usr/lib/slider/bin/slider install-package --package
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
--name jmemcached --debug --replacepkg
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:12,948 [main] INFO client.RMProxy - Connecting to
ResourceManager at /0.0.0.0:8032
2017-04-21 19:22:13,093 [main] INFO client.SliderClient - Installing package
file:/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
at file:/root/.slider/package/jmemcached/jmemcached-1.0.1.zip and overwrite is
true.
2017-04-21 19:22:13,109 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Listing applications - before starting jmemcached'
Listing applications - before starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:15,070 [main] INFO client.RMProxy - Connecting to
ResourceManager at localhost/127.0.0.1:8032
2017-04-21 19:22:15,222 [main] INFO util.ExitUtil - Exiting with status 0
+ /usr/lib/slider/bin/slider create mymemcached --template
/tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources
/tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
--manager localhost:8032 --debug
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:17,183 [main] INFO client.RMProxy - Connecting to
ResourceManager at localhost/127.0.0.1:8032
2017-04-21 19:22:17,661 [main] INFO agent.AgentClientProvider - Validating app
definition
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:17,662 [main] INFO agent.AgentUtils - Reading metainfo at
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:17,685 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 2202
2017-04-21 19:22:17,909 [main] INFO client.SliderClient - No credentials
requested
2017-04-21 19:22:18,108 [main] INFO agent.AgentUtils - Reading metainfo at
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:18,116 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 2202
2017-04-21 19:22:18,177 [main] INFO launch.AbstractLauncher - Log include
patterns:
2017-04-21 19:22:18,177 [main] INFO launch.AbstractLauncher - Log exclude
patterns:
2017-04-21 19:22:18,938 [main] INFO slideram.SliderAMClientProvider - Loading
all dependencies for AM.
2017-04-21 19:22:18,939 [main] INFO tools.SliderUtils - Loading all
dependencies from /usr/lib/slider/lib
2017-04-21 19:22:20,687 [main] INFO agent.AgentClientProvider - Automatically
uploading the agent tarball at
file:/root/.slider/cluster/mymemcached/tmp/application_1492799085220_0003/agent
2017-04-21 19:22:20,727 [main] INFO agent.AgentClientProvider - Validating app
definition
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:20,727 [main] INFO agent.AgentUtils - Reading metainfo at
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:20,735 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 2202
2017-04-21 19:22:20,807 [main] INFO Configuration.deprecation -
slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
2017-04-21 19:22:20,813 [main] INFO launch.AppMasterLauncher - Submitting
application to Resource Manager
2017-04-21 19:22:20,838 [main] INFO impl.YarnClientImpl - Submitted
application application_1492799085220_0003
2017-04-21 19:22:20,845 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Listing applications - after starting jmemcached'
Listing applications - after starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:22,857 [main] INFO client.RMProxy - Connecting to
ResourceManager at localhost/127.0.0.1:8032
mymemcached ACCEPTED application_1492799085220_0003
2017-04-21 19:22:23,142 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Finished ./test_slider.sh'
Finished ./test_slider.sh
I am still getting the dreaded failure as reported in the yarn resource manager:
Application application_1492799085220_0003 failed 2 times due to AM Container
for appattempt_1492799085220_0003_000002 exited with exitCode: -1000
For more detailed output, check application tracking
page:http://quickstart.cloudera:8088/proxy/application_1492799085220_0003/Then,
click on links to logs of each attempt.
Diagnostics: File
file:/root/.slider/cluster/mymemcached/tmp/application_1492799085220_0003/am/lib/protobuf-java-2.5.0.jar
does not exist
java.io.FileNotFoundException: File
file:/root/.slider/cluster/mymemcached/tmp/application_1492799085220_0003/am/lib/protobuf-java-2.5.0.jar
does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:542)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:755)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:532)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Failing this attempt. Failing the application.
With best regards:
Bill
________________________________
From: Billie Rinaldi <[email protected]>
Sent: Wednesday, April 19, 2017 5:36 PM
To: [email protected]
Subject: Re: Trying to get he memcached example to run for a new build
The protobuf-java-2.5.0.jar should be in the Slider lib directory. It will
get pushed out to the AM lib directory from there.
There are some other issues with the script output. The install package
command has not succeeded, and says "ERROR main.ServiceLauncher - Unable to
access supplied pkg file at /tmp/mybuild/incubator-slider/
app-packages/memcached/jmemcached-1.0.0.zip"
Also the line INFO agent.AgentClientProvider - Validating app definition
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
indicates that it is reading the app definition from the local filesystem,
when it should be reading it from HDFS. Maybe since this is a single VM
setup it is able to continue despite these errors, but if you were running
on a real cluster you'd have to make sure install package was working and
fix the app definition path in the appConfig.
On Tue, Apr 18, 2017 at 6:35 PM, Foolish Ewe <[email protected]> wrote:
> Hello All:
>
>
> I'm trying to run the memcached example locally but it fails pretty quick
> and claims "protobuf-java-2.5.0.jar does not exist".
>
>
> Consider the following script (test-slider.sh):
>
> #!/bin/bash
>
> # this does not work when run during docker-build, we need to do this once
> we are logged in
>
> echo "memcached built, registering memcached with slider and then running
> it"
>
> /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
> --name jmemcached --debug
>
> echo "Listing applications - before starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> # startup memcached properly, yarn task name will be mymemcached
>
> /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources
> /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> echo "Listing applications - after starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> echo "Finished $0"
>
> I am having difficulty in locating the logs after the failure, any idea
> where they would be? yarn logs cannot find them, and I'm not having luck
> with the name node. I did modify the yarn-site.xml read at startup to
> include the lines:
>
> <!-- Begin modifications for debugging slider -->
>
> <!-- 60 minutes after a failure to see what is left in the directory-->
>
> <property>
>
> <name>yarn.nodemanager.delete.debug-delay-sec</name>
>
> <value>3600</value>
>
> </property>
>
>
> <!--time before the process gets a -9 (Should it be 30 seconds?)-->
>
> <property>
>
> <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
>
> <value>3600000</value>
>
> </property>
>
> <!-- End modifications for debugging slider -->
>
>
> Regarding the error, I'm seeing, the slider install-package appears to
> work correctly from what I can tell, but the slider create encounters run
> time errors when launched, it seems to be looking for
> protobuf-java-2.5.0.jar. I tried making fat jars for the memcached
> application and the various slider packages, to no avail. Any ideas how to
> resolve these errors reported in the resource manager:
> Application application_1492557996590_0002 failed 2 times due to AM
> Container for appattempt_1492557996590_0002_000002 exited with exitCode:
> -1000
> For more detailed output, check application tracking page:
> http://quickstart.cloudera:8088/proxy/application_1492557996590_0002/Then,
> click on links to logs of each attempt.
> Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_
> 1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
> java.io.FileNotFoundException: File file:/root/.slider/cluster/
> mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar
> does not exist
> at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(
> RawLocalFileSystem.java:542)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(
> RawLocalFileSystem.java:755)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(
> RawLocalFileSystem.java:532)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(
> FilterFileSystem.java:425)
> at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1693)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Failing this attempt. Failing the application.
>
>
> The test-slider.sh script's output is as follows:
>
> # bash -x /tmp/mybuild/test_slider.sh
>
> + echo 'memcached built, registering memcached with slider and then
> running it'
>
> memcached built, registering memcached with slider and then running it
>
> + /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
> --name jmemcached --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were
found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to
> ResourceManager at /0.0.0.0:8032
>
> 2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to
> access supplied pkg file at /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.0.zip
>
> 2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status 40
>
> + echo 'Listing applications - before starting jmemcached'
>
> Listing applications - before starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were
found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status 0
>
> + /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources
> /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were
found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials
> requested
>
> 2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log include
> patterns:
>
> 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log exclude
> patterns:
>
> 2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider -
> Loading all dependencies for AM.
>
> 2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all
> dependencies from /usr/lib/slider/lib
>
> 2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider -
> Automatically uploading the agent tarball at file:/root/.slider/cluster/
> mymemcached/tmp/application_1492557996590_0003/agent
>
> 2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation -
> slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
>
> 2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher - Submitting
> application to Resource Manager
>
> 2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted
> application application_1492557996590_0003
>
> 2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Listing applications - after starting jmemcached'
>
> Listing applications - after starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were
found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> mymemcached ACCEPTED application_1492557996590_0003
>
> 2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Finished /tmp/mybuild/test_slider.sh'
>
> Finished /tmp/mybuild/test_slider.sh
>
>
> Thanks:
>
>
> Bill
>
