[jira] [Created] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
Liu You created MYRIAD-255:
--
Summary: unable to run yarn application with hadoop2.7.3 and
mesos1.2.0
Key: MYRIAD-255
URL: https://issues.apache.org/jira/browse/MYRIAD-255
Project: Myriad
Issue Type: Bug
Components: Executor, Scheduler
Affects Versions: Myriad 0.2.0
Environment: Ubuntu 14.04.3 LTS
master hostname: s18
slaves hostname: s19, s20, s21
hadoop 2.7.3
mesos 1.2.0
Reporter: Liu You
Attachments: mapred-site.xml,
mesos-master.s18.mesos.log.INFO.20170720-173553.21812,
mesos-master.s18.mesos.log.WARNING.20170720-173553.21812, Mesos Web UI.png,
myriad-config-default.yml, myriad ui-1.png, myriad ui-2.png, stucked job.png,
yarn-mesos-resourcemanager-s18.log, yarn-mesos-resourcemanager-s18.out,
yarn-site.xml
I can't run yarn application, and the job gets stucked every time I execute the
command 'hadoop jar xxx.'. Please help me. Thank you very much.
!stucked job.png|thumbnail!
Myriad Web UI:
!myriad ui-1.png|thumbnail!
!myriad ui-2.png|thumbnail!
Mesos Web UI:
!Mesos Web UI.png|thumbnail!
yarn-mesos-resourcemanager-s18.log:
{code:java}
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on 8032: starting
2017-07-20 17:36:46,955 INFO
org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
value:
"myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
lost with exit status: 9
2017-07-20 17:36:46,969 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
TASK_FAILED
2017-07-20 17:36:46,970 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to
active state
2017-07-20 17:36:46,971 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked as
pending failed task with id value:
"nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
{code}
myriad-config-default.yml:
{code:java}
mesosMaster: 192.168.130.18:5050
checkpoint: false
frameworkFailoverTimeout: 4320
frameworkName: MyriadAlpha
frameworkRole: "yarn"
frameworkUser: mesos # User the Node Manager runs as, required if
nodeManagerURI set, otherwise defaults to the user
# running the resource manager.
frameworkSuperUser: root # To be deprecated, currently permissions need set by
a superuser due to Mesos-1790. Must be
# root or have passwordless sudo. Required if
nodeManagerURI set, ignored otherwise.
#nativeLibrary: /usr/local/lib/libmesos.so
nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
zkServers: 192.168.130.18:2181
zkTimeout: 2
restApiPort: 8192
#servedConfigPath: dist/config.tgz
servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
#servedBinaryPath: dist/hadoop-2.6.0.tgz
servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
profiles:
zero: # NMs launched with this profile dynamically obtain cpu/mem from Mesos
cpu: 0
mem: 0
small:
cpu: 2
mem: 2048
medium:
cpu: 4
mem: 4096
large:
cpu: 10
mem: 12288
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
profile.
medium: 1 #
rebalancer: false
haEnabled: false
nodemanager:
jvmMaxMemoryMB: 1024
cpus: 0.2
cgroups: false
executor:
jvmMaxMemoryMB: 256
path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
#path:
file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
#The following should be used for a remotely distributed URI, hdfs assumed
but other URI types valid.
#nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
#configUri: http://s18:8192/api/artifacts/config.tgz
#jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
yarnEnvironment:
YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
#HADOOP_CONF_DIR=config
#HADOOP_TMP_DIR=$MESOS_SANDBOX
#YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
#JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes
necessary
#JAVA_HOME: /home/mesos/jdk/jdk1.8.0_131
#JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox
directory
#JAVA_LIBRARY_PATH: /opt/mycompany/lib
#mesosAuthenticationPrincipal:
#mesosAuthenticationSecretFilename:
{code}
yarn-site.xml:
{code:java}
yarn.resourcemanager.hostname
s18
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.nodemanager.aux-services.mapreduce_shuffle.class
or
[jira] [Updated] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu You updated MYRIAD-255:
---
Description:
I can't run yarn application, and the job gets stucked every time I execute the
command 'hadoop jar xxx.'. Please help me. Thank you very much.
!stucked job.png|thumbnail!
Myriad Web UI:
!myriad ui-1.png|thumbnail!
!myriad ui-2.png|thumbnail!
Mesos Web UI:
!Mesos Web UI.png|thumbnail!
yarn-mesos-resourcemanager-s18.log:
{code:java}
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on 8032: starting
2017-07-20 17:36:46,955 INFO
org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
value:
"myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
lost with exit status: 9
2017-07-20 17:36:46,969 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
TASK_FAILED
2017-07-20 17:36:46,970 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to
active state
2017-07-20 17:36:46,971 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked as
pending failed task with id value:
"nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
{code}
myriad-config-default.yml:
{code:java}
mesosMaster: 192.168.130.18:5050
checkpoint: false
frameworkFailoverTimeout: 4320
frameworkName: MyriadAlpha
frameworkRole: "yarn"
frameworkUser: mesos # User the Node Manager runs as, required if
nodeManagerURI set, otherwise defaults to the user
# running the resource manager.
frameworkSuperUser: root # To be deprecated, currently permissions need set by
a superuser due to Mesos-1790. Must be
# root or have passwordless sudo. Required if
nodeManagerURI set, ignored otherwise.
#nativeLibrary: /usr/local/lib/libmesos.so
nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
zkServers: 192.168.130.18:2181
zkTimeout: 2
restApiPort: 8192
#servedConfigPath: dist/config.tgz
servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
#servedBinaryPath: dist/hadoop-2.6.0.tgz
servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
profiles:
zero: # NMs launched with this profile dynamically obtain cpu/mem from Mesos
cpu: 0
mem: 0
small:
cpu: 2
mem: 2048
medium:
cpu: 4
mem: 4096
large:
cpu: 10
mem: 12288
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
profile.
medium: 1 #
rebalancer: false
haEnabled: false
nodemanager:
jvmMaxMemoryMB: 1024
cpus: 0.2
cgroups: false
executor:
jvmMaxMemoryMB: 256
path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
#path:
file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
#The following should be used for a remotely distributed URI, hdfs assumed
but other URI types valid.
#nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
#configUri: http://s18:8192/api/artifacts/config.tgz
#jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
yarnEnvironment:
YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
#HADOOP_CONF_DIR=config
#HADOOP_TMP_DIR=$MESOS_SANDBOX
#YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
#JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes
necessary
#JAVA_HOME: /home/mesos/jdk/jdk1.8.0_131
#JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox
directory
#JAVA_LIBRARY_PATH: /opt/mycompany/lib
#mesosAuthenticationPrincipal:
#mesosAuthenticationSecretFilename:
{code}
yarn-site.xml:
{code:java}
yarn.resourcemanager.hostname
s18
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService
yarn.nm.liveness-monitor.expiry-interval-ms
2000
yarn.am.liveness-monitor.expiry-interval-ms
1
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}
yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}
yarn.nodemanager.pmem-check-enabled
false
[jira] [Updated] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu You updated MYRIAD-255:
---
Description:
I can't run yarn application, and the job gets stucked every time I execute the
command 'hadoop jar xxx.'.
Please help me. Thank you very much.
!stucked job.png|thumbnail!
Myriad Web UI:
!myriad ui-1.png|thumbnail!
!myriad ui-2.png|thumbnail!
Mesos Web UI:
!Mesos Web UI.png|thumbnail!
yarn-mesos-resourcemanager-s18.log:
{code:java}
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on 8032: starting
2017-07-20 17:36:46,955 INFO
org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
value:
"myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
lost with exit status: 9
2017-07-20 17:36:46,969 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
TASK_FAILED
2017-07-20 17:36:46,970 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to
active state
2017-07-20 17:36:46,971 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked as
pending failed task with id value:
"nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
{code}
myriad-config-default.yml:
{code:java}
mesosMaster: 192.168.130.18:5050
checkpoint: false
frameworkFailoverTimeout: 4320
frameworkName: MyriadAlpha
frameworkRole: "yarn"
frameworkUser: mesos # User the Node Manager runs as, required if
nodeManagerURI set, otherwise defaults to the user
# running the resource manager.
frameworkSuperUser: root # To be deprecated, currently permissions need set by
a superuser due to Mesos-1790. Must be
# root or have passwordless sudo. Required if
nodeManagerURI set, ignored otherwise.
#nativeLibrary: /usr/local/lib/libmesos.so
nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
zkServers: 192.168.130.18:2181
zkTimeout: 2
restApiPort: 8192
#servedConfigPath: dist/config.tgz
servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
#servedBinaryPath: dist/hadoop-2.6.0.tgz
servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
profiles:
zero: # NMs launched with this profile dynamically obtain cpu/mem from Mesos
cpu: 0
mem: 0
small:
cpu: 2
mem: 2048
medium:
cpu: 4
mem: 4096
large:
cpu: 10
mem: 12288
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
profile.
medium: 1 #
rebalancer: false
haEnabled: false
nodemanager:
jvmMaxMemoryMB: 1024
cpus: 0.2
cgroups: false
executor:
jvmMaxMemoryMB: 256
path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
#path:
file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
#The following should be used for a remotely distributed URI, hdfs assumed
but other URI types valid.
#nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
#configUri: http://s18:8192/api/artifacts/config.tgz
#jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
yarnEnvironment:
YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
#HADOOP_CONF_DIR=config
#HADOOP_TMP_DIR=$MESOS_SANDBOX
#YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
#JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes
necessary
#JAVA_HOME: /home/mesos/jdk/jdk1.8.0_131
#JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox
directory
#JAVA_LIBRARY_PATH: /opt/mycompany/lib
#mesosAuthenticationPrincipal:
#mesosAuthenticationSecretFilename:
{code}
yarn-site.xml:
{code:java}
yarn.resourcemanager.hostname
s18
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService
yarn.nm.liveness-monitor.expiry-interval-ms
2000
yarn.am.liveness-monitor.expiry-interval-ms
1
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}
yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}
yarn.nodemanager.pmem-check-enabled
false
[jira] [Updated] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu You updated MYRIAD-255:
---
Description:
I can't run yarn application, and the job gets stucked every time I execute the
command 'hadoop jar xxx.'.
Please help me. Thank you very much.
The Myriad Web UI shows that the status of the task changes from staging to
pending, then staging, then pending.
There are some screenshots in the attachments below.
yarn-mesos-resourcemanager-s18.log:
{code:java}
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on 8032: starting
2017-07-20 17:36:46,955 INFO
org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
value:
"myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
lost with exit status: 9
2017-07-20 17:36:46,969 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
TASK_FAILED
2017-07-20 17:36:46,970 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to
active state
2017-07-20 17:36:46,971 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked as
pending failed task with id value:
"nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
{code}
myriad-config-default.yml:
{code:java}
mesosMaster: 192.168.130.18:5050
checkpoint: false
frameworkFailoverTimeout: 4320
frameworkName: MyriadAlpha
frameworkRole: "yarn"
frameworkUser: mesos # User the Node Manager runs as, required if
nodeManagerURI set, otherwise defaults to the user
# running the resource manager.
frameworkSuperUser: root # To be deprecated, currently permissions need set by
a superuser due to Mesos-1790. Must be
# root or have passwordless sudo. Required if
nodeManagerURI set, ignored otherwise.
#nativeLibrary: /usr/local/lib/libmesos.so
nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
zkServers: 192.168.130.18:2181
zkTimeout: 2
restApiPort: 8192
#servedConfigPath: dist/config.tgz
servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
#servedBinaryPath: dist/hadoop-2.6.0.tgz
servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
profiles:
zero: # NMs launched with this profile dynamically obtain cpu/mem from Mesos
cpu: 0
mem: 0
small:
cpu: 2
mem: 2048
medium:
cpu: 4
mem: 4096
large:
cpu: 10
mem: 12288
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
profile.
medium: 1 #
rebalancer: false
haEnabled: false
nodemanager:
jvmMaxMemoryMB: 1024
cpus: 0.2
cgroups: false
executor:
jvmMaxMemoryMB: 256
path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
#path:
file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
#The following should be used for a remotely distributed URI, hdfs assumed
but other URI types valid.
#nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
#configUri: http://s18:8192/api/artifacts/config.tgz
#jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
yarnEnvironment:
YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
#HADOOP_CONF_DIR=config
#HADOOP_TMP_DIR=$MESOS_SANDBOX
#YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
#JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes
necessary
#JAVA_HOME: /home/mesos/jdk/jdk1.8.0_131
#JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox
directory
#JAVA_LIBRARY_PATH: /opt/mycompany/lib
#mesosAuthenticationPrincipal:
#mesosAuthenticationSecretFilename:
{code}
yarn-site.xml:
{code:java}
yarn.resourcemanager.hostname
s18
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService
yarn.nm.liveness-monitor.expiry-interval-ms
2000
yarn.am.liveness-monitor.expiry-interval-ms
1
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}
yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}
yarn.nodemanager.pmem-
[jira] [Updated] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu You updated MYRIAD-255:
---
Description:
I can't run yarn application, and the job gets stucked every time I execute the
command 'hadoop jar xxx.'.
The Myriad Web UI shows that the status of the task changes from staging to
pending, then staging, then pending.
There are some screenshots in the attachments below.
Please help me. Thank you very much.
yarn-mesos-resourcemanager-s18.log:
{code:java}
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on 8032: starting
2017-07-20 17:36:46,955 INFO
org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
value:
"myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
lost with exit status: 9
2017-07-20 17:36:46,969 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
TASK_FAILED
2017-07-20 17:36:46,970 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to
active state
2017-07-20 17:36:46,971 INFO
org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked as
pending failed task with id value:
"nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
{code}
myriad-config-default.yml:
{code:java}
mesosMaster: 192.168.130.18:5050
checkpoint: false
frameworkFailoverTimeout: 4320
frameworkName: MyriadAlpha
frameworkRole: "yarn"
frameworkUser: mesos # User the Node Manager runs as, required if
nodeManagerURI set, otherwise defaults to the user
# running the resource manager.
frameworkSuperUser: root # To be deprecated, currently permissions need set by
a superuser due to Mesos-1790. Must be
# root or have passwordless sudo. Required if
nodeManagerURI set, ignored otherwise.
#nativeLibrary: /usr/local/lib/libmesos.so
nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
zkServers: 192.168.130.18:2181
zkTimeout: 2
restApiPort: 8192
#servedConfigPath: dist/config.tgz
servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
#servedBinaryPath: dist/hadoop-2.6.0.tgz
servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
profiles:
zero: # NMs launched with this profile dynamically obtain cpu/mem from Mesos
cpu: 0
mem: 0
small:
cpu: 2
mem: 2048
medium:
cpu: 4
mem: 4096
large:
cpu: 10
mem: 12288
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
profile.
medium: 1 #
rebalancer: false
haEnabled: false
nodemanager:
jvmMaxMemoryMB: 1024
cpus: 0.2
cgroups: false
executor:
jvmMaxMemoryMB: 256
path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
#path:
file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
#The following should be used for a remotely distributed URI, hdfs assumed
but other URI types valid.
#nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
#configUri: http://s18:8192/api/artifacts/config.tgz
#jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
yarnEnvironment:
YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
#HADOOP_CONF_DIR=config
#HADOOP_TMP_DIR=$MESOS_SANDBOX
#YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
#JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes
necessary
#JAVA_HOME: /home/mesos/jdk/jdk1.8.0_131
#JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox
directory
#JAVA_LIBRARY_PATH: /opt/mycompany/lib
#mesosAuthenticationPrincipal:
#mesosAuthenticationSecretFilename:
{code}
yarn-site.xml:
{code:java}
yarn.resourcemanager.hostname
s18
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService
yarn.nm.liveness-monitor.expiry-interval-ms
2000
yarn.am.liveness-monitor.expiry-interval-ms
1
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}
yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}
yarn.nodemanager.pmem-
[jira] [Commented] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095683#comment-16095683
]
Liu You commented on MYRIAD-255:
Thanks for your reply.
Strangely, the stdout and stderr from the task logs in the mesos ui is empty,
0 size and no content.
I read some log files related to mesos:
File: mesos-agent.ERROR
E0720 17:36:52.323874 23854 fetcher.cpp:558] Failed to run mesos-fetcher:
Failed to chown
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout'
to user 'root' : Operation not permitted
E0720 17:36:52.324268 23856 slave.cpp:4650] Container
'7506cfd4-5777-4a02-9e92-5e566ae30dec' for executor
'myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0'
of framework 5b12e719-6990-4546-9370-b984fc35b759- failed to start: Failed
to chown
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout'
to user 'root' : Operation not permitted
File: mesos-agent.WARNING
Running on machine: s19
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
W0721 10:07:42.287765 25785 backend.cpp:76] Failed to create 'aufs' backend:
AufsBackend requires root privileges W0721 10:07:42.287967
25785 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires
root privileges W0721 10:08:26.406080 25789 paths.cpp:555]
Failed to chown executor directory
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/14a83b03-a84d-4cd2-99a9-01c553b3dfb6-/executors/myriad_executor14a83b03-a84d-4cd2-99a9-01c553b3dfb6-14a83b03-a84d-4cd2-99a9-01c553b3dfb6-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/e8ed1280-4dd9-41b1-88ac-2244a49efab4'.
This may be due to attempting to run the executor as a nonexistent user on the
agent; see the description for the `--switch_user` flag for more information:
Operation not permitted E0721
10:08:26.431746 25793 fetcher.cpp:558] Failed to run mesos-fetcher: Failed to
chown
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/14a83b03-a84d-4cd2-99a9-01c553b3dfb6-/executors/myriad_executor14a83b03-a84d-4cd2-99a9-01c553b3dfb6-14a83b03-a84d-4cd2-99a9-01c553b3dfb6-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/e8ed1280-4dd9-41b1-88ac-2244a49efab4/stdout'
to user 'root' : Operation not permitted
> unable to run yarn application with hadoop2.7.3 and mesos1.2.0
> --
>
> Key: MYRIAD-255
> URL: https://issues.apache.org/jira/browse/MYRIAD-255
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
> Environment: Ubuntu 14.04.3 LTS
> master hostname: s18
> slaves hostname: s19, s20, s21
> hadoop 2.7.3
> mesos 1.2.0
>Reporter: Liu You
> Attachments: mapred-site.xml,
> mesos-master.s18.mesos.log.INFO.20170720-173553.21812,
> mesos-master.s18.mesos.log.WARNING.20170720-173553.21812, Mesos Web UI.png,
> myriad-config-default.yml, myriad ui-1.png, myriad ui-2.png, stucked job.png,
> yarn-mesos-resourcemanager-s18.log, yarn-mesos-resourcemanager-s18.out,
> yarn-site.xml
>
>
> I can't run yarn application, and the job gets stucked every time I execute
> the command 'hadoop jar xxx.'.
> The Myriad Web UI shows that the status of the task changes from staging to
> pending, then staging, then pending.
> There are some screenshots in the attachments below.
> Please help me. Thank you very much.
> yarn-mesos-resourcemanager-s18.log:
> {code:java}
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> Responder: starting
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 8032: starting
> 2017-07-20 17:36:46,955 INFO
> org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
> value:
> "myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> lost with exit status: 9
> 20
[jira] [Comment Edited] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[ https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095683#comment-16095683 ] Liu You edited comment on MYRIAD-255 at 7/21/17 2:29 AM: - [~darinj] Thanks for your reply. Strangely, the stdout and stderr from the task logs in the mesos ui is empty, 0 size and no content. I read some log files related to mesos: File: mesos-agent.ERROR E0720 17:36:52.323874 23854 fetcher.cpp:558] Failed to run mesos-fetcher: Failed to chown '/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout' to user 'root' : Operation not permitted E0720 17:36:52.324268 23856 slave.cpp:4650] Container '7506cfd4-5777-4a02-9e92-5e566ae30dec' for executor 'myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0' of framework 5b12e719-6990-4546-9370-b984fc35b759- failed to start: Failed to chown '/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout' to user 'root' : Operation not permitted File: mesos-agent.WARNING Running on machine: s19 Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg W0721 10:07:42.287765 25785 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges W0721 10:07:42.287967 25785 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges W0721 10:08:26.406080 25789 paths.cpp:555] Failed to chown executor directory '/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/14a83b03-a84d-4cd2-99a9-01c553b3dfb6-/executors/myriad_executor14a83b03-a84d-4cd2-99a9-01c553b3dfb6-14a83b03-a84d-4cd2-99a9-01c553b3dfb6-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/e8ed1280-4dd9-41b1-88ac-2244a49efab4'. This may be due to attempting to run the executor as a nonexistent user on the agent; see the description for the `--switch_user` flag for more information: Operation not permitted E0721 10:08:26.431746 25793 fetcher.cpp:558] Failed to run mesos-fetcher: Failed to chown '/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/14a83b03-a84d-4cd2-99a9-01c553b3dfb6-/executors/myriad_executor14a83b03-a84d-4cd2-99a9-01c553b3dfb6-14a83b03-a84d-4cd2-99a9-01c553b3dfb6-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/e8ed1280-4dd9-41b1-88ac-2244a49efab4/stdout' to user 'root' : Operation not permitted was (Author: elimsy): Thanks for your reply. Strangely, the stdout and stderr from the task logs in the mesos ui is empty, 0 size and no content. I read some log files related to mesos: File: mesos-agent.ERROR E0720 17:36:52.323874 23854 fetcher.cpp:558] Failed to run mesos-fetcher: Failed to chown '/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout' to user 'root' : Operation not permitted E0720 17:36:52.324268 23856 slave.cpp:4650] Container '7506cfd4-5777-4a02-9e92-5e566ae30dec' for executor 'myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0' of framework 5b12e719-6990-4546-9370-b984fc35b759- failed to start: Failed to chown '/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout' to user 'root' : Operation not permitted File: mesos-agent.WARNING Running on machine: s19 Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg W0721 10:07:42.287765 25785 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges W0721 10:07:42.287967 25785 backend.cpp:76] Failed to create 'bind' backend: BindBa
[jira] [Issue Comment Deleted] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu You updated MYRIAD-255:
---
Comment: was deleted
(was: [~darinj] Thanks for your reply.
Strangely, the stdout and stderr from the task logs in the mesos ui is empty,
0 size and no content.
I read some log files related to mesos:
File: mesos-agent.ERROR
E0720 17:36:52.323874 23854 fetcher.cpp:558] Failed to run mesos-fetcher:
Failed to chown
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout'
to user 'root' : Operation not permitted
E0720 17:36:52.324268 23856 slave.cpp:4650] Container
'7506cfd4-5777-4a02-9e92-5e566ae30dec' for executor
'myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0'
of framework 5b12e719-6990-4546-9370-b984fc35b759- failed to start: Failed
to chown
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/5b12e719-6990-4546-9370-b984fc35b759-/executors/myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/7506cfd4-5777-4a02-9e92-5e566ae30dec/stdout'
to user 'root' : Operation not permitted
File: mesos-agent.WARNING
Running on machine: s19
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
W0721 10:07:42.287765 25785 backend.cpp:76] Failed to create 'aufs' backend:
AufsBackend requires root privileges W0721 10:07:42.287967
25785 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires
root privileges W0721 10:08:26.406080 25789 paths.cpp:555]
Failed to chown executor directory
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/14a83b03-a84d-4cd2-99a9-01c553b3dfb6-/executors/myriad_executor14a83b03-a84d-4cd2-99a9-01c553b3dfb6-14a83b03-a84d-4cd2-99a9-01c553b3dfb6-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/e8ed1280-4dd9-41b1-88ac-2244a49efab4'.
This may be due to attempting to run the executor as a nonexistent user on the
agent; see the description for the `--switch_user` flag for more information:
Operation not permitted E0721
10:08:26.431746 25793 fetcher.cpp:558] Failed to run mesos-fetcher: Failed to
chown
'/home/mesos/mesos/mesos-1.2.0/mesos_install/mesos_work/slaves/ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/frameworks/14a83b03-a84d-4cd2-99a9-01c553b3dfb6-/executors/myriad_executor14a83b03-a84d-4cd2-99a9-01c553b3dfb6-14a83b03-a84d-4cd2-99a9-01c553b3dfb6-O8ce5a20f6-7e1f-4af2-9061-f07859b40bbe-S0/runs/e8ed1280-4dd9-41b1-88ac-2244a49efab4/stdout'
to user 'root' : Operation not permitted
)
> unable to run yarn application with hadoop2.7.3 and mesos1.2.0
> --
>
> Key: MYRIAD-255
> URL: https://issues.apache.org/jira/browse/MYRIAD-255
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
> Environment: Ubuntu 14.04.3 LTS
> master hostname: s18
> slaves hostname: s19, s20, s21
> hadoop 2.7.3
> mesos 1.2.0
>Reporter: Liu You
> Attachments: mapred-site.xml,
> mesos-master.s18.mesos.log.INFO.20170720-173553.21812,
> mesos-master.s18.mesos.log.WARNING.20170720-173553.21812, Mesos Web UI.png,
> myriad-config-default.yml, myriad ui-1.png, myriad ui-2.png, stucked job.png,
> yarn-mesos-resourcemanager-s18.log, yarn-mesos-resourcemanager-s18.out,
> yarn-site.xml
>
>
> I can't run yarn application, and the job gets stucked every time I execute
> the command 'hadoop jar xxx.'.
> The Myriad Web UI shows that the status of the task changes from staging to
> pending, then staging, then pending.
> There are some screenshots in the attachments below.
> Please help me. Thank you very much.
> yarn-mesos-resourcemanager-s18.log:
> {code:java}
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> Responder: starting
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 8032: starting
> 2017-07-20 17:36:46,955 INFO
> org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
> value:
> "myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> lost with exit status: 9
> 2017-07-20 17:36:
[jira] [Commented] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095707#comment-16095707
]
Liu You commented on MYRIAD-255:
[~darinj] Thanks for your reply.
It seems that myriad needs to run with root privilege, and I add the current
user to the sudoers and it works.
Thanks again.
> unable to run yarn application with hadoop2.7.3 and mesos1.2.0
> --
>
> Key: MYRIAD-255
> URL: https://issues.apache.org/jira/browse/MYRIAD-255
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
> Environment: Ubuntu 14.04.3 LTS
> master hostname: s18
> slaves hostname: s19, s20, s21
> hadoop 2.7.3
> mesos 1.2.0
>Reporter: Liu You
> Attachments: mapred-site.xml,
> mesos-master.s18.mesos.log.INFO.20170720-173553.21812,
> mesos-master.s18.mesos.log.WARNING.20170720-173553.21812, Mesos Web UI.png,
> myriad-config-default.yml, myriad ui-1.png, myriad ui-2.png, stucked job.png,
> yarn-mesos-resourcemanager-s18.log, yarn-mesos-resourcemanager-s18.out,
> yarn-site.xml
>
>
> I can't run yarn application, and the job gets stucked every time I execute
> the command 'hadoop jar xxx.'.
> The Myriad Web UI shows that the status of the task changes from staging to
> pending, then staging, then pending.
> There are some screenshots in the attachments below.
> Please help me. Thank you very much.
> yarn-mesos-resourcemanager-s18.log:
> {code:java}
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> Responder: starting
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 8032: starting
> 2017-07-20 17:36:46,955 INFO
> org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
> value:
> "myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> lost with exit status: 9
> 2017-07-20 17:36:46,969 INFO
> org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
> Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
> TASK_FAILED
> 2017-07-20 17:36:46,970 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned
> to active state
> 2017-07-20 17:36:46,971 INFO
> org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked
> as pending failed task with id value:
> "nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
> {code}
> myriad-config-default.yml:
> {code:java}
> mesosMaster: 192.168.130.18:5050
> checkpoint: false
> frameworkFailoverTimeout: 4320
> frameworkName: MyriadAlpha
> frameworkRole: "yarn"
> frameworkUser: mesos # User the Node Manager runs as, required if
> nodeManagerURI set, otherwise defaults to the user
> # running the resource manager.
> frameworkSuperUser: root # To be deprecated, currently permissions need set
> by a superuser due to Mesos-1790. Must be
> # root or have passwordless sudo. Required if
> nodeManagerURI set, ignored otherwise.
> #nativeLibrary: /usr/local/lib/libmesos.so
> nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
> zkServers: 192.168.130.18:2181
> zkTimeout: 2
> restApiPort: 8192
> #servedConfigPath: dist/config.tgz
> servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
> #servedBinaryPath: dist/hadoop-2.6.0.tgz
> servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
> profiles:
> zero: # NMs launched with this profile dynamically obtain cpu/mem from
> Mesos
> cpu: 0
> mem: 0
> small:
> cpu: 2
> mem: 2048
> medium:
> cpu: 4
> mem: 4096
> large:
> cpu: 10
> mem: 12288
> nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
> profile.
> medium: 1 #
> rebalancer: false
> haEnabled: false
> nodemanager:
> jvmMaxMemoryMB: 1024
> cpus: 0.2
> cgroups: false
> executor:
> jvmMaxMemoryMB: 256
> path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
> #path:
> file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
> #The following should be used for a remotely distributed URI, hdfs assumed
> but other URI types valid.
> #nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
> #configUri: http://s18:8192/api/artifacts/config.tgz
> #jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
> yarnEnvironment:
> YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
> #HADOOP_CONF_DIR=config
> #HADOOP_TMP_DIR=$MESOS_SANDBOX
> #YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
> #JAVA_HOME: /usr/lib/jvm/java-default #System
[jira] [Closed] (MYRIAD-255) unable to run yarn application with hadoop2.7.3 and mesos1.2.0
[
https://issues.apache.org/jira/browse/MYRIAD-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu You closed MYRIAD-255.
--
Resolution: Not A Bug
> unable to run yarn application with hadoop2.7.3 and mesos1.2.0
> --
>
> Key: MYRIAD-255
> URL: https://issues.apache.org/jira/browse/MYRIAD-255
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
> Environment: Ubuntu 14.04.3 LTS
> master hostname: s18
> slaves hostname: s19, s20, s21
> hadoop 2.7.3
> mesos 1.2.0
>Reporter: Liu You
> Attachments: mapred-site.xml,
> mesos-master.s18.mesos.log.INFO.20170720-173553.21812,
> mesos-master.s18.mesos.log.WARNING.20170720-173553.21812, Mesos Web UI.png,
> myriad-config-default.yml, myriad ui-1.png, myriad ui-2.png, stucked job.png,
> yarn-mesos-resourcemanager-s18.log, yarn-mesos-resourcemanager-s18.out,
> yarn-site.xml
>
>
> I can't run yarn application, and the job gets stucked every time I execute
> the command 'hadoop jar xxx.'.
> The Myriad Web UI shows that the status of the task changes from staging to
> pending, then staging, then pending.
> There are some screenshots in the attachments below.
> Please help me. Thank you very much.
> yarn-mesos-resourcemanager-s18.log:
> {code:java}
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> Responder: starting
> 2017-07-20 17:36:46,941 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 8032: starting
> 2017-07-20 17:36:46,955 INFO
> org.apache.myriad.scheduler.event.handlers.ExecutorLostEventHandler: Executor
> value:
> "myriad_executor5b12e719-6990-4546-9370-b984fc35b759-5b12e719-6990-4546-9370-b984fc35b759-O08e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> of slave value: "8e5e7115-060b-42c6-b24f-64642048cfa1-S0"
> lost with exit status: 9
> 2017-07-20 17:36:46,969 INFO
> org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Status
> Update for task: nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898 | state:
> TASK_FAILED
> 2017-07-20 17:36:46,970 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned
> to active state
> 2017-07-20 17:36:46,971 INFO
> org.apache.myriad.scheduler.event.handlers.StatusUpdateEventHandler: Marked
> as pending failed task with id value:
> "nm.medium.ddd2cdab-db6b-4dbc-9d5d-8c339c056898"
> {code}
> myriad-config-default.yml:
> {code:java}
> mesosMaster: 192.168.130.18:5050
> checkpoint: false
> frameworkFailoverTimeout: 4320
> frameworkName: MyriadAlpha
> frameworkRole: "yarn"
> frameworkUser: mesos # User the Node Manager runs as, required if
> nodeManagerURI set, otherwise defaults to the user
> # running the resource manager.
> frameworkSuperUser: root # To be deprecated, currently permissions need set
> by a superuser due to Mesos-1790. Must be
> # root or have passwordless sudo. Required if
> nodeManagerURI set, ignored otherwise.
> #nativeLibrary: /usr/local/lib/libmesos.so
> nativeLibrary: /home/mesos/mesos/mesos-1.2.0/mesos_install/lib/libmesos.so
> zkServers: 192.168.130.18:2181
> zkTimeout: 2
> restApiPort: 8192
> #servedConfigPath: dist/config.tgz
> servedConfigPath: /home/mesos/mesos/hdfs/hadoop-2.7.3/etc/config.tgz
> #servedBinaryPath: dist/hadoop-2.6.0.tgz
> servedBinaryPath: /home/mesos/mesos/hdfs/binary.tgz
> profiles:
> zero: # NMs launched with this profile dynamically obtain cpu/mem from
> Mesos
> cpu: 0
> mem: 0
> small:
> cpu: 2
> mem: 2048
> medium:
> cpu: 4
> mem: 4096
> large:
> cpu: 10
> mem: 12288
> nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
> profile.
> medium: 1 #
> rebalancer: false
> haEnabled: false
> nodemanager:
> jvmMaxMemoryMB: 1024
> cpus: 0.2
> cgroups: false
> executor:
> jvmMaxMemoryMB: 256
> path: hdfs://s18:8020/dist/myriad-executor-0.2.0.jar
> #path:
> file:///home/mesos/mesos/hdfs/hadoop-2.7.3/share/hadoop/yarn/lib/myriad-executor-0.2.0.jar
> #The following should be used for a remotely distributed URI, hdfs assumed
> but other URI types valid.
> #nodeManagerUri: hdfs://s18:8020/user/MR/binary.tgz
> #configUri: http://s18:8192/api/artifacts/config.tgz
> #jvmUri: https://downloads.mycompany.com/java/jre-7u76-linux-x64.tar.gz
> yarnEnvironment:
> YARN_HOME: /home/mesos/mesos/hdfs/hadoop-2.7.3
> #HADOOP_CONF_DIR=config
> #HADOOP_TMP_DIR=$MESOS_SANDBOX
> #YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set
> #JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes
> necessary
> #JAVA_HOME: /home/mesos/jdk/jdk1.8.0_131
> #JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox
> directory
> #JAVA_LIBRARY_PATH
