[
https://issues.apache.org/jira/browse/MESOS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796507#comment-16796507
]
Vinod Kone commented on MESOS-9500:
-----------------------------------
[~atheethkaup] Can you paste all the relevant agent log lines related to this
task? It's hard to tell from just the line you posted above. We need log lines
from the task launch all the way to task failing.
> spark submit with docker image on mesos cluster fails.
> ------------------------------------------------------
>
> Key: MESOS-9500
> URL: https://issues.apache.org/jira/browse/MESOS-9500
> Project: Mesos
> Issue Type: Bug
> Components: docker
> Affects Versions: 1.7.0
> Reporter: atheeth kaup
> Priority: Critical
>
> We have 3 node cluster with mesos(V 1.7), spark 2.4 and docker 18.06.1
> installed. one is master and other two are agents. while doing spark submit
> job fails. UI shows only one task(Driver) launching on one of the slaves in
> the failed state.
>
> command:
> spark-submit \
> --master mesos://****.32:7077 \
> --deploy-mode cluster \
> --class com.learning.spark.WordCount \
> --conf
> spark.mesos.executor.docker.image=mesosphere/spark:2.4.0-2.2.1-3-hadoop-2.7 \
> --conf spark.master.rest.enabled=true \
> /home/mapr/mesos/wordcount.jar hdfs://***.36:8020/user/mapr/sparkL/input.txt
> hdfs://***.36:8020/user/output
>
> Error in one of the Logs:
>
> Running on machine: **-i0058
> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid
> [file:line|file://line/]] msg
> W1221 16:51:23.857431 17978 state.cpp:478] Failed to find executor forked pid
> file
> '/home/**/mesos/mesos-1.7.0/build/workDir/meta/slaves/822a5d52-b8ba-459f-ade2-7f3a2ebd240f-S0/frameworks/77c39bdf-09e3-4cb9-9026-21e900d08318-0007/executors/driver-20181221112019-0006/runs/7c1399ca-4e0a-4bd9-b02e-9c5ca3854c77/pids/forked.pid'
>
> Below is the only property that we have set on all the nodes and have started
> the dispatcher:
> *export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/libmesos.so*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
