[
https://issues.apache.org/jira/browse/SPARK-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906336#comment-15906336
]
Sridhar Rana commented on SPARK-19920:
--------------------------------------
[[email protected]]How to send it to [email protected]? I send a mail and
it bounced back.
> How to capture reasons (log/trace/info/anything?) for "[ERROR]Driver
> 172.31.25.77:45151 disassociated! Shutting down."
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-19920
> URL: https://issues.apache.org/jira/browse/SPARK-19920
> Project: Spark
> Issue Type: Question
> Components: Spark Submit, YARN
> Affects Versions: 2.1.0
> Reporter: Sridhar Rana
> Priority: Critical
>
> We have an AWS Cloudera Spark environment. It is a yarn cluster with 1 driver
> node and 3 executor node. We use Spark SQL heavily and log4J for logging.
> Ours is a 24X7 long running process in a iterative loop. The process runs
> fine but after several iterations (after few hours), it reports this error
> "[ERROR]Driver 172.31.25.77:45151 disassociated! Shutting down.". At the
> sames second, there is this warning "[WARN ]Error sending message [message =
> Heartbeat(1,[Lscala.Tuple2;@24452d3d,BlockManagerId(1,
> ip-172-31-21-121.ec2.internal, 40378))] in 1 attempts". The spark process is
> able to recover from this failure but takes more time to finish that
> iteration. Other than that there is not much info on this. How do we know
> what is the cause of this error condition or what's causing it so that
> appropriate measure can be taken? Can we capture those using log4j?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
