Niranjani created SPARK-24700:
---------------------------------
Summary: AM shutdown terminates streaming application but marks it
as successful
Key: SPARK-24700
URL: https://issues.apache.org/jira/browse/SPARK-24700
Project: Spark
Issue Type: Bug
Components: Spark Core, Structured Streaming, YARN
Affects Versions: 2.2.0
Reporter: Niranjani
While running Spark streaming applications on YARN in cluster mode, AM
reboot/shutdown causes the application to terminate SparkContext and mark it as
SUCCEEDED. This is misleading as:
# streaming applications are long-running and not expected to terminate
# {color:#172b4d}setting
{color}{color:#172b4d}{{spark.yarn.maxAppAttempts}}{color}{color:#172b4d} for
application re-attempt does not take effect since this scenario is not
recognized as a failure{color}
{color:#172b4d}Please provide a workaround for AM termination so
streaming/long-running applications can be resubmitted.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
