[
https://issues.apache.org/jira/browse/SPARK-13743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-13743.
----------------------------------
Resolution: Incomplete
> Adding configurable support for Spark Streaming gracefull timeout
> -----------------------------------------------------------------
>
> Key: SPARK-13743
> URL: https://issues.apache.org/jira/browse/SPARK-13743
> Project: Spark
> Issue Type: Improvement
> Components: DStreams
> Reporter: Yuval Itzchakov
> Priority: Minor
> Labels: bulk-closed
>
> Spark Streaming supports gracefull shutdown via the
> "spark.streaming.stopGracefullyOnShutdown" configuration property. The actual
> gracefull shutdown period for the spark job is hardcoded into
> `JobScheduler.stop()`:
> // Wait for the queued jobs to complete if indicated
> val terminated = if (processAllReceivedData) {
> jobExecutor.awaitTermination(1, TimeUnit.HOURS) // just a very large
> period of time
> } else {
> jobExecutor.awaitTermination(2, TimeUnit.SECONDS)
> }
> I think we can greatly benefit if this setting wasn't hardcoded to an hour,
> but configurable via an additional spark configuration key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
