[ https://issues.apache.org/jira/browse/SPARK-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14206406#comment-14206406 ]
Sean Owen commented on SPARK-4301: ---------------------------------- Sorry, a bit late, but I noticed this is pretty related to https://issues.apache.org/jira/browse/SPARK-2645 which discusses calling stop() twice. > StreamingContext should not allow start() to be called after calling stop() > --------------------------------------------------------------------------- > > Key: SPARK-4301 > URL: https://issues.apache.org/jira/browse/SPARK-4301 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.1.0 > Reporter: Josh Rosen > Assignee: Josh Rosen > Fix For: 1.1.1, 1.2.0, 1.0.3 > > > In Spark 1.0.0+, calling {{stop()}} on a StreamingContext that has not been > started is a no-op which has no side-effects. This allows users to call > {{stop()}} on a fresh StreamingContext followed by {{start()}}. I believe > that this almost always indicates an error and is not behavior that we should > support. Since we don't allow {{start() stop() start()}} then I don't think > it makes sense to allow {{stop() start()}}. > The current behavior can lead to resource leaks when StreamingContext > constructs its own SparkContext: if I call {{stop(stopSparkContext=True)}}, > then I expect StreamingContext's underlying SparkContext to be stopped > irrespective of whether the StreamingContext has been started. This is > useful when writing unit test fixtures. > Prior discussions: > - https://github.com/apache/spark/pull/3053#discussion-diff-19710333R490 > - https://github.com/apache/spark/pull/3121#issuecomment-61927353 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org