Repository: spark Updated Branches: refs/heads/master 6bcbf9b74 -> ea6622865
[SPARK-17261] [PYSPARK] Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext" ## What changes were proposed in this pull request? Set SparkSession._instantiatedContext as None so that we can recreate SparkSession again. ## How was this patch tested? Tested manually using the following command in pyspark shell ``` spark.stop() spark = SparkSession.builder.enableHiveSupport().getOrCreate() spark.sql("show databases").show() ``` Author: Jeff Zhang <zjf...@apache.org> Closes #14857 from zjffdu/SPARK-17261. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ea662286 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ea662286 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ea662286 Branch: refs/heads/master Commit: ea662286561aa9fe321cb0a0e10cdeaf60440b90 Parents: 6bcbf9b Author: Jeff Zhang <zjf...@apache.org> Authored: Fri Sep 2 10:08:14 2016 -0700 Committer: Davies Liu <davies....@gmail.com> Committed: Fri Sep 2 10:08:14 2016 -0700 ---------------------------------------------------------------------- python/pyspark/sql/session.py | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/ea662286/python/pyspark/sql/session.py ---------------------------------------------------------------------- diff --git a/python/pyspark/sql/session.py b/python/pyspark/sql/session.py index 61fa107..8418abf 100644 --- a/python/pyspark/sql/session.py +++ b/python/pyspark/sql/session.py @@ -595,6 +595,7 @@ class SparkSession(object): """Stop the underlying :class:`SparkContext`. """ self._sc.stop() + SparkSession._instantiatedContext = None @since(2.0) def __enter__(self): --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org