HyukjinKwon opened a new pull request #35517: URL: https://github.com/apache/spark/pull/35517
### What changes were proposed in this pull request? This PR is a followup of https://github.com/apache/spark/pull/35410 to fix a mistake. `HiveContext` should set `spark.sql.catalogImplementation` to `hive` instead of `in-memory`. This PR also includes several changes: - Make `HiveContext.getOrCreate` works identically as `SQLContext.getOrCreate` - Match the signature of `HiveContext.__init__` and `SQLContext.__init__` (both are not supported to be directly called by users though). ### Why are the changes needed? See https://github.com/apache/spark/pull/35410#discussion_r806358814 ### Does this PR introduce _any_ user-facing change? No to end users because this change has not been released out yet. It creates a non-Hive supported `SparkSession` if there isn't an existing SparkSession running. See also https://github.com/apache/spark/pull/35410#discussion_r806358814. ### How was this patch tested? Manually tested: ```python spark.stop() from pyspark import SparkContext from pyspark.sql import HiveContext HiveContext.getOrCreate(SparkContext.getOrCreate()).getConf("spark.sql.catalogImplementation") ``` **Before:** ```pyspark 'in-memory' ``` **After:** ```pyspark 'hive' ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
