jonmio opened a new pull request, #53820: URL: https://github.com/apache/spark/pull/53820
### What changes were proposed in this pull request? Allow users to call SparkSession.Builder.create from PySpark in Classic Mode so that they can a new session with the provided builder configs without mutating configurations in any existing sessions. ### Why are the changes needed? Currently, `create` is not supported in PySpark classic. Users can call `getOrCreate` to obtain a session with the provided configurations, but if a session already exists, `getOrCreate` will mutate the existing configurations in the session. We would like to provide an API for users which guarantees creation of a session and does not have side effects on existing sessions. This change also aligns the behavior between Classic and Connect mode since `create` is already supported in Connect. ### Does this PR introduce _any_ user-facing change? Previously calling `SparkSessionBuilder.create()` would throw an exception saying that create is only supported in Connect mode. After this change, `SparkSessionBuilder.create()` will create a new SparkSession with the specified builder configurations. ### How was this patch tested? Added the new test class `SparkSessionBuilderCreateTests` in `python/pyspark/sql/tests/test_session.py` ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
