gatorsmile commented on code in PR #53820:
URL: https://github.com/apache/spark/pull/53820#discussion_r2699729030
##########
python/pyspark/sql/tests/test_session.py:
##########
@@ -461,6 +462,114 @@ def test_invalid_create(self):
messageParameters={},
)
+class SparkSessionBuilderCreateTests(unittest.TestCase, PySparkErrorTestUtils):
Review Comment:
Thank you for adding the tests!
Could you help clarify coverage for the following areas?
Error Handling Tests
• How does create() behave when called with invalid
configurations?
• What is the expected behavior if SparkContext creation fails?
• How should conflicting SparkContext-level configurations be
handled when a context already exists?
Thread Safety Tests
• How should the implementation behave when multiple threads call
create() concurrently?
• Are there any expected race conditions between create() and
getOrCreate() that we should guard against or test explicitly?
Default/Active Session Update Logic
The PR description states:
“This method will update the default and/or active session if they are not
set.”
Could we clarify:
• What is the expected behavior for updating the default session,
and how should this be tested?
• In what cases should the default session be updated or
preserved?
• Are there additional scenarios around active session updates
that should be validated?
Session Stop/Cleanup
• What should happen when stop() is called on a session created
via create()?
• How can we ensure resources are properly cleaned up?
• Should stopping one session affect other sessions sharing the
same SparkContext?
Connect Mode Fallback
• How do we want to validate that the existing Connect mode logic
continues to work correctly after these changes?
appName Configuration
• Should we explicitly verify that appName() and other common
builder configurations work correctly with create() in addition to master()?
Catalog/Database Operations
• What expectations do we have around catalog operations for
sessions created via create()? For example:
• Creating databases or tables
• Cross-session visibility of catalogs
Duplicate Config Keys
• What should the expected behavior be if the same config key is
set multiple times in the builder before calling create()?
Remote Mode Error Paths
• Are there specific error scenarios in the Connect/remote path
(e.g., invalid remote URLs) that we should explicitly test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
