fwc opened a new pull request, #56190:
URL: https://github.com/apache/spark/pull/56190
### What changes were proposed in this pull request?
- Introduces `sql.SparkSessionBinder` and `classic.SparkSessionBinder` as
'implementor' counterparts to the 'declarators' `sql.SparkSessionProvider` and
`classic.SparkSessionProvider`.
- Deprecates `SharedSparkSession` with the hint that `SparkSessionBinder`
and `QueryTest` shall be used instead.
### Why are the changes needed?
Currently, most tests use `SharedSparkSession` to obtain the `spark` object.
This effectively prevents specializing these tests in `sql/connect` as
`SharedSparkSession` provides a `classic.SparkSession`, thus preventing
overriding.
This PR deprecates `SharedSparkSession` and instead introduces
`sql.SparkSessionBinder` and `classic.SparkSessionBinder`. If some `FooSuite`
now uses the `sql.SparkSessionBinder` trait like e.g.
```
class FooSuite extends SparkSessionBinder with QueryTest {
checkAnswer(
sql("SELECT 1"),
Seq(1)
)
}
```
We can now add a connect variant of that suite as follows:
```
class FooWithConnectSuite extends FooSuite
with connect.SparkSessionBinder
with connect.QueryTest
```
### Does this PR introduce _any_ user-facing change?
This PR extends the `beforeAll`/`afterAll` of `SharedSparkSessionBase` to
include the the thread audit check, which was previously only present in
`SharedSparkSession`.
AFAICS, `SparkSessionBase` is neither used in delta lake nor in apache
iceberg.
### How was this patch tested?
This patch is test-only.
### Was this patch authored or co-authored using generative AI tooling?
Parts of this patch were authored by claude code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
