Yang Jie created SPARK-58223:
--------------------------------
Summary: SparkSession.Builder.classic() builds a Connect session
instead of a Classic one
Key: SPARK-58223
URL: https://issues.apache.org/jira/browse/SPARK-58223
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.1.2
Reporter: Yang Jie
`SparkSession.Builder.classic()` passes `CONNECT_COMPANION` to `mode(...)`,
identical to `connect()`:
{code:scala}
def classic(): this.type = mode(CONNECT_COMPANION)
def connect(): this.type = mode(CONNECT_COMPANION)
{code}
`companion` drives `builder()` (`companion.builder()`), so on a classpath with
both implementations, `SparkSession.builder().classic().getOrCreate()` builds a
Connect session rather than a Classic one. The config-key path already maps
correctly (`API_MODE_CLASSIC => CLASSIC_COMPANION`), and `CLASSIC_COMPANION` is
defined, so the fix is to have `classic()` pass `CLASSIC_COMPANION`.
Introduced by SPARK-49700 (unified Scala interface for Connect and Classic);
present since 4.1.0. There was no test covering the `classic()` / `connect()`
mode selectors, which is why it went unnoticed.
Fix: `classic()` should call `mode(CLASSIC_COMPANION)`, plus regression
coverage for both mode selectors in
`SparkSessionBuilderImplementationBindingSuite`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
