cht42 opened a new pull request, #19865: URL: https://github.com/apache/datafusion/pull/19865
## Which issue does this PR close? - Closes #19843 ## Rationale for this change Currently, combining DataFusion's default features with Spark features is awkward because: 1. Expression planners must be registered **before** calling `with_default_features().build()` to take precedence 2. UDFs must be registered **after** the state is built (if using `register_all`) This requires splitting the setup into multiple phases, which is verbose and error-prone. ## What changes are included in this PR? - Added `with_spark_features()` method to `SessionStateBuilder` that registers both the Spark expression planner (with correct precedence) and all Spark UDFs in one call - Added `spark` feature flag to `datafusion-core` with `datafusion-spark` as an optional dependency - Updated `datafusion-spark` crate documentation with usage example - Simplified test context setup in `datafusion-sqllogictest` to use the new method - Changes to CI file to include `spark` feature flag in test/checks ## Are these changes tested? Yes, there is a unit test plus the existing Spark SQLLogicTest suite validates that all Spark functions work correctly. The test context in datafusion-sqllogictest now uses with_spark_features(), serving as both a usage example and integration test. ## Are there any user-facing changes? Yes, this adds a new public API: `SessionStateBuilder::with_spark_features()` method (behind the spark feature flag) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
