kosiew commented on code in PR #22529:
URL: https://github.com/apache/datafusion/pull/22529#discussion_r3330357941
##########
datafusion/spark/src/session_state.rs:
##########
@@ -108,4 +111,27 @@ mod tests {
"Apache Spark expr planners should be registered"
);
}
+
+ #[tokio::test]
+ async fn test_spark_dialect_with_spark_functions() {
+ let mut config = SessionConfig::new();
+ config.options_mut().sql_parser.dialect = Dialect::Spark;
+ let state = SessionStateBuilder::new()
+ .with_config(config)
+ .with_default_features()
+ .with_spark_features()
+ .build();
+ let ctx = SessionContext::new_with_state(state);
+
+ // Spark function + Spark dialect parsing
+ let result = ctx
+ .sql("SELECT sha2('abc', 256)")
Review Comment:
Nice addition. This test shows that Spark functions are registered, but
`SELECT sha2('abc', 256)` also parses under the generic dialect, so it does not
quite exercise the new Spark-dialect path.
Could we make the query use Spark-specific (or Spark-sensitive) syntax, or
add a second assertion that would fail under `Dialect::Generic`? That would
help ensure the test covers the full invariant: Spark functions are registered
and Spark SQL parsing is active in the same session.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
