SteNicholas commented on code in PR #1308:
URL: https://github.com/apache/paimon-rust/pull/1308#discussion_r4439787781


##########
crates/integrations/datafusion/src/sql_context.rs:
##########
@@ -157,6 +158,7 @@ impl SQLContextBuilder {
         let mut state_builder = SessionStateBuilder::new()
             .with_config(crate::lateral_vector_search::session_config())
             .with_default_features()
+            .with_spark_features()

Review Comment:
   **Preserve DATE_PART result types when enabling the Spark planner**: Spark 
54's `SparkDatePart::return_field_from_args` declares `Int32` for every part, 
but its simplifier delegates `epoch` to native `date_part`, which returns 
`Float64` (`nanosecond` similarly returns `Int64`). With this registration, 
type coercion runs against the incorrect integer declaration before 
simplification. For example:
   
   ```sql
   SELECT EXTRACT(EPOCH FROM TIMESTAMP '1970-01-01 00:00:01.500') / 2;
   ```
   
   Tracing the coercion and simplification paths gives integer `0` instead of 
the previous `0.75`: the analyzer inserts an `Int64` cast around the extraction 
and selects integer division, then simplification replaces the extraction with 
a `Float64` expression inside that cast.
   
   Please preserve the actual result types for these parts and add arithmetic 
regression coverage for both `date_part` calls and EXTRACT syntax. 
Re-registering `date_part` alone will not fix EXTRACT because 
`SparkFunctionPlanner::plan_extract` constructs the upstream UDF directly.
   
   Sources: [Spark 
DATE_PART](https://github.com/apache/datafusion/blob/54.0.0/datafusion/spark/src/function/datetime/date_part.rs),
 [native 
DATE_PART](https://github.com/apache/datafusion/blob/54.0.0/datafusion/functions/src/datetime/date_part.rs),
 [Spark 
planner](https://github.com/apache/datafusion/blob/54.0.0/datafusion/spark/src/planner.rs).
   
   This finding is based on source tracing; the SQL reproduction was not 
executed locally.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to