ziting-openai commented on code in PR #5262:
URL: https://github.com/apache/datafusion-comet/pull/5262#discussion_r3930137552
##########
native/Cargo.toml:
##########
@@ -35,18 +35,18 @@ license = "Apache-2.0"
edition = "2021"
# Comet uses the same minimum Rust version as DataFusion
-rust-version = "1.88"
+rust-version = "1.94.0"
[workspace.dependencies]
-arrow = { version = "58.4.0", features = ["prettyprint", "ffi", "chrono-tz"] }
-arrow-select = { version = "58.4.0" }
+arrow = { version = "59.2.0", features = ["prettyprint", "ffi", "chrono-tz"] }
+arrow-select = { version = "59.2.0" }
async-trait = { version = "0.1" }
bytes = { version = "1.11.1" }
-parquet = { version = "58.4.0", default-features = false, features =
["experimental"] }
-datafusion = { version = "54.1.0", default-features = false, features =
["unicode_expressions", "crypto_expressions", "nested_expressions", "parquet"] }
-datafusion-datasource = { version = "54.1.0" }
-datafusion-physical-expr-adapter = { version = "54.1.0" }
-datafusion-spark = { version = "54.1.0", features = ["core"] }
+parquet = { version = "59.2.0", default-features = false, features =
["experimental"] }
+datafusion = { version = "55.0.0", default-features = false, features =
["unicode_expressions", "crypto_expressions", "nested_expressions", "parquet"] }
Review Comment:
Update for `3ee7fa65b2f5e7018b5c60f4088a57eba2b9934f`: the compiler
migrations and obsolete `collect_set` drift sentinel are fixed. [Rust
CI](https://github.com/apache/datafusion-comet/actions/runs/33891332915/job/101084453021)
passed all 1,131 executed tests, including the updated collect-aggregate and
Iceberg fixtures (four skipped). The revised aggregate test retains the
`collect_list` negative check and positive coerced-state checks for both
aggregates.
The two nullable-map-key diagnostic assertions still fail at
`CometCelebornShuffleReaderSuite.scala:1343`, now reproduced by [Linux Spark
3.5](https://github.com/apache/datafusion-comet/actions/runs/33891332915/job/101093022179),
[Linux Spark
4.2](https://github.com/apache/datafusion-comet/actions/runs/33891332915/job/101093022178),
and [macOS Spark
4.0](https://github.com/apache/datafusion-comet/actions/runs/33891332915/job/101094487398).
Each job's only two failures are the JVM-reader and native-ShuffleScan
nullable-key cases; the preceding empty-row assertion succeeds, but the
expected `type mismatch` substring is absent. The source trace remains that
Arrow 59.3 rejects the nullable key field during IPC decoding with `Map key
field must not be nullable` before Comet's later type check. Please adapt the
diagnostic expectation while preserving no-output and fetch-failure checks.
This remains one DF/Arrow test-migration P2, not a new production
correctness finding. These jobs tested synthetic merge
`282cc8faa482c4254a3173cfd7087e1130128712`; the relevant test, decoder and
dependency blobs match the reviewed head. The Rust fixtures/helper/parser
function bodies also match. No local tests were run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
