manuzhang opened a new pull request, #4278: URL: https://github.com/apache/datafusion-comet/pull/4278
## Summary - Enable the nested `ArrayType` cast matrix in `CometCastSuite`. - Add bounded nested array inputs to avoid recursively generated huge task results. - Exclude `DecimalType(38,18)` from the nested matrix for the same known decimal-to-floating ULP mismatch as the one-dimensional matrix. - Replace deprecated `JavaConverters` usage with `scala.jdk.CollectionConverters`. ## Root Cause The previous nested test used `generateArrays(100, ArrayType(dt))`, which let Spark random data generation build very large nested arrays. The `array<array<string>>` to `array<array<boolean>>` case produced a task result around 401 MiB and failed deserialization with `unread block data`. After bounding the nested inputs, the remaining failure was the existing decimal38-to-floating precision mismatch, so the nested matrix now mirrors the one-dimensional exclusion. ## Tests - `make core` - `./mvnw test -Pspark-4.1 -Dtest=none -Dsuites="org.apache.comet.CometCastSuite cast nested ArrayType to nested ArrayType"` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
