unikdahal opened a new pull request, #5414: URL: https://github.com/apache/datafusion-comet/pull/5414
## Which issue does this PR close? Closes #5413. ## Rationale for this change Six copies of the same guard reject a zero-field `StructType`, treating it the same as a genuinely unsupported type. It's a legitimate Arrow value. Iceberg's `_partition` metadata column is exactly this shape on an unpartitioned table, so any plan carrying it silently fell back to Spark at the first shuffle/sink/scan boundary. ## What changes are included in this PR? Removed the empty-struct exclusion from all six checks: native shuffle, columnar shuffle, `CometSink`, `QueryPlanSerde.supportedDataType`, `DataTypeSupport`, and `from_json`'s target-schema check. The last one alone wasn't safe to fix Scala-side -- it uncovered a real native panic in `from_json.rs` (`StructArray::new` can't derive row count with zero child arrays), fixed by branching to `StructArray::new_empty_fields`. ## How are these changes tested? New general-purpose tests (no Iceberg involved): native/columnar shuffle on an empty-struct column, and `from_json(..., 'struct<>')`. Also ran `CometNativeShuffleSuite`, `CometShuffleSuite`, `CometExecSuite`, `CometJsonExpressionSuite` -- all pass, including the existing tests that assert legitimate cases still fall back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
