comphead opened a new issue, #5544: URL: https://github.com/apache/datafusion-comet/issues/5544
@comphead, smaller follow-ups are reasonable, provided this PR preserves the previous safe behavior for the affected cases. The remaining **[P2]** concerns come from the actual BASE-to-HEAD changes: - The TRY/bracket and collation examples contain folded map literals. [BASE declines those non-null complex literals](https://github.com/apache/datafusion-comet/blob/e0ab0a6fe60c05bd679654f0201ebe88319cc3a7/spark/src/main/scala/org/apache/comet/serde/literals.scala#L43-L64); [HEAD newly admits and rebuilds them](https://github.com/apache/datafusion-comet/blob/67fbae59727d108dda1a875e5f00c33a545855e4/spark/src/main/scala/org/apache/comet/serde/literals.scala#L43-L86), exposing the consumer limitations. That admission is not restricted to an outer array-of-map literal. - The stateful example exercises the [new ANSI guard](https://github.com/apache/datafusion-comet/blob/67fbae59727d108dda1a875e5f00c33a545855e4/spark/src/main/scala/org/apache/comet/serde/arrays.scala#L623-L696), which serializes the left operand twice. [BASE has no second serialization](https://github.com/apache/datafusion-comet/blob/e0ab0a6fe60c05bd679654f0201ebe88319cc3a7/spark/src/main/scala/org/apache/comet/serde/arrays.scala#L601-L639). This change affects nullable array lookups beyond literal expansion. - The Slice example does construct an array of maps. The [new `deepNullable` casts](https://github.com/apache/datafusion-comet/blob/67fbae59727d108dda1a875e5f00c33a545855e4/spark/src/main/scala/org/apache/comet/serde/arrays.scala#L520-L534) change its nested map type, while [Slice still declares the original element type](https://github.com/apache/datafusion-comet/blob/67fbae59727d108dda1a875e5f00c33a545855e4/spark/src/main/scala/org/apache/comet/serde/arrays.scala#L465-L484). BASE passes that single map child through without the new cast. So a narrow solution here is fine: restrict the newly enabled paths or retain safe fallback until the corresponding consumer fix lands; broader collation/collection improvements can remain separate PRs. The scope distinction above is from pinned source comparison, not an additional executed reproduction. _Originally posted by @sunchao in https://github.com/apache/datafusion-comet/pull/5452#pullrequestreview-5058500152_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
