RRXXZZYY commented on code in PR #5603:
URL: https://github.com/apache/datafusion-comet/pull/5603#discussion_r3905648477
##########
spark/src/main/scala/org/apache/comet/serde/structs.scala:
##########
@@ -31,12 +31,16 @@ import
org.apache.comet.CometSparkSessionExtensions.withFallbackReason
import org.apache.comet.DataTypeSupport
import org.apache.comet.serde.QueryPlanSerde.{exprToProtoInternal,
serializeDataType}
-object CometCreateNamedStruct extends CometExpressionSerde[CreateNamedStruct] {
+object CometCreateNamedStruct
+ extends CometExpressionSerde[CreateNamedStruct]
+ with CodegenDispatchFallback {
Review Comment:
Thanks for tracing both remaining boundaries. I covered them together in
c458c31b:
- the shared StreamReader and broadcast coalescing now use a
CometArrowStreamReader that allocates duplicate-name struct children
positionally while retaining Arrow's ordinary allocation path for unaffected
fields;
- the broadcast target root uses the same duplicate-safe factory; and
- CometUdfBridge now supplies that factory to ArrowImporter for C Data
inputs.
I added a low-level IPC/broadcast regression plus query-level broadcast hash
join and retained-producer JVM to_json regressions. Fresh follow-up validation:
Spark 3.4 UtilsSuite 7/7; Spark 3.5 UtilsSuite + NativeUtilSuite 18/18 on the
final implementation; Spark 4.0 UtilsSuite + NativeUtilSuite 18/18 before the
final close-idempotence adjustment, which was subsequently compiled and
exercised by the Spark 3.5 run. Targeted Spotless for all eight follow-up files
and git diff --check passed.
The two query-level integration regressions are present but could not run
locally because the Windows Spark environment stops at NativeIO.Windows without
HADOOP_HOME; I did not introduce an untrusted winutils binary. CI is still
awaiting first-time-contributor approval. I updated the PR description with
these exact boundaries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
