ReemaAlzaid commented on PR #12471:
URL: https://github.com/apache/gluten/pull/12471#issuecomment-5203684994
Thanks @Marin-ma. Operator dump attached two q15 runs on the current branch
with `spark.gluten.sql.debug.cudf=true`: default, and with
`autoBroadcastJoinThreshold=-1` (the config the original crash occurred under).
Result: I can no longer reproduce the escape. Every driver whose output
leaves the task now ends with a trailing CudfToVelox (the only GPU-terminated
pipeline is the join-build sink, which is correct producesGpuOutput=0), and q15
passes value correct in both configs. The original crash
was against the earlier Velox pin, and an intervening change closed the
gap; at that time the C2R input was verifiably a device CudfVector (SIGSEGV on
garbage VARCHAR offsets). Given that, the conversions are no longer
load-bearing — I've made them log a warning when they actually fire, plus an
opt-in cudf.strictResidency conf that fails instead of converting, so CI can
catch any future insertion regression rather than bury it. Happy to trim the
guard sites further if you prefer.
On bhj build-once: agreed that's what this PR does.
`VeloxBroadcastBuildSideRDD` skips the prebuilt hash table path entirely when
cuDF is enabled and streams the broadcast batches instead (the GPU join can't
consume the opaque table, which is exactly why builds arrived empty). If you'd
rather see that as a dedicated config or plan rule instead of the branch in the
RDD, I can restructure
[q15_log_dump.md](https://github.com/user-attachments/files/30784218/q15_log_dump.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
