dwsmith1983 commented on PR #5880: URL: https://github.com/apache/datafusion-comet/pull/5880#issuecomment-5832348678
Merged main, which now includes #5265. That change widens the `hasScanInput` gate from `CometNativeScanExec` to `CometLeafExec`, so an Iceberg scan fused into a native block or inlined under a native shuffle writer now reports task input metrics at all. It does not change when or how the report runs. This PR still adds the rest. The report listener is registered before `super.compute` creates the `CometExecIterator` at the Parquet scan, Iceberg scan and fused block sites, so it runs after the iterator's close has published the final metrics. That matters when a limit stops pulling early in a block with a JVM input. The report also adds to the task's input counters instead of replacing them, and each accumulator is claimed once per task, so a fallback Spark scan or a second native tree in the same task keeps its share. With the gate widened, a fused Iceberg scan now goes through the same reordered `CometNativeExec` site as a fused Parquet scan. The Iceberg limit test now reuses the table and listener helpers from #5265, and its comment points at the broadcast join limit test in `CometTaskMetricsSuite` as the guard for that shared site. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
