rich7420 opened a new issue, #6075: URL: https://github.com/apache/datafusion-comet/issues/6075
### Describe the bug `CometParquetWriterTestBase.captureWritePlan` registers a listener without draining pending events, then returns once any matching callback arrives. A delayed callback from a setup write can be mistaken for the write under test. In this [fork CI failure](https://github.com/rich7420/datafusion-comet/actions/runs/35569382593/job/106241292041), `INSERT INTO ... SELECT writes the target table's column names` captured the earlier insert into `comet_rename_source`, where Comet was deliberately disabled. The intended write was into `comet_rename_target`, so the assertion incorrectly reported zero native writers. ### Steps to reproduce Observed in the Spark 4.0.4 / JDK 21 scans job at `46ef424d0`. The same test passed locally on the unchanged commit when run alone; I have not reproduced the failing schedule locally. ### Expected behavior Capture the requested write's plan, without accepting callbacks from earlier writes. ### Additional context The [helper](https://github.com/apache/datafusion-comet/blob/5ca149928f7743bfe7a96feadea5e0f9bed1412f/spark/src/test/scala/org/apache/comet/parquet/CometParquetWriterTestBase.scala#L63-L102) and failing test are unchanged on current upstream main (`5ca149928`). Drain the listener bus before registration and after the write using `CometListenerBusUtils.waitUntilEmpty`, replacing the polling loop. The columnar-to-row benchmark already uses this pattern. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
