shivadarshan-devadiga opened a new pull request, #13250: URL: https://github.com/apache/gluten/pull/13250
## What changes are proposed in this pull request? `GlutenSQLQuerySuite` on spark34 sometimes fails with an NPE in `AdaptiveSparkPlanHelper.subqueriesAll` (#13222). The problem is in `WithQueryPlanListener`. Spark calls `onSuccess` on the listener bus thread, and it appends to a plain `ArrayBuffer`. Meanwhile `invokeAll()` copies and clears the same buffer from the test thread. If those overlap, the copy can end up with a `null`, which is the NPE we see. We also never waited for the bus to catch up, so plans from the last few queries could arrive after the check had already run. The fix waits for the listener bus (`listenerBus.waitUntilEmpty()`) before running the plan listeners, and synchronizes access to the buffer. Fixes #13222. ## How was this patch tested? I couldn't make the test fail on demand. Instead I reproduced the race outside Gluten with the same append vs. copy-and-clear pattern on two threads. On Scala 2.12 it produced nulls and dropped plans; with the lock it didn't. Also checked that `gluten-substrait` compiles against Spark 3.4, `spotless:check` passes, and `waitUntilEmpty()` exists in every Spark version we support. ## Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
