LinSimon-901101 commented on PR #5889: URL: https://github.com/apache/datafusion-comet/pull/5889#issuecomment-6008179854
A quick update: I've pushed the review fixes and resolved the merge conflict with main. The test adjustment for Spark 3.4 remains on my fork pending CI validation. The previous fork CI run exposed six failures on Spark 3.4.3 while collecting the Spark reference results with Comet disabled. I reproduced this locally; the failure matches [[SPARK-45584](https://issues.apache.org/jira/browse/SPARK-45584)], where Spark's TopK does not wait for subquery execution. This is separate from the Comet subquery-registration issue in #6676. The test adjustment uses a regular Spark sort for the reference results, while retaining Comet TopK execution and its plan assertions. Removing the Comet registration fix still makes all six cases fail, confirming that the regression coverage remains effective. After merging the latest main, all six affected Spark 3.4 cases pass locally. The Spark 4.1 codegen, native-UDF nesting, and TopK checks also passed (116 tests). I've triggered a [[new full CI run on my fork](https://github.com/LinSimon-901101/datafusion-comet/actions/runs/37404806115)], including both the merge resolution and the test adjustment. I'll add the test adjustment to this PR once CI passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
