Surbhi-Vijay opened a new issue, #6694: URL: https://github.com/apache/incubator-gluten/issues/6694
### Backend VL (Velox) ### Bug description We have observed performance drop in TPCDS runs after the patch #6009. Top regressing Queries -------------------------------------------- QueryId New runtime Previous Runtime query64 50712 22841 query24a 44883 27452 query24b 45003 28742 When we disabled the feature using ` "spark.gluten.sql.columnar.backend.velox.coalesceBatchesBeforeShuffle": "false"`. We see the same runtime as previous runs. We are using azure cluster and reading data from remote storage account. The regression is seen in `VeloxAppendBatches` where in some instances, it is taking a lot of time Below are the plan snippets from query64   ### Spark version Spark-3.4.x ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
