WangGuangxin commented on PR #6480: URL: https://github.com/apache/incubator-gluten/pull/6480#issuecomment-2595383208
> > Yes, the issue exists in every Spark version. > > @jinchengchenghh I wonder, why do we only fix this in the shims of Spark 3.2? @kecookier I think it was because higher spark version can pull outer the sorter from `InsertHiveTable` with spark conf `spark.sql.optimizer.plannedWrite.enabled=true`, so that we can use columnar sorter instead of unsafe sorter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
