Zouxxyy commented on PR #10646: URL: https://github.com/apache/incubator-gluten/pull/10646#issuecomment-3262328731
@jinchengchenghh I tried to offload the CTAS command, and I found two main reasons that prevented me from doing so: 1. Many classes need to be copied, and it is difficult to validate fallback logic: This is because we can only construct the v2 `write` after creating the table, which is required for performing the validation. By that point, it's already too late to perform proper fallback. 2. Spark 3.5 alreadly supports columnar v2 write for CTAS after this PR was introduced https://github.com/apache/spark/pull/40734, this PR is essentially a copy of it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
