HyukjinKwon commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions URL: https://github.com/apache/spark/pull/23556#issuecomment-475462720 We still can disable the rule for job-based. This PR sounds it only adds some fine-grind partial automatic logic to disable the rule on the certain condition, which still needs a manual estimation from users to set the configuration value. I don't see so much value on it to be honest. How common the case is it, and how much does it save the memory before/after? And, are there only applicable in `CollapseProject` and `PhysicalOperation`? I don't think we should add each configuration for each rule to set the threshold. Looks an overkill as well.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
