HyukjinKwon commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions URL: https://github.com/apache/spark/pull/23556#issuecomment-475823794 @j-esse, to reiterate, - It disables a specific rule on a certain condition - that's why you have multiple projects after this PR. - I don't think we should add each configuration for each rule to avoid this. Since you already see similar instances, I think it's better to have a more general solution rather than to fix each rule. - You're basically just working around the issue within Spark. If you have multiple plans, it will cause OOM anyway. - The default is measured based on numbers but the problem is driver memory size. Why it's not taken into account?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
