2010YOUY01 commented on PR #25688: URL: https://github.com/apache/datafusion/pull/25688#issuecomment-6017924769
My current intuition is that we can keep refactoring the existing optimizer by making more invariants and boundaries explicit, as in #26069, and use them to continuously reduce overall complexity. I think the concerns from @alamb and @zhuqi-lucas are: - How should we enforce those invariants? - How far can we go with this approach? Adding more constraints might make future changes harder to make, and even increase maintenance overhead at some point. Those concerns make a lot of sense to me. My next step is to prototype the idea and try to identify/enforce ~3 more useful boundaries or invariants, to see whether this approach can consistently reduce complexity: - If the result is strong, I'll share it as a PoC PR. - Otherwise, I think the lighter-weight approach @zhuqi-lucas suggested makes sense: document these preconditions locally in the relevant optimizer rules, with debug assertions where practical, rather than introducing a general boundary mechanism. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
