JanKaul commented on issue #17719: URL: https://github.com/apache/datafusion/issues/17719#issuecomment-3394117894
Sorry @LorrensP-2158466, I wanted to experiment with how a JoinGraph should look like and went ahead and implemented a sketch of the API in [#18023](https://github.com/apache/datafusion/pull/18023). The draft PR assumes Datafusion provides a JoinGraph/QueryGraph structs and implements the closed form join ordering algorithm presented here: https://youtu.be/CcUVvnYv7Hg?list=PLSE8ODhjZXjYCZfIbmEWH7f6MnYqyPwCE&t=1952 The implementation raised a couple of questions: ### QueryGraph relation to original LogicalPlan Should the QueryGraph store some kind of index/reference as nodes and only reference parts of the original LogicalPlan or should the QueryGraph decompose the original LogicalPlan and actually store parts of it. I feels this decision affects whether the complexity lies more at the side of constructing the QueryGraph from the LogicalPlan or outputting the optimal plan with the use of the QueryGraph. I think using indexes/references makes outputting the optimized plan more difficult whereas constructing the QueryGraph from a LogicalPlan is more complex when the LogicalPlan is to be decomposed. ### Should we perform certain transformations before creating the QueryGraph from the original LogicalPlan? It feels like it is easier to create the QueryGraph from a LogicalPlan if the LogicalPlan is in a certain shape. I was thinking that the construction is easiest if there aren't any other LogicalPlan nodes between the join nodes. I'm not sure this is entirely possible but it would be helpful if all filters & projections are either pushed down below the joins or are pushed up before any join. Maybe you guys have some thought or experience on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
