Hi Steve, The way that we currently represent lateral views on the physical plan is not great. I'm sure there are good reasons of why people went forward with the approach of introducing specialized operators for that purpose but as the history has shown the current representation is causing us lots of problems in many parts of the compiler where we need to traverse the plan DAG (rules + hooks + explain) leading to exponential visit of the plan nodes.
If we decorrelate the plan as usual maybe we could get rid of the problems mentioned above and the need for specialized lateral view physical operators which I think would be a good thing in the long run. Best, Stamatis On Thu, Apr 6, 2023, 6:31 PM Stephen Carlin <scar...@cloudera.com.invalid> wrote: > Hi, > > I noticed recently that for most lateral views, we do not convert to a CBO > plan. I was hoping to make some changes to make this possible. > > I was wondering if anyone out there had any thoughts on how to do this. > > If not, I did have one in mind and wanted to bring it up just to start the > conversation before I did any work on it. > > I was thinking maybe we could have our own RelNode deriving off of > LogicalCorrelate. I know we already handle correlated queries and > decorrelate them. But in this special case, I think we should not > decorrelate the RelNode. We should leave this new LogicalCorrelate node in > through the whole optimization process and then put in code to translate > the RelNode into the physical plan. > > Thoughts? > > Thanks! >
