roger-mike commented on PR #16911: URL: https://github.com/apache/beam/pull/16911#issuecomment-1111399812
Hi @apilloud, I asked the Calcite devs about this task, specifically about how to create a Relational Operator Tree for a window function. I changed the approach and added the window functions as ```RexOver``` to a ```LogicalProject```, instead of building the tree from a ```LogicalWindow```. This way we build a simpler tree and let part of the conversion to the Calcite planner. This is the comment I received from Julian Hyde on that matter: > It’s complicated. Windowed aggregates start off as instances of RexOver expressions inside a Project and are later converted (by ProjectToWindowRule [1]) into Window relational operators. > > Both of these representations exist in RelNode-land and I’m not sure which should be considered ‘canonical’. Window is a little bit more ‘physical’ than Project. At some stage in planning, it is assumed that all RexOver instances have been converted. > > > Julian > > [1] https://github.com/apache/calcite/blob/bebe473fab2e242736614659ed6e5d04eeeb8bf5/core/src/main/java/org/apache/calcite/rel/rules/ProjectToWindowRule.java#L74 <https://github.com/apache/calcite/blob/bebe473fab2e242736614659ed6e5d04eeeb8bf5/core/src/main/java/org/apache/calcite/rel/rules/ProjectToWindowRule.java#L74> So far so good. I tested and it seems it creates a valid BEAMPlan. The issue here is that I had to hardcode the value of this [index ]([beam/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java](https://github.com/apache/beam/blob/77e924fd5a025326b50352c29598ea1eb48d385a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java#L213))to make it point to a valid column reference. Not sure how to solve this issue. I was hoping you could give me some pointers on this. Thank you. P.S. I also got this one comment, I think the Calcite community might be interested in this translation feature. I said I was going to pass the message. What do you think? > It’s very interesting that you want to convert ZetaSQL to and from Calcite. I think other people will be interested in this. (It would help some of the things I am doing in my $dayjob at Google.) Would you consider contributing this as a new module in Calcite? Or creating a standalone project on GitHub? > > Julian -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
