Hi Calcite Folks! The Apache Flink community is currently looking into how to use Calcite for optimization of both batch and streaming programs.
We are looking to compile two different kinds of higher level APIs via Calcite to Flink's APIs: - Table API (a LINQ-style DSL) - SQL Our current thought is to use largely the same translation paths for both, with different entry points into Calcite: - the Table API creates a Calcite operator tree directly - the SQL interface goes through the full stack, including parser, ... >From what I have seen so far in Calcite, it looks pretty promising, with its configurable and extensible rule set, and the pluggable schema/metadata providers. A few questions remain for us, to see how feasible this is: 1) Are DAG programs supported? The table API produces operator DAGs, rather than pure trees. Do DAGs affect/limit the space explored by the optimizer engine? 2) For streaming programs, we will probably want to add some custom syntax, specific to Flink's windows. Is it possible to also customize the SQL dialect of the parser? These answers are quite crucial for us figure out how to best use Calcite in our designs. Thanks for helping us... Greetings, Stephan
