I'm working on updating our Beam DDL code to use the DDL execution functionality that recently merged into core calcite. This enables us to take advantage of Calcite JDBC as a way to use Beam SQL. As part of that I need to reconcile the Beam SQL Environments with the Calcite Schema (which is calcite's environment). We currently have copies of our tables in the Beam meta/store, Calcite Schema, BeamSqlEnv, and BeamQueryPlanner. I have a pending PR which merges the later two to just use the Calcite Schema copy. Merging the Beam MetaStore and Calcite Schema isn't as simple. I have two options I'm looking for feedback on:
1. Make Calcite Schema authoritative and demote MetaStore to be something more like a Calcite TableFactory. Calcite Schema already implements the semantics of our InMemoryMetaStore. If the Store interface is just over built, this approach would result in a significant reduction in code. This would however eliminate the CRUD part of the interface leaving just the buildBeamSqlTable function. 2. Pass the Beam MetaStore into Calcite wrapped with a class translating to Calcite Schema (like we do already with tables). Instead of copying tables into the Calcite Schema we would pass in Beam meta/store as the source of truth and Calcite would manipulate tables directly in the Beam meta/store. This is a bit more complicated but retains the ability for DDL operations to be processed by a custom MetaStore. Thoughts? Andrew
