Among these options I would lean towards option 1. We already support a lot of infrastructure to call into Calcite for non-JDBC path, so adding some code to generate config does not seem like a big of a deal, especially if it will be a supported way at some point in Calcite.
Pulling implementation RelNode out of JDBC seems to bring a lot more unknowns: - it feels it goes against the JDBC approach as we're basically going around JDBC result sets; - we will expose 2 ways to extract results, with different schemas, types, etc; I think the third option is to implement the JDBC driver ourselves without using Calcite JDBC infrastructure. This way we have the only path into Calcite and control everything. I don't know how much effort it would take to implement a functional JDBC to cover our use cases though, but I think it's on a similar order of magnitude as we don't have to implement a lot of the API in the beginning, e.g. transactions, cursors, DML. On Wed, May 16, 2018 at 10:15 AM Kenneth Knowles <k...@google.com> wrote: > IIUC in #2 Beam SQL would live on the other side of a JDBC boundary from > any use of it (including the BeamSQL transform). I'm a bit worried we'll > have a problem plumbing all the info we need, either now or later, > especially if we make funky extensions to support our version of SQL. > > Kenn > > On Wed, May 16, 2018 at 10:08 AM Andrew Pilloud <apill...@google.com> > wrote: > >> I'm currently adding JDBC support to Beam SQL! Unfortunately Calcite has >> two distinct entry points, one for JDBC and one for everything else (see >> CALCITE-1525). Eventually that will change, but I'd like to avoid having >> two versions of Beam SQL until Calcite converges on a single path for >> parsing SQL. Here are the options I am looking at: >> >> 1. Make JDBC the source of truth for Calcite config and state. Generate a >> FrameworkConfig based on the JDBC connection and continue to use the >> non-JDBC interface to Calcite. This option comes with the risk that the two >> paths into Calcite will diverge (as there is a bunch of code copied from >> Calcite to generate the config), but is the easiest to implement and >> understand. >> >> 2. Make JDBC the only path into Calcite. Use prepareStatement and unwrap >> to extract a BeamRelNode out of the JDBC interface. This eliminates a >> significant amount of code in Beam, but the unwrap path is a little >> convoluted. >> >> Both options leave the user facing non-JDBC interface to Beam SQL >> unchanged, these changes are internal. >> >> Andrew >> >
