It doesn't, I want to use the translation layer outside of beam. A good chunk of the code in the library is beam agnostic, but it also contains a lot of beam dependencies that I don't want to pull in.
I think if ZetaSQLPlannerImpl and its dependency graph were separate, that'd be all that's needed for it to stand alone. From what I can tell (I ended up extracting the code into my own repo), there are very few beam dependencies involved there. On Thu, Mar 26, 2020 at 5:23 PM Rui Wang <[email protected]> wrote: > Hi Steve, > > Could you clarify a bit: could you use [1] directly to solve your case? If > not, why? > > > [1]: > https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-extensions-sql-zetasql/2.17.0 > > > > > -Rui > > On Thu, Mar 26, 2020 at 1:23 PM Steve Niemitz <[email protected]> wrote: > >> Oh I think I actually remember seeing that email on the calcite list. :) >> >> I agree that it being an alternate parser implementation in calcite >> itself would be ideal, but also agree (sadly) that that'll probably be a >> very slow process. >> >> Splitting it into its own library in beam seems ideal, the only problem I >> can see is that beam is using a vendored version of calcite. I think to be >> useful the library itself would need to use a "stock" version of calcite. >> >> I think I'd have some time to spend on this as well, if we can figure out >> a good way forward and agree on splitting it out. >> >> On Thu, Mar 26, 2020 at 4:04 PM Andrew Pilloud <[email protected]> >> wrote: >> >>> I think it makes sense for the ZetaSQL to Calcite translation layer to >>> live in Calcite itself, and did suggest it at one point on their dev list >>> (See: >>> https://lists.apache.org/thread.html/38942fcb4775ed71f9b2ab8880ab68a4238166ea5e904111ca184a12%40%3Cdev.calcite.apache.org%3E). >>> I don't think there is a quick way to get there, but we could split up the >>> interfaces within Beam so they are cleaner. >>> >>> It seems like a good next step would be to split up packages within >>> Beam. We could add a set of core SQL interfaces that only depend on Calcite >>> and then split our ZetaSQL translator into a piece that only depends on >>> those interfaces, Calcite, and ZetaSQL. >>> >>> Andrew >>> >>> On Thu, Mar 26, 2020 at 12:41 PM Steve Niemitz <[email protected]> >>> wrote: >>> >>>> The ZetaSQL to calcite translation layer that is bundled with beam >>>> seems generally useful in cases other than for beam. In fact, we're using >>>> (essentially a fork of) it internally outside of beam right now (and I've >>>> fixed a bunch of bugs in it). >>>> >>>> Has there ever been any thought about splitting into a separate library >>>> without any beam dependencies? >>>> >>>
