Hi everyone, You may remember the proposal [1] I sent out early this year aiming to unify Beam SQL Tables and Row-oriented cross-language IOs. I received a lot of feedback that it was a good idea, but the specific implementation was a little odd since it relied on existing Beam SQL concepts.
I've re-written the proposal with a different approach: adding a new set of interfaces to the core Beam Java SDK [2]. These interfaces can be used for both Beam SQL Tables, as well as for cross-language IOs. Please take a look and let me know what you think! Full disclosure: there is a Google intern starting this week that I would like to have help out with this work once they are ramped up. Thanks, Brian [1] https://s.apache.org/xlang-table-provider [2] https://s.apache.org/beam-schema-io On Mon, Mar 30, 2020 at 3:38 PM Robert Bradshaw <[email protected]> wrote: > A belated but very enthusiastic +1 to this proposal. Added some > comments to the doc. > > On Thu, Jan 16, 2020 at 9:05 AM Kenneth Knowles <[email protected]> wrote: > > > > Nice! This is quite clever. > > > > Kenn > > > > On Mon, Jan 13, 2020 at 5:08 PM Chamikara Jayalath <[email protected]> > wrote: > >> > >> Thanks Brian. Added some comments. > >> > >> On Mon, Jan 13, 2020 at 2:25 PM Brian Hulette <[email protected]> > wrote: > >>> > >>> Hi everyone, > >>> I have a proposal that I think can unify two problem sets: > >>> 1) adding more IOs for Beam SQL, and > >>> 2) making more (Row-based) Java IOs available in Python as > cross-language transforms > >>> > >>> The basic idea is to create a single cross-language transform that > exposes all Beam SQL IOs via the TableProvider interface. A design document > is available here: https://s.apache.org/xlang-table-provider > >>> > >>> Please take a look and let me know what you think. Thanks! > >>> Brian >
