This sounds really interesting, thanks for sharing! We've just begun to explore making Beam SQL interactive. The Interactive Runner you've proposed sounds like it would solve a bunch of the problems SQL faces as well. SQL is written in Java right now, so we can't immediately reuse any code.
Andrew On Wed, Jun 13, 2018 at 11:48 AM Sindy Li <[email protected]> wrote: > Resending after subscribing to dev list. > > ---------- Forwarded message ---------- > From: Sindy Li <[email protected]> > Date: Fri, Jun 8, 2018 at 5:57 PM > Subject: Proposing interactive beam runner > To: [email protected] > Cc: Harsh Vardhan <[email protected]>, Chamikara Jayalath < > [email protected]>, Anand Iyer <[email protected]>, Robert Bradshaw < > [email protected]> > > > Hello, > > We were exploring ways to provide an interactive notebook experience for > writing Beam Python pipelines. The design doc > <https://docs.google.com/document/d/10bTc97GN5Wk-nhwncqNq9_XkJFVVy0WLT4gPFqP6Kmw/edit?usp=sharing> > provides > an overview/vision of what we would like to achieve. Pull request > <https://github.com/apache/beam/pull/5595> provides a prototype for the > same. The document also provides demo screen shots and instructions for > running a demo in Jupyter. Please take a look. We believe this would be a > useful addition to Beam. > > Thanks! > > > >
