This is awesome, thanks Sindy! I hope that the questions related to portability will get resolved in a way that will allow to reuse some of the work for other interactive Beam experiences, including SQL as Andrew says, and providing a REPL e.g. for users of Scala or other JVM-based languages.
+Neville Li <[email protected]> Do I remember correctly that you guys had some sort of interactivity going in Scio but were looking forward to Beam developing a native solution? On Wed, Jun 13, 2018 at 2:22 PM Sindy Li <[email protected]> wrote: > *Thanks, Andrew!* > > *Here is a link to the demo on Youtube for people interested:* > *https://www.youtube.com/watch?v=c5CjA1e3Cqw&feature=youtu.be > <https://www.youtube.com/watch?v=c5CjA1e3Cqw&feature=youtu.be>* > > On Wed, Jun 13, 2018 at 1:23 PM, Andrew Pilloud <[email protected]> > wrote: > >> This sounds really interesting, thanks for sharing! We've just begun to >> explore making Beam SQL interactive. The Interactive Runner you've proposed >> sounds like it would solve a bunch of the problems SQL faces as well. SQL >> is written in Java right now, so we can't immediately reuse any code. >> >> Andrew >> >> On Wed, Jun 13, 2018 at 11:48 AM Sindy Li <[email protected]> wrote: >> >>> Resending after subscribing to dev list. >>> >>> ---------- Forwarded message ---------- >>> From: Sindy Li <[email protected]> >>> Date: Fri, Jun 8, 2018 at 5:57 PM >>> Subject: Proposing interactive beam runner >>> To: [email protected] >>> Cc: Harsh Vardhan <[email protected]>, Chamikara Jayalath < >>> [email protected]>, Anand Iyer <[email protected]>, Robert Bradshaw < >>> [email protected]> >>> >>> >>> Hello, >>> >>> We were exploring ways to provide an interactive notebook experience for >>> writing Beam Python pipelines. The design doc >>> <https://docs.google.com/document/d/10bTc97GN5Wk-nhwncqNq9_XkJFVVy0WLT4gPFqP6Kmw/edit?usp=sharing> >>> provides >>> an overview/vision of what we would like to achieve. Pull request >>> <https://github.com/apache/beam/pull/5595> provides a prototype for the >>> same. The document also provides demo screen shots and instructions for >>> running a demo in Jupyter. Please take a look. We believe this would be a >>> useful addition to Beam. >>> >>> Thanks! >>> >>> >>> >>> >
