This is awesome, thanks Sindy! I hope that the questions related to portability will get resolved in a way that will allow to reuse some of the work for other interactive Beam experiences, including SQL as Andrew says, and providing a REPL e.g. for users of Scala or other JVM-based languages.
+Neville Li <nevi...@spotify.com> Do I remember correctly that you guys had some sort of interactivity going in Scio but were looking forward to Beam developing a native solution? On Wed, Jun 13, 2018 at 2:22 PM Sindy Li <qiny...@google.com> wrote: > *Thanks, Andrew!* > > *Here is a link to the demo on Youtube for people interested:* > *https://www.youtube.com/watch?v=c5CjA1e3Cqw&feature=youtu.be > <https://www.youtube.com/watch?v=c5CjA1e3Cqw&feature=youtu.be>* > > On Wed, Jun 13, 2018 at 1:23 PM, Andrew Pilloud <apill...@google.com> > wrote: > >> This sounds really interesting, thanks for sharing! We've just begun to >> explore making Beam SQL interactive. The Interactive Runner you've proposed >> sounds like it would solve a bunch of the problems SQL faces as well. SQL >> is written in Java right now, so we can't immediately reuse any code. >> >> Andrew >> >> On Wed, Jun 13, 2018 at 11:48 AM Sindy Li <qiny...@google.com> wrote: >> >>> Resending after subscribing to dev list. >>> >>> ---------- Forwarded message ---------- >>> From: Sindy Li <qiny...@google.com> >>> Date: Fri, Jun 8, 2018 at 5:57 PM >>> Subject: Proposing interactive beam runner >>> To: dev@beam.apache.org >>> Cc: Harsh Vardhan <anan...@google.com>, Chamikara Jayalath < >>> chamik...@google.com>, Anand Iyer <ian...@google.com>, Robert Bradshaw < >>> rober...@google.com> >>> >>> >>> Hello, >>> >>> We were exploring ways to provide an interactive notebook experience for >>> writing Beam Python pipelines. The design doc >>> <https://docs.google.com/document/d/10bTc97GN5Wk-nhwncqNq9_XkJFVVy0WLT4gPFqP6Kmw/edit?usp=sharing> >>> provides >>> an overview/vision of what we would like to achieve. Pull request >>> <https://github.com/apache/beam/pull/5595> provides a prototype for the >>> same. The document also provides demo screen shots and instructions for >>> running a demo in Jupyter. Please take a look. We believe this would be a >>> useful addition to Beam. >>> >>> Thanks! >>> >>> >>> >>> >
