Thanks Max for sharing. On Fri, Oct 5, 2018 at 7:05 AM Maximilian Michels <m...@apache.org> wrote:
> Hi, > > What do you think about collecting some of the feedback from the > community at Beam Summit last week? Here's what I've come across: > > > * The Kubernetes / Docker Story > > Multiple users reported that they would like a Beam-Kubernetes story. > What is the best way to deploy Beam with Kubernetes? Will there be > built-in support? > > Especially with regards to the portability, there are some unsolved > problems, e.g. how to start Beam containerized and bootstrap the SDK > Harness container from within a container? For local testing with the > JobServer we support that via mounting the Docker socket, but this will > be too fragile in production scenarios. Now that we have process-based > execution, we could just use that inside the main container. > > Deployment is a very important topic for users and we should try to > reduce complexity as much as possible. > > * External SDKs / Scio > > Users have asked why Scio is not part of the main repository. Generally, > I don't think that has to be the case, same for the Runners which are > not part of the main repo. However, it does raise the question, what > will be the future model for maintaining SDKs/IOs/Runners? How do we > ensure easy development and a consistent quality of internal/external > components? > > * Documenting Timers & State > > These two have excellent blog posts but are not part of the official > documentation. Since they are part of the model, it would be good to > eventually update the docs. > > * Better Debuggability of pipelines > > Even a simple WordCount in Beam leads to a quite complex Flink execution > graph (due to the the involved I/O logic). How can we make pipelines > easier to understand? Will we provide a way to visualize the > architecture of high-level Beam pipelines? If so, do we provide a way to > gain insight into how it is mapped to the Runner execution model? Users > would like to have more insight. > > * Current Roadmap > > This was asked in the context of portability. By the end of the year we > should have at least the FlinkRunner in a ready state, with the rest > following up. There are a lot of others threads in Beam. The newsletter > is a great way to keep up with the project development. > > > Looking forward to any other points you might have. > > Best, > Max >
