On Fri, May 11, 2018 at 11:40 AM Kenneth Knowles <[email protected]> wrote:
> This is great. "The Beam Vision in a spreadsheet" and/or what the > capability matrix wishes it always had been. > > - I don't know how to interpret the DirectRunner column. Is it that it > uses ye olde proto round trip? Another level is that it actually directly > links in the SDK harness as a dep and uses the exact code paths (seems like > overkill). > > Its up to the direct runner here to decide what level of execution is actually done via portability APIs but it is meant to be a single process to ease debugging for users. > - For runners*SDK pairs that don't have a batch/streaming distinction how > about collapsing the columns? > > Runners may not have a distinction but the portability framework may require more work from a runner to support a use case. A good example of this is side input readiness checking for streaming pipelines. > - Anyone have spreadsheet-fu to do a permanent global automatic > hyperlinking of BEAM-xxxx? > > Kenn > > On Fri, May 11, 2018 at 10:38 AM Henning Rohde <[email protected]> wrote: > >> Hi everyone, >> >> While the portability framework moves forward, it is often hard to >> figure out exactly what is supported to work at any given time. There >> are still many irregularities, TODOs, bugs and small differences between >> batch and streaming and the portable SDK and runner implementations. For >> example, the answer to the question "Does Wordcount run portably?" >> depends on the SDK, Runner and where the output is written. >> >> To this end, I've started a spreadsheet to better track the "swiss >> cheese" of what works portably: >> >> >> https://docs.google.com/spreadsheets/d/1KDa_FGn1ShjomGd-UUDOhuh2q73de2tPz6BqHpzqvNI/edit?usp=sharing >> >> Note that is is a work in progress. The intended audience is for >> everyone working on or interested in portability. I am hoping we can >> populate, expand and maintain the information as a community, until the >> portability framework support is mature enough to allow SDKs and runners to >> be considered independently. >> >> Comments and suggestions welcome! >> >> Thanks, >> Henning >> >> >> >>
