Thanks for picking this up again Pablo, I wrote some small comments concerning the TestStream.
On Tue, Oct 15, 2019 at 4:42 PM Robert Bradshaw <[email protected]> wrote: > Very excited to see this! I've added some comments to the doc. > > On Tue, Oct 15, 2019 at 3:43 PM Pablo Estrada <[email protected]> wrote: > >> I've just been informed that access wasn't open. I've since opened access >> to it. >> Thanks >> -P. >> >> On Tue, Oct 15, 2019 at 2:10 PM Pablo Estrada <[email protected]> wrote: >> >>> Hello all, >>> I am planning to work on removing the old BundleBasedDirectRunner, and >>> expand the FnApiRunner to work on streaming as well as batch. >>> Currently, the FnApiRunner orders the processing graph topologically, >>> and "pushes" all the data through each stage in topological order (deferred >>> inputs such as residuals and timers are immediately pushed to the SDK as >>> well). >>> The new design would change from this >>> push-all-data-through-topologically-sorted-stages model to having queues >>> for "bundles", or for elements that are awaiting processing, and routing >>> them to the appropriate bundle processing subgraph. >>> >>> The design is here: http://s.apache.org/streaming-fn-runner-py >>> >>> I expect >>> >>> I'd appreciate comments and everything : ) >>> Best >>> -P. >>> >>
