+1 now that I've read it. On Wed, Nov 3, 2021, 12:20 PM Robert Bradshaw <[email protected]> wrote:
> +1 to this proposal. (Interestingly, this was how the very first > prototype of the FnAPI worked, back before we had a working data plane > :). > > On Wed, Nov 3, 2021 at 12:10 PM Robert Burke <[email protected]> wrote: > > > > Nit: The FnAPI can also send particularly large iterable values over the > State API instead, (so called State Backed Iterables) but that doesn't > apply to the Small Bundle case being described here. > > Actually, there's no reason one couldn't send a state backed iterable > element embedded in the process bundle request, though of course there > would be less benefit there. > True! My nit was predominantly against the use of the word "only" in the OP. > > On Wed, Nov 3, 2021, 11:49 AM Pablo Estrada <[email protected]> wrote: > >> > >> Thanks for sharing Yichi. Do you have an idea of the potential gains / > the existing cost on small bundles? > >> Thanks > >> -P. > >> > >> On Wed, Nov 3, 2021 at 11:00 AM Yichi Zhang <[email protected]> wrote: > >>> > >>> Hi, beam dev community, > >>> > >>> Today the Fn API only uses the data plane for data transferring in > bundle processing, it is observed that the protocol has some extent of > inefficiency when dealing with small cheap bundles (mostly seen in > streaming pipelines). I put up a proposal to enable embedding of a small > amount of data in ProcessBundleRequest and ProcessBundleResponse in order > to circumvent such inefficiency. > >>> > >>> Proposal Doc: > https://docs.google.com/document/d/14p8Y_n4IY5n9L_I9l5x9lVGgml4ZzdCw645HldndCrw/edit?usp=sharing > >>> > >>> PTAL. > >>> > >>> Thanks, > >>> Yichi >
