Hi Josh,
I think you probably mean something like buffering elements in a field on
the DoFn, emitting batches as appropriate, and emitting the remainder in
finishBundle.
Unfortunately there are two issues:
- in the presence of windowing the DoFn might be invoked in different
windows, so you'll
Hi Dev,
After offline discussions with Gus, I'd like propose we include a Batcher
function into contrib/. This would be a DoFn that behaves like this:
[1,2,3,4,5] -> Batcher(max_size=2) -> [[1,2],[3,4],[5]]
Its simple code, but it also shows off that values can still be yielded
from
