Thanks Luke for adding more context. The proposal was intentionally limited to only cover the FnApi proposal that adds the ability to batch a list of state requests at the end. The complete discussion for how the runner and SDK should do prefetching/batching is out of scope of this doc. Those could be covered by separate documents. The core idea is that having batching supported in FnApi will help runner/SDK implement their prefetching/batching strategy.
-Rui On Mon, Jun 14, 2021 at 3:21 PM Luke Cwik <lc...@google.com> wrote: > Enhancements to the SDK which allow for greater declaration of intent > would be useful but the overall issue is that the SDK can send multiple > read requests through the use of readLater() and then block on read() > without the runner being aware that the SDK is blocked. > > The runner could implement these strategies today: > * have X pending state lookups, gather all new incoming state requests > into one batch and as soon as one of the pending state lookups has finished > then issue the new batch. This likely will increase load on the state > lookup system since if there are two readLater()'s per element that are > back to back then those won't get batched together > * gather incoming state requests until there are X requests or Y time has > passed. This is strictly slower since we will have to wait till Y time has > passed pretty regularly. > > On Mon, Jun 14, 2021 at 3:05 PM Kenneth Knowles <k...@apache.org> wrote: > >> I didn't see user API or SDK changes that I would expect in this >> proposal. Maybe I missed it? The main big win for state batching in the >> runners core trigger & window implementation is batching requests across a >> whole bundle. Certainly across elements. This probably requires something >> like either: >> >> - a user API change (to have something like this: >> https://github.com/apache/beam/blob/f47a9424723e20abf807098dd6e9eef6e74c16cc/runners/core-java/src/main/java/org/apache/beam/runners/core/triggers/AfterAllStateMachine.java#L52 >> ) >> - the SDK to use metadata/annotations (like this: >> https://github.com/apache/beam/blob/f47a9424723e20abf807098dd6e9eef6e74c16cc/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java#L420) >> instead >> >> That will allow the hints to be gathered across multiple elements. And it >> would see the Fn API streaming protocol might mean the implementation is >> different than it is in ReduceFnRunner. >> >> Kenn >> >> On Mon, Jun 14, 2021 at 2:46 PM Luke Cwik <lc...@google.com> wrote: >> >>> The third approach prevents you from batching across state keys which >>> would be the most common type of batching. >>> >>> On Thu, May 6, 2021 at 3:13 PM Rui Wang <ruw...@google.com> wrote: >>> >>>> At this moment, the third approach in the doc is preferred. To recap, >>>> the third approach is the one that only changes FnApi by adding a repeated >>>> field in the state request to support batching over FnApi. >>>> >>>> This approach has the following benefits: >>>> 1. Avoid double requests problem introduced by prefetching (prefetching >>>> needs two requests, one for prefetch and one for blocking fetch). >>>> 2. This approach does not conflict with prefetching so no backward >>>> compatibility issue even when we want to add prefetching in FnApi. So this >>>> approach can be a good starting point. >>>> >>>> The caveat though is this approach does not support smart prefetching >>>> (which needs runners support). However we can add that in the future if >>>> necessary and that won't conflict with existing design. >>>> >>>> Please let us know if you have any objection before the implementation. >>>> >>>> >>>> -Rui >>>> >>>> On Mon, Mar 22, 2021 at 12:27 PM Rui Wang <amaliu...@apache.org> wrote: >>>> >>>>> Hi Community, >>>>> >>>>> Andrew Crites and I drafted a document to discuss how to support state >>>>> prefetching and batching over FnApi, which seems a missing functionality >>>>> in >>>>> FnApi. This will help us support Java state readLater() Api over FnApi. >>>>> >>>>> Please see: >>>>> https://docs.google.com/document/d/1Z3a5YOZyYsN8MeS6hRhCXX31m9bKCXSOtjKSl7wX40c/edit?usp=sharing&resourcekey=0-eiNl525kmb3Av2bqgCsZUA >>>>> >>>>> >>>>> -Rui >>>>> >>>>
