Thanks for the intro to "GroupIntoBatches" Luke. It looks very interesting and easy for someone to understand when they read the pipeline. I will give it a try for my batched RPC use case.
Regards, Praveen Kumar Viswanathan On Fri, Jun 19, 2020 at 9:24 AM Luke Cwik <[email protected]> wrote: > I would suggest looking at the GroupIntoBatches[1] transform if you want > to support batching of elements instead of writing it yourself. Single RPC > calls should be invoked from the DoFn like you would from a regular program. > > 1: > https://beam.apache.org/documentation/transforms/java/aggregation/groupintobatches/ > > On Fri, Jun 19, 2020 at 8:20 AM Brian Hulette <[email protected]> wrote: > >> Kenn wrote a blog post showing how to do batched RPCs with the state and >> timer APIs: https://beam.apache.org/blog/timely-processing/ >> >> Is that helpful? >> >> Brian >> >> On Thu, Jun 18, 2020 at 5:29 PM Praveen K Viswanathan < >> [email protected]> wrote: >> >>> Hello Everyone, >>> >>> In my pipeline I have to make a *single RPC call* as well as a *Batched >>> RPC call* to fetch data for enrichment. I could not find any reference >>> on how to make these call within your pipeline. I am still covering my >>> grounds in Apache Beam and would appreciate if anyone has done this and >>> could share a sample code or details on how to do this. >>> -- >>> Thanks, >>> Praveen K Viswanathan >>> >> -- Thanks, Praveen K Viswanathan
