Thank you for letting me know. It is also available in the Python SDK - https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/util.py#L1656
However, it doesn't seem to meet the requirement that the side input values can change over time because the mentioned transform just seems to wait until the previous one gets completed. What I look into is, let's say, a customer attribute changes then order records should be enriched with the updated attribute. On Thu, 1 Aug 2024 at 13:14, LDesire <two_som...@icloud.com> wrote: > Hello. I found a similar example code. > You can use `Wait` PTransform. > Wait (Apache Beam 2.13.0) > <https://beam.apache.org/releases/javadoc/2.13.0/org/apache/beam/sdk/transforms/Wait.html> > beam.apache.org > <https://beam.apache.org/releases/javadoc/2.13.0/org/apache/beam/sdk/transforms/Wait.html> > [image: favicon.ico] > <https://beam.apache.org/releases/javadoc/2.13.0/org/apache/beam/sdk/transforms/Wait.html> > <https://beam.apache.org/releases/javadoc/2.13.0/org/apache/beam/sdk/transforms/Wait.html> > > Hope this helps. > > [image: stateful-beam-realtime.png] > > stateful-beam-realtime/pipeline/src/main/java/org/stjimmy/beam/LtvPipelineSqlLookup.java > at 2cc16a9cf8460c5b0e4d749e81654273c14ffb00 · Jimmyst/stateful-beam-realtime > <https://github.com/Jimmyst/stateful-beam-realtime/blob/2cc16a9cf8460c5b0e4d749e81654273c14ffb00/pipeline/src/main/java/org/stjimmy/beam/LtvPipelineSqlLookup.java#L92> > github.com > <https://github.com/Jimmyst/stateful-beam-realtime/blob/2cc16a9cf8460c5b0e4d749e81654273c14ffb00/pipeline/src/main/java/org/stjimmy/beam/LtvPipelineSqlLookup.java#L92> > > <https://github.com/Jimmyst/stateful-beam-realtime/blob/2cc16a9cf8460c5b0e4d749e81654273c14ffb00/pipeline/src/main/java/org/stjimmy/beam/LtvPipelineSqlLookup.java#L92> > > > 2024. 8. 1. 오전 11:52, Jaehyeon Kim <dott...@gmail.com> 작성: > > Hello, > > I'm looking into side input patterns especially slowly updating global > window side inputs - > https://beam.apache.org/documentation/patterns/side-inputs/ > > It'd be useful if we need to enrich eg) order records with customer > details where customer details would be taken as a side input. > > Let's say we have two Kafka topics, one for client records and the other > for order records. For the enrichment to work properly, consumption of > order records should wait until all customer records are read. > > Can you please inform me if it is achievable? > > Cheers, > Jaehyeon > > >
favicon.ico
Description: Binary data
