Hi, A couple of thoughts;
1- If the amount of data in Hbase that you need to join with is small and does not change, could you use a Side Input? If it does change you could try making use of pattern slowly changing lookup cache (ref below). 2- If the amount of data is large, would a direct hbase client call from a DoFn work to get the data you need to enrich the element? Similar to pattern Calling external service, (ref below) Ref : https://cloud.google.com/blog/products/gcp/guide-to-common-cloud-dataflow-use-case-patterns-part-1 Cheers Reza On Tue, 11 Dec 2018 at 00:12, Shrijit Pillai <[email protected]> wrote: > Hello, > > I'm trying to join an unbounded data source and a bounded one using > CoGroupByKey. The bounded data source is HBase and the unbounded one is > Kafka. > > The co-group works if the global window strategy is used but not with a > non-global one. I've tried the accumulatingFiredPanes mode(using the > non-global window) but that didn't help either. Am I missing something to > make the co-group work using a non-global window like FixedWindows or is > the GlobalWindow the only way to go about it? I'm using beam 2.8.0 > > Here's the code snippet: > https://gist.github.com/shrijitpillai/5e9e642f92dd23b3b7bd60e3ce8056bb > > Thanks > Shrijit >
