Re: How to merge two streams and perform stateful operations on merged stream using Apache Beam
Yeah !This is same question .. thank you for you detailed explanation On Thu, 25 Jul 2019 at 9:32 PM, Kenneth Knowles wrote: > Is this the same question as your other email about your StackOverflow > question? If it is, then please see my answer on StackOverflow. If it is > not, can you explain a little more? > > Kenn > > On Wed, Jul 24, 2019 at 10:48 PM Kiran Hurakadli > wrote: > >> I have 2 kafka streams , i want to merge by some key and on top of the >> merged stream i want to perform stateful operation so that i can sum up >> counts from both streams >> >> this what i tried, but dint work .. >> >> PCollection stream1 = .. read from kafka >> >> PCollection stream2 = .. read from kafka >> >> PCollection wonrdCount1 = stream1.apply(...) >> >> PCollection wonrdCount2 = stream2.apply(...) >> >> PCollection merged = merge wordcount1 and wordcount2 using >> CoGroupByKey >> >> >> >> Pcolection finalStream = mergred.apply(...) >> >> >> for finalstream apply state.. >> >> Please suggest solution .. >> >> -- >> Regards, >> *Kiran M Hurakadli.* >> > -- Regards, *Kiran M Hurakadli.*
Re: How to merge two streams and perform stateful operations on merged stream using Apache Beam
Is this the same question as your other email about your StackOverflow question? If it is, then please see my answer on StackOverflow. If it is not, can you explain a little more? Kenn On Wed, Jul 24, 2019 at 10:48 PM Kiran Hurakadli wrote: > I have 2 kafka streams , i want to merge by some key and on top of the > merged stream i want to perform stateful operation so that i can sum up > counts from both streams > > this what i tried, but dint work .. > > PCollection stream1 = .. read from kafka > > PCollection stream2 = .. read from kafka > > PCollection wonrdCount1 = stream1.apply(...) > > PCollection wonrdCount2 = stream2.apply(...) > > PCollection merged = merge wordcount1 and wordcount2 using > CoGroupByKey > > > > Pcolection finalStream = mergred.apply(...) > > > for finalstream apply state.. > > Please suggest solution .. > > -- > Regards, > *Kiran M Hurakadli.* >
How to merge two streams and perform stateful operations on merged stream using Apache Beam
I have 2 kafka streams , i want to merge by some key and on top of the merged stream i want to perform stateful operation so that i can sum up counts from both streams this what i tried, but dint work .. PCollection stream1 = .. read from kafka PCollection stream2 = .. read from kafka PCollection wonrdCount1 = stream1.apply(...) PCollection wonrdCount2 = stream2.apply(...) PCollection merged = merge wordcount1 and wordcount2 using CoGroupByKey Pcolection finalStream = mergred.apply(...) for finalstream apply state.. Please suggest solution .. -- Regards, *Kiran M Hurakadli.*
