You can try setting different txnId values for spout which is used in maintaining opaque transactional spout's state in ZK. I guess you may have same txnId for both of those topologies.
TridentTopology#newStream(txnId, spout); On Wed, Jul 27, 2016 at 2:20 PM, Satish Duggana <[email protected]> wrote: > Hi Amber, > I thought you wanted to set clientId to be used. I missed other part of > the question to set consumer group to consume messages within that topology. > > > > On Wed, Jul 27, 2016 at 1:03 PM, Amber Kulkarni < > [email protected]> wrote: > >> Tried deploying 2 topologies using above change, did not work. Still both >> are not getting same data from kafka. >> I think 3rd param is client id and not consumer group id. any ideas ? >> >> On Wed, Jul 27, 2016 at 12:21 PM, Satish Duggana < >> [email protected]> wrote: >> >>> You can use below constructor and use respective clientId. >>> >>> TridentKafkaConfig(BrokerHosts hosts, String topic, String clientId) >>> >>> Can not you see that constructor in the version that you are using? I >>> guess you should have that. >>> >>> Thanks, >>> Satish. >>> >>> >>> On Wed, Jul 27, 2016 at 11:42 AM, Amber Kulkarni < >>> [email protected]> wrote: >>> >>>> Hey, >>>> >>>> Cannot find a way to specify consumer ids to spout in trident. >>>> Can you suggest a way to do this. >>>> What I have tried: TridentKafkaConfig does not seem to have a >>>> constructor which takes group id, >>>> whereas Spoutconfig has the provision for id. >>>> I am using Storm 0.9.6 >>>> >>>> >>>> On Tue, Jul 26, 2016 at 6:12 PM, Stephen Powis <[email protected]> >>>> wrote: >>>> >>>>> Do you have unique consumer Ids for each spout in each topology? >>>>> >>>>> On Tue, Jul 26, 2016 at 8:40 AM, Amber Kulkarni < >>>>> [email protected]> wrote: >>>>> >>>>>> So I want multiple storm topologies to read from the same kafka topic. >>>>>> >>>>>> So suppose topic contains: A B C (data) >>>>>> >>>>>> I want both the topologies to get A,B,C >>>>>> >>>>>> But currently what is happening is messaged are getting >>>>>> distributed.(eg. topology 1 is getting A and other topology is getting B >>>>>> and C) >>>>>> >>>>>> How do I achieve this is trident. >>>>>> >>>>>> I am using OpaqueTridentKafkaSpout. >>>>>> >>>>>> I think what needs to be done is both topologies need to specify >>>>>> different consumer group ids. >>>>>> >>>>>> But could not find how to specify them in trident(TridentKafkaConfig) >>>>>> >>>>>> Any ideas ? >>>>>> >>>>>> Regards, >>>>>> Amber Kulkarni >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Amber Kulkarni >>>> >>> >>> >> >> >> -- >> Regards, >> Amber Kulkarni >> > >
