Re: More consumer *groups* than topics OK?

[Jens Geyer]

Hi Stu,

that sounds very much like a PUB/SUB scenario to me, is that correct?
In that case, I'd question whether an RPC framework like Thrift is the right 
tool for the job.

That's not to say that you can't build things like this, or that you can't 
use Thrift with Messaging or MQ systems. Have a look in the contrib folder.

Have fun,
JensG


-----Ursprüngliche Nachricht----- 
From: Stuart Reynolds
Sent: Thursday, February 26, 2015 1:42 AM
To: user@thrift.apache.org
Subject: More consumer *groups* than topics OK?

Suppose I have 1 topic with 4 partitions, and I want to do 10
different things with every published element.

Can I have 10 processes that consume *every* message sent to the topic?

That is to say:

for consumerGroup_i in TenConsumerGroupsNames:
  runConsumerConnectorThread(topic, consumerGroup_i)
     // launches ConsumerConnector runs 4 consumers, one for each partition

OR (equivalently. I think)

for consumerGroup_i in TenConsumerGroupsNames:
    for ( partition in 0..3 )
         launchConsumerThread(topic, consumerGroup_i, partition)

For a given consumer group, #consumers == #partitions, but I have many 
groups.

Should I expect each *group* to receive every message published to the 
topic?



Thanks,
- Stu