Hello,
I am working on experimental project on message distribution and load balancing
across cluster using Apache Kafka and Zookeeper. The goal of the project is to
equally distribute the messages to the cluster for concurrent processing.
For example; the server cluster contains 3 servers nam
I suggest you look up what a 'partition' is in kafka terms, they have
a quite good description at
http://incubator.apache.org/kafka/design.html
On Fri, Sep 14, 2012 at 2:57 AM, wrote:
> Hello,
>
> I am working on experimental project on message distribution and load
> balancing across cluster u
>
> Hi,
>
> In our application we are using Kafka for activity stream. We wanted to
> know if there is a way in Kafka to send a single message to multiple topics
> and store it only once, then later consume these messages from these topics
> individually and maintain their offsets separately.
>
>
Hi,
Is there any way to replicate data without waiting for Kafka 0.8.
Is anyone (incuding LinkedIn) doing this right now?
I can think of few ways to do it -- having an intermediary between kafka,
which writes to N partitions/brokers simultaneously. If anyone is already
doing it, please share your
Hello,
Sorry for doing thread necromancy on this one, but I have a little question
hehe... Can you confirm whether my understanding, below, is correct please?
1. Every time I extract a message from a KafkaMessageStream, it sets my
consumer offset to the offset of the beginning of the messag
Actually, every time you consume a message, the offset moves to the
beginning of the next message.
Thanks,
Jun
On Fri, Sep 14, 2012 at 6:52 PM, Felix GV wrote:
> Hello,
>
> Sorry for doing thread necromancy on this one, but I have a little question
> hehe... Can you confirm whether my understa
The ZK-based producer has a bug that may not send events to all brokers.
Try setting broker.list in the producer instead.
Thanks,
Jun
On Fri, Sep 14, 2012 at 2:57 AM, wrote:
> Hello,
>
> I am working on experimental project on message distribution and load
> balancing across cluster using Apac
Kafka supports multi-subscription. You can put the events in a single topic
and have different consumer groups consume the same topic. Each consumer
group maintains its own offsets.
Thanks,
Jun
On Fri, Sep 14, 2012 at 11:14 AM, Rohit Chandra <
rohitchandra.sa...@gmail.com> wrote:
> >
> > Hi,
>
There is a tool mirror maker that allows to you replicate data from 1 kafka
cluster to another. The replication in mirror maker is asynchronous though.
Thanks,
Jun
On Fri, Sep 14, 2012 at 1:53 PM, Rohit Prasad wrote:
> Hi,
>
> Is there any way to replicate data without waiting for Kafka 0.8.
>
