Re: custom offsets in ProduceRequest

Hi! > On 30 Dec 2016, at 09:52, radai wrote: > > or even better - if topic creation is done dynamically by the replicator, > setting the initial offsets for partitions could be made part of topic > creation ? even less API changes this way > > On Thu, Dec 29, 2016

Re: custom offsets in ProduceRequest

or even better - if topic creation is done dynamically by the replicator, setting the initial offsets for partitions could be made part of topic creation ? even less API changes this way On Thu, Dec 29, 2016 at 10:49 PM, radai wrote: > ah, I didnt realize we are

Re: custom offsets in ProduceRequest

ah, I didnt realize we are limiting the discussion to master --> slave. but - if we're talking about master-slave replication, and under the conditions i outlined earlier (src and dest match in #partitions, no foreign writes to dest) it "just works", seems to me the only thing youre really

Re: custom offsets in ProduceRequest

> On 29 Dec 2016, at 20:43, radai wrote: > > so, if i follow your suggested logic correctly, there would be some sort of > : > > produce(partition, msg, requestedOffset) > > which would fail if requestedOffset is already taken (by another previous > such explicit

Re: custom offsets in ProduceRequest

so, if i follow your suggested logic correctly, there would be some sort of : produce(partition, msg, requestedOffset) which would fail if requestedOffset is already taken (by another previous such explicit call or by another regular call that just happened to get assigned that offset by the

Re: custom offsets in ProduceRequest

Hi! > On 27 Dec 2016, at 19:35, radai wrote: > > IIUC if you replicate from a single source cluster to a single target > cluster, the topic has the same number of partitions on both, and no one > writes directly to the target cluster (so master --> slave) the offsets

Re: custom offsets in ProduceRequest

IIUC if you replicate from a single source cluster to a single target cluster, the topic has the same number of partitions on both, and no one writes directly to the target cluster (so master --> slave) the offsets would be preserved. but in the general case - how would you handle the case where

custom offsets in ProduceRequest

Hi all! Suppose you have two Kafka clusters and want to replicate topics from primary cluster to secondary one. It would be very convenient for readers if the message offsets for replicated topics would be the same as for primary topics. As far as I know, currently there is no way to achieve