Hello Aki, Can you elaborate on "when there are some messages left in the original cluster" ?
On 2021/01/12 13:01:40, Aki Yoshida <elak...@gmail.com> wrote: > Hi Ryanne, > Thanks for the information regarding the manual translation approach. > But for 2.7.0, I have a question. I thought this translation would > happen implicitly. > I saw test testOneWayReplicationWithAutoOffsetSync of > MirrorConnectorsIntegrationTest that is supposed to test this > behavior. But I have an impression that the consumers can't be > migrated when there are some messages left in the original cluster. I > posted this question to users@kafka a while ago but I didn't receive > any responses. > https://lists.apache.org/thread.html/r0728e9ea89b2713865a33396f108027ac3f0949c2496583532ee963c%40%3Cusers.kafka.apache.org%3E > Could you or someone comment on this? > Thanks. > Regards, aki > > El sáb, 2 ene 2021 a las 20:31, Ryanne Dolan (<ryannedo...@gmail.com>) > escribió: > > > > Aki, that's right. Prior to 2.7, you can use the translateOffsets method > > from within your client code, or you can write a little command-line tool > > to do it. I've done the latter for Cloudera's srm-control tool, as > > documented here: > > https://docs.cloudera.com/csp/2.0.1/srm-using/topics/srm-migrating-consumer-groups.html > > > > Ryanne > > > > On Thu, Dec 17, 2020, 5:23 PM Aki Yoshida <elak...@gmail.com> wrote: > > > > > I'm answering to my own mail and would like to hear if this assumption > > > is correct. > > > it looks like I need 2.7.0 to have the automatic translation. > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-545%3A+support+automated+consumer+offset+sync+across+clusters+in+MM+2.0 > > > for earlier versions, there is no way to use the console client but > > > use java API RemoteClusterUtils.translateOffsets() to get the offset > > > and re-consuming from the new topic. > > > Is this correct? > > > > > > El jue, 17 dic 2020 a las 21:18, Aki Yoshida (<elak...@gmail.com>) > > > escribió: > > > > > > > > Hi, > > > > I have a question regarding how to migrate a consumer when the > > > > subscribing topic has been migrated to the target Kafka broker. > > > > Suppose a consumer is consuming from topic1 at the source Kafka > > > > broker. When MM2 mirrors this topic using options > > > > source.cluster.alias="", replication.policy.separator="", topic named > > > > topic1 shows up at the target Kafka broker. > > > > > > > > When the consumer instance simply switches the bootstrap server to > > > > start consuming from this topic1 at the target broker, the consumer > > > > seems to start subscribing from the latest offset. I thought the > > > > consumer offsets were also migrated to the target and the consumer > > > > could simply start subscribing from the mirrored topic from the > > > > continued offset. Did I miss some configuration parameters or does the > > > > consumer need to perform some actions to be able to consume the > > > > records seamlessly? > > > > > > > > I appreciate for your help. Thanks. > > > > > > > > regards, aki > > > >
