Re: Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map

2020-08-04 Thread Sunny Lohani
Hi Ryanne, I tried setting the groups in mm2.properties, still no luck. And, the consumer application is not kafka-console-consumer, so no issues there. I noticed in the MM logs that MirrorCheckpointConnector is created but without any connector task, not even a single error in the logs though. I

Re: best approach to rotate the logs and archive on Kubernetes

2020-08-04 Thread Srinivas Seema
Thank you Sachin, this solution is very helpful.. ~Srinivas On Tue, 4 Aug, 2020, 21:31 Sachin Mittal, wrote: > I think if you just log to the console, kubernetes will manage the log > rotation for you. > https://kubernetes.io/docs/concepts/cluster-administration/logging/ > You can use

Re: best approach to rotate the logs and archive on Kubernetes

2020-08-04 Thread Sachin Mittal
I think if you just log to the console, kubernetes will manage the log rotation for you. https://kubernetes.io/docs/concepts/cluster-administration/logging/ You can use "kubectl logs" command to fetch the logs or use some logging agent to move the logs centrally. On Tue, Aug 4, 2020 at 8:15 PM

Re: Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map

2020-08-04 Thread Ryanne Dolan
Sunny, check the groups and groups.blacklist properties. By default, MM2 won't replicate consumer groups from kafka-console-consumer, for example. Sometimes that confuses people when testing MM2. Also check the logs to see if there is any reason MirrorCheckpointConnector might be failing to

Re: Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map

2020-08-04 Thread Sunny Lohani
Hi Ryanne, First of all, thanks for a quick revert. Actually, I have a consumer application consuming messages in cluster A and then I failover the consumer from A to cluster B. Also, the consumer is subscribed to both local and remote topics. Let's say the local topic name in both the clusters A

Re: Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map

2020-08-04 Thread Ryanne Dolan
Sunny, is it possible there are no consumer groups? There will be no checkpoints, and thus nothing to use for offset translation, if there are no upstream consumer groups. Ryanne On Tue, Aug 4, 2020, 9:28 AM Sunny Lohani wrote: > Hi, > > I have 2 data centers, each having single node Zookeeper

[RESULTS] [VOTE] Release Kafka version 2.5.1

2020-08-04 Thread John Roesler
Hello all, This vote passes with four +1 votes (3 binding) and no 0 or -1 votes. +1 votes PMC Members (in voting order): * Ismael Juma * Manikumar Reddy * Mickael Maison Committers (in voting order): * John Roesler Community: * No votes 0 votes * No votes -1 votes * No votes Vote thread:

best approach to rotate the logs and archive on Kubernetes

2020-08-04 Thread Srinivas Seema
Hi All, We have Kafka cluster deployed in Kubernetes and running with docker image (solsson/kafka:2.4.0) I have below logging configuration: config/log4j.properties # Unspecified loggers and loggers with additivity=true output to server.log and stdout # Note that INFO only applies to

Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map

2020-08-04 Thread Sunny Lohani
Hi, I have 2 data centers, each having single node Zookeeper and Kafka cluster. I have a topic (single partition) in both the data center kafka clusters. I am using MM 2.0 as a dedicated cluster for bi-directional replication of the topic as well as using RemoteClusterUtils.translateOffsets() in

Re: [ANNOUNCE] New Kafka PMC Member: Mickael Maison

2020-08-04 Thread Mickael Maison
Thanks everyone! Regards On Mon, Aug 3, 2020 at 12:42 AM Matthias J. Sax wrote: > > Congrats Mickael! > > > -Matthias > > On 8/1/20 3:46 PM, Ismael Juma wrote: > > Congratulations Mickael! > > > > Ismael > > > > On Fri, Jul 31, 2020, 8:15 AM Jun Rao wrote: > > > >> Hi, Everyone, > >> > >>