Re: Kafka mirror maker help

I run instances of Mirror Maker as supervisord tasks (http://supervisord.org <http://supervisord.org/>). I’d recommend looking into it. In addition to letting you sidestep the service issue, supervisord watches the processes and can auto-restart them if they stop for any reason. —

Re: multi-disk brokers data replication

, assuming you're running 1.0+. If you're not running 1.0+, then a single disk failure on a broker configured with JBOD will bring down the broker. Hope this helps, Peter Bukowinski On Thu, May 10, 2018 at 1:49 AM, Andrian Jardan <andrianjar...@gmail.com> wrote: > Hello everyo

Re: multi-disk brokers data replication

Oops, sorry about the name misspelling, Andrian. (spell-check just tried to correct it again). On Thu, May 10, 2018 at 11:41 AM, Peter Bukowinski <pmb...@gmail.com> wrote: > Adrian, > > Replicas are *always* assigned to different brokers. You cannot, for > example, depl

Re: multi-disk brokers data replication

version <1, the whole broker dies ? > > What happens when the disk is replaced then ? > > > On May 10, 2018, at 20:42, Peter Bukowinski <pmb...@gmail.com> wrote: > > > > Oops, sorry about the name misspelling, Andrian. (spell-check just tried > to > > c

Re: How frequent does the follower replica issue a fetch request to leader replica?

in the num.replica.fetchers broker parameter). For one of my small 5-broker clusters, my follower request rates range from 130 per second on the high end to 3 per second on the low end. I’ve configured 16 fetchers. -- Peter Bukowinski > On Apr 10, 2018, at 10:54 PM, Yu Watanabe <yu.w.ten...@gma

Re: log retention bytes and log segment bytes

Hi Amit, This is from the broker config section of the very good documentation on the kafka web site: https://kafka.apache.org/0100/documentation.html#brokerconfigs log.segment.bytes: The maximum size of a single log file (default 1GB) log.retention.bytes: The maximum size of the log before

Re: Default kafka log.dir /tmp | tmp-file-cleaner process

for config parameters. That being said, perhaps this particular config parameter should be highlighted for update when moving to production. -- Peter Bukowinski > On Apr 18, 2018, at 11:27 AM, adrien ruffie <adriennolar...@hotmail.fr> wrote: > > Hi Marc, > > > I t

Re: Using Kafka CLI without specifying the URLs every single time?

call the wrapper scripts by passing the name of the cluster as an argument and then passing the standard kafka options, e.g. ktopics --cluster my_cluster --list -- Peter Bukowinski > On Apr 20, 2018, at 3:23 AM, Horváth Péter Gergely > <horvath.peter.gerg...@gmail.com> wrote: &g

Re: Kafka Brokers compatibility with Jumbo Frames option.

is not set to jumbo frames, you will most likely see dropped or fragmented packets. — Peter Bukowinski > On Mar 31, 2018, at 6:51 AM, Pena Quijada Alexander <a.penaquij...@reply.it> > wrote: > > Hi all, > > My name is Alexander, a Linux System Administrator. I'm using K

Re: Number of partitions for offsets topic cannot be changed

cluster data from zookeepers, then starting brokers again with your desired server properties settings. -- Peter Bukowinski > On Mar 25, 2018, at 1:01 PM, Anu P <iamkafkau...@gmail.com> wrote: > > Thanks Swapnil. > > I changed *offsets.topic.num.partitions* in server.properties

Re: Visual tool for kafka?

https://github.com/yahoo/kafka-manager This does all that, and has admin features (which you can disable) that allow you to change or create topics, do partition reassignment and preferred leader election. — Peter > On Oct 18, 2018, at 11:52 PM,

Re: When Kafka stores group information in zookeeper?

It all depends on which type of consumer you are using. If you use an old (original) consumer, you must specify one or more zookeepers since group management info is stored in zookeeper. If you use a new consumer, group management is handled by the kafka cluster itself so you must specify one

Re: Open Source Schema Registry

Have a look at https://github.com/confluentinc/schema-registry > On Oct 23, 2018, at 9:28 AM, chinchu chinchu wrote: > > Hi folks, > We are looking to use open source schema registry with apache kafka 1.0.1 > and avro. *Do we need to write a

Re: Total Volume metrics of Kafka

Amitav, When you say total volume, do you want a topic’s size on disk, taking into account replication and retention, or do you want the aggregate bytes published into a topic? If you have a metrics collector and a graphing tool such as grafana, you can transform the rate metrics to a byte sum

Re: Total Volume metrics of Kafka

t; collector that I can use? Is it sort of a cron-job that notes the rate > every minute or every 15 mins? > > Regards, > Amitav > > On Thu, Jan 17, 2019 at 3:23 AM Peter Bukowinski wrote: > >> Amitav, >> >> When you say total volume, do you want a topic’s size on

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

I don’t want to be a downer, but because kafka is relatively new, the reference material you seek probably doesn’t exist. Kafka is flexible and can be made to work in many different scenarios — not all of the ideal. It sounds like you’ve already reached a conclusion that kafka is the wrong

Produce Message Conversions Per Sec

for the conversion metrics I’m seeing? If so, I’ll stop worrying since it is working as intended. — Peter Bukowinski

Re: Kafka partitioning and auto-scaling in k8s

I’ll assume when you say load, you mean data rate flowing into your kafka topic(s). One instance can consume from multiple partitions, so on a variable load workflow, it’s a good idea to have more partitions than your average workload will require. When the data rate is low, fewer consumers

Re: Any way to set a quota for a consumer group?

You can set consumer client.id to be the same as the consumer group.id for all the consumer in your consumer group to accomplish this. — Peter > On Feb 21, 2019, at 7:56 AM, 洪朝阳 <15316036...@163.com> wrote: > > It’s very great that Apache Kafka get a feature of setting quota since 0.9. >

Re: Proxying the Kafka protocol

https://docs.confluent.io/3.0.0/kafka-rest/docs/intro.html The Kafka REST proxy may be what you need. You can put multiple instances behind a load balancer to scale to your needs. -- Peter (from phone) > On Mar 19, 2019, at 8:30 AM, Ryanne Dolan wrote: > > Hello James, I'm not aware of

Re: Kafka Mirror Maker place of execution

> cluster) on each of the 100 Kafka Nodes of your core datacenter cluster. > So you run the Mirror Maker on the same machine as the Kafka Nodes and do not > use a dedicated machines for the Mirror Maker process? > > > Best regards, > Franz > > > Gesendet: D

Re: Kafka Mirror Maker place of execution

with about 25 threads, so each broker runs 25*30=750 threads of MirrorMaker. I’ve been running this setup for many months and it’s proved to be stable with very low consumer lag. -- Peter Bukowinski > On Mar 12, 2019, at 6:42 AM, Ryanne Dolan wrote: > > Franz, you can run MM on or ne

Re: Whether kafka broker will have impact with 2 MB message size

We have many production clusters with three topics in the 1-3MB range and the rest in the multi-kb to sub-kb range. We do use gzip compression, implemented at the broker rather than the producer level. The clusters don’t usually break a sweat. We use MirrorMaker to aggregate these topics to a

Re: Lag checking from producer

From your description, it sounds like kafka may be ill-suited for your project. A backpressure mechanism essentially requires producers to be aware of consumers and that is counter to Kafka’s design. Also, it sounds like your producers are logical (if not actual) consumers of data generated by

Re: Kafka memory estimation

The memory that a kafka broker uses is the java heap + the page cache. If you’re able to split your memory metrics by memory-used and memory-cached, you should see that the majority of a broker’s memory usage is cached memory. As a broker receives data from producers, the data first enters the

Re: Help - Updating Keystore Dynamically - KAFKA-6810

It’s my understanding that dynamic configuration requires you to write znodes, e.g. /config/brokers/ssl.keystore.location. I believe you can use the same path. Brokers should be watching that path and if a node is added or updated the config values will be read in and loaded over existing

Re: Help - Updating Keystore Dynamically - KAFKA-6810

t? Thanks. > > On Thu, May 16, 2019 at 2:54 PM Peter Bukowinski wrote: > >> It’s my understanding that dynamic configuration requires you to write >> znodes, e.g. /config/brokers/ssl.keystore.location. I believe you can use >> the same path. Brokers should be watching tha

Re: Need some guidance to handle Kafka issues in Cloudera

Lacking any details makes it difficult to assist. Do you have Cloudera support? -- Peter > On May 25, 2019, at 8:27 AM, PRASADA RAO Baratam wrote: > > Need some guidance to handle Kafka issues in Cloudera > > Regards > Prasad

Re: Newb Trying to Publish via Kafka CLI

Hi, Leave the ’ssl://' part off your --broker-list argument and it should work. You only need ‘host:port’. — Peter Bukowinski > On Jun 5, 2019, at 12:41 PM, jbail...@gmail.com wrote: > > Hello, > > I am trying to connect to kafka via CLI to publish messages to a topic from a

Re: Kafka delaying message

ssages that should be processed > immediately and delayed messages at the same time? > > *Pavel Molchanov* > > > > On Wed, May 22, 2019 at 2:41 PM Peter Bukowinski wrote: > >> There is no out-of-the-box way to tell a consumer to not consume an offset >> until it is x

Re: Kafka delaying message

There is no out-of-the-box way to tell a consumer to not consume an offset until it is x minutes old. Your best bet is encode the creation time into the message themselves and add some processing logic into your consumer. Let’s assume your topic has a single partition or your partitions are

Re: kafka ssl config

If you can access the remote file via a mounted filesystem, you can specify '/mountpoint/truststore.jks’ as the value for ssl.truststore.location. You cannot use a url to specify a remote resource. > On May 2, 2019, at 11:38 AM, anurag wrote: > > Hi All, > > Is it possible to set the value

Re: How to count number of available messages per topic?

You’ll need to do this programmatically with some simple math. There’s a binary included with kafka called kafka-run-class that you can use to expose earliest and latest offset information. This will return the earliest unexpired offsets for each partition in a topic: kafka-run-class.sh

Re: Kafka Topic Partition Consumer Lags

Is there a correlation between the lagging partitions and the consumer assigned to them? > On Jun 26, 2019, at 4:25 PM, Garvit Sharma wrote: > > Can anyone please help me with this. > > On Wed, Jun 26, 2019 at 8:56 PM Garvit Sharma wrote: > >> Hey Steve, >> >> I have checked, count of

Re: Can the consumer know the user who sent the message ?

Kafka’s authorization layer is entirely separate from topic data, other than granting or denying access. If you don’t want to alter the messages themselves to hold information about the producers, then you should consider using separate topics. -- Peter (from phone) > On Apr 21, 2019, at

Re: Question for min.insync.replicas

--zookeeper 127.0.0.1:2181 \ --create \ --topic new_topic \ --replication-factor 3 \ --partitions 20 \ --config min.insync.replicas=1 — Peter Bukowinski

Re: Kafka logs are getting deleted too soon

Can you share your broker and topic config here? > On Jul 17, 2019, at 10:09 AM, Sachin Nikumbh > wrote: > > Thanks for the quick response, Tom. > I should have mentioned in my original post that I am always using > --from-beginning with my console consumer. Even then I don't get any data.

Re: Justin Trudeau: support Justin Trudeau to postpone the decision of banning HUAWEI

I’m not even Canadian. No. -- Peter (from phone) > On Jul 17, 2019, at 7:33 AM, jiang0...@gmail.com wrote: > > Hey, > > I just signed the petition "Justin Trudeau: support Justin Trudeau to > postpone the decision of banning HUAWEI" and wanted to see if you could > help by adding your name. >

Re: Kafka logs are getting deleted too soon

Are you running the console consumer with the ‘--from-beginning’ option? It defaults to reading from tail of the log, so if there is nothing being produced it will be idle. -- Peter (from phone) > On Jul 17, 2019, at 8:00 AM, Sachin Nikumbh > wrote: > > Hi all, > I have ~ 96GB of data in

Re: Kafka logs are getting deleted too soon

I saw periodic .deleted files. Does it mean that Kafka > was deleting logs? > Any help would be highly appreciated. > On Wednesday, July 17, 2019, 01:47:44 PM EDT, Peter Bukowinski > wrote: > > Can you share your broker and topic config here? > >> On Jul 17

Re: Kafka logs are getting deleted too soon

g to understand which Kafka settings are playing role here. > I am sure 96GB of data is really not a big deal for Kafka and I am not the > first one to do this. >On Wednesday, July 17, 2019, 04:58:48 PM EDT, Peter Bukowinski > wrote: > > Are you setting a group.id for yo

Re: How to enable RACK awareness on Already Running Kafka Cluster

Hi Ashu, It’s possible to enable rack-awareness in a rolling manner. Kafka will never automatically move existing partitions, unless you tell it to or have a separate tool (e.g. Cruise Control) that does it for you. Rack-awareness comes into play when topics are initially created and

Re: Broker shutdown slowdown between 1.1.0 and 2.3.1

How many partitions are on each of your brokers? That’s a key factor affecting shutdown and startup time. Even if it is large, though, I’ve seen a notable reduction in shutdown and startup times as I’ve moved from kafka 0.11 to 1.x to 2.x. I’m currently doing a rolling restart of a 150-broker

Re: Topics marked for deletion stuck as ineligible for deletion

If any brokers are offline, kafka can’t successfully delete a topic. What’s the state of broker 5? -- Peter (from phone) > On Dec 13, 2019, at 8:55 AM, Vincent Rischmann wrote: > > Hi, > > I've deleted a bunch of topics yesterday on our cluster but some are now > stuck in "marked for

Re: Topics marked for deletion stuck as ineligible for deletion

e topics, I didn't realize that it was > necessary for all brokers to be online. > > Since broker 5 is never coming back again I'm guessing my only choice is to > manually modify the znodes ? > >> On Fri, Dec 13, 2019, at 19:07, Peter Bukowinski wrote: >> If any

Re: Broker that stays outside of the ISR, how to recover

Hi Bart, Before changing anything, I would verify whether or not the affected broker is trying to catch up. Have you looked at the broker’s log? Do you see any errors? Check your metrics or the partition directories themselves to see if data is flowing into the broker. If you do want to reset

Re: More partitions => less throughput?

Testing multiple brokers VMs on a single host won’t give you accurate performance numbers unless that is how you will be deploying kafka in production. (Don’t do this.) All your kafka networking is being handled by a single host, so instead of being spread out between machines to increase total

Re: Moving partition(s) to different broker

If the only replicas for that topic partition exist on brokers 15 and 24 and they are both down, then you cannot recover the partition until either of them is replaced or repaired and rejoins the cluster. You may need to enable unclean leader election, as well. As you’ve discovered, adding

Re: How Kafka leader replica decides to advance Highwater Mark (HW) based on Kafka producer configurations.

The behavior of the high water mark is independent of producer ack settings. It is a property of topic partitions only. (Remember that multiple producers with different configurations can write to the same topic.) The high water mark advances to the latest offset that exists in all topic

Re: How Kafka leader replica decides to advance Highwater Mark (HW) based on Kafka producer configurations.

the producer acks configuration is, Leader replica > waits for *all* other follower replicas to persist the message before > advancing HW(without considering the *current ISR* set)? > > Thanks > >> On Tue, 8 Oct 2019 at 14:54, Peter Bukowinski wrote: >> >> The behavior of

Brokers occasionally dropping out of cluster

r 14 (kafka.controller.KafkaController) Has anyone seen this issue on recent versions of kafka? — Peter Bukowinski

Re: when to expand cluster

Disk usage is one reason to expand. Another reason is if you need more ingest or output throughout for your topic data. If your producers aren’t able to send data to kafka fast enough or your consumers are lagging, you might benefit from more brokers and more partitions. -- Peter > On Feb 26,

Re: when to expand cluster

. -- Peter > On Feb 26, 2020, at 10:23 PM, 张祥 wrote: > > Thanks. What influence does it have for consumers and producers when > partition number is more than broker number, which means at least one > broker serves two partitions for one topic ? performance wise. > > Peter Buko

Re: what happened in case of single disk failure

e disk has been replaced, manually > restart is necessary. > > 张祥 于2020年3月4日周三 下午2:48写道： > >> Thanks Peter, it makes a lot of sense. >> >> Peter Bukowinski 于2020年3月3日周二 上午11:56写道： >> >>> Whether your brokers have a single data directory or multip

Re: what happened in case of single disk failure

Whether your brokers have a single data directory or multiple data directories on separate disks, when a disk fails, the topic partitions located on that disk become unavailable. What happens next depends on how your cluster and topics are configured. If the topics on the affected broker have

Re: when to expand cluster

ers when >> partition number is more than broker number, which means at least one >> broker serves two partitions for one topic ? performance wise. >> >> Peter Bukowinski 于2020年2月26日周三 下午11:02写道： >> >>> Disk usage is one reason to expand. Another reason is if

Re: Confluent Replicator

That is possible as long and you include a topic.rename.format argument in the replication.properties file. The origin and destination cluster configs can point to the same cluster. See the example here

Re: Replicas more than replication-factor

I’ve had this happen a few times when a partition reassignment was underway and one of the brokers that is a destination for the reassignment became unhealthy. This essentially stalls the reassignment indefinitely. The partition with 10 instead of 5 replicas was undergoing a reassignment where

Re: Replicas more than replication-factor

or the recipient), please contact the sender by reply > email and delete all copies of this message. > http://www.cisco.com/c/en/us/about/legal/terms-sale-software-license-agreement/company-registration-information.html > -Original Message- > From: Peter Bukowinski > Sent: Wednesday,

Re: JMX Metrics to display disk Usage

Kafka does not report collect/report on topic data filesystem usage. I used this collectd project to help me collect the topic usage data and export it to graphite: https://github.com/HubSpot/collectd-kafka-disk/blob/master/README.md The plugin collects the size of each topic-partition

Re: log.dirs and SSDs

ropriate number of partitions. — Peter Bukowinski

Re: what happened in case of single disk failure

ons on that broker instead of only one malfunctioned disk. Am I > right? Thanks. > > 张祥 于2020年3月5日周四 上午9:25写道： > >> Thanks Peter, really appreciate it. >> >> Peter Bukowinski 于2020年3月4日周三 下午11:50写道： >> >>> Yes, you should restart the broker. I don’t beli

Re: log.dirs and SSDs

hput of GB/s? > > This is in fact the case for Apache Pulsar, and the openmessaging benchmark > uses 4 directories on the same SSD to increase throughput. > > ________ > 差出人: Peter Bukowinski > 送信日時: 2020年3月12日 8:51 > 宛先: users@kafka.apache.org > 件名: Re:

Re: log.dirs and SSDs

ions here!  > So is that the way to tune the broker if it does not achieve disk throughput? > > ____ > 差出人: Peter Bukowinski > 送信日時: 2020年3月12日 9:38 > > Couldn’t the same be accomplished by increasing the num.io.threads broker > setting? > &

Re: Kafka topic partition directory

Kafka doesn’t monitor the contents of the log data directories unless it created the file or directory. If it didn’t create the directory/file it will ignore it. -- Peter > On Mar 28, 2020, at 4:17 PM, anila devi > wrote: > > Hi Users, > If I create a directory or a file in the same

Re: Partitioning issue when a broker is going down

If your producer is set to use acks=all, then it won’t be able to produce to the topic topic partitions that had replicas on the missing broker until the replacement broker has finished catching up to be included in the ISR. What method are you using that reports on the number of topic

Re: Partitioning issue when a broker is going down

the number of current > partitions... The logic is similar as that is how kafka is designed. The client will fetch the topic’s metadata (including partitions available for writing) on connect, on error, and by the interval determined by topic.metadata.refresh.interval.ms, unless it is

Re: Adding additional nodes to Existing ZK cluster

keeper is used and you want to make sure high > availability of zookeeper? > > What if tomorrow I add 2 more instances of zookeeper, total 5. Is it > required to update 2 new zK instances to my kafka brokers? > > > Regards, > Sunil. > >> On Sat, 7 Mar 2020 at 11:08

Intended behavior when a broker loses its log volume

location per broker. — Peter Bukowinski

Re: Kafka BrokerState Metric Value 3

The broker state metric just reports on the state of the broker itself, not whether it is in sync. A replacement broker will quickly reach a broker state of 3 on startup even though it has to catch up on many replicas. Don’t rely on it for checking if a cluster/broker is healthy with no

Re: Can we use VIP ip rather than Kafka Broker host name in bootstrap string

I do something like this in my environment to simplify things. We use a consul service address, e.g ‘kafka.service.subdomain.consul', to provide the VIP, which returns the address of a live broker in the cluster. Kafka clients use that address in their configs. It works very well. — Peter >

Re: Disk space - sharp increase in usage

> On Jun 2, 2020, at 12:56 AM, Victoria Zuberman > wrote: > > Hi, > > Background: > Kafka cluster > 7 brokers, with 4T disk each > version 2.3 (recently upgraded from 0.1.0 via 1.0.1) > > Problem: > Used disk space went from 40% to 80%. > Looking for root cause. > > Suspects: > > 1.

Re: Kafka compatibility with ZK

That procedure looks safe and sane to me, Marina. > On Aug 2, 2020, at 10:04 AM, Marina Popova > wrote: > >  > Actually, I'm very interested in your experience as well I'm about to > start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK > 3.5.6 > > I have Kafka

Re: Kafka partitions replication issue

> On Jun 17, 2020, at 5:16 AM, Karnam, Sudheer wrote: > > Team, > We are using kafka version 2.3.0 and we are facing issue with brokers > replication > > 1.Kafka has 6 brokers. > 2.Mainly 7 topics exist in kafka cluster and each topic has

Re: How to Change number of partitions without Rolling restart?

You can’t use a wildcard and must address each topic individually. You can automate it with a for loop that takes an array/list of topics as the item to iterate over. -- Peter Bukowinski > On Jun 21, 2020, at 9:16 PM, sunil chaudhari > wrote: > > Manoj, > You mean

Re: Kafka compatibility with ZK

Zookeeper is not part of the kafka project and must be installed separately. Confluent maintain a version compatibility table you can use as a reference: https://docs.confluent.io/current/installation/versions-interoperability.html#zk

Re: Kafka compatibility with ZK

Agreed. We use a cloudera distribution of zookeeper, that is versioned at 3.4.5 (plus a bunch of backported patches) with kafka 2.4 and haven’t had any issues. > On Jul 23, 2020, at 1:19 PM, Andrey Klochkov wrote: > > We are running a separate ZK cluster and its version is not really tied to

Re: Is it safe to delete old log segments manually?

, they started catching up on the deleted topic-partition directories. -- Peter Bukowinski > On Mar 25, 2021, at 8:00 AM, Sankalp Bhatia wrote: > > Hi All, > > Brokers in one of our Apache Kafka clusters are continuously crashing as > they have run out of disk space. As pe

Re: Is it safe to delete old log segments manually?

In this case, yes, for any given topic-partition on the broker, you should be able to delete the oldest log segment, its associated index and timeindex files, and the snapshot file (which will be recreated on startup) in order to gain some free space. — Peter Bukowinski > On Mar 25, 2

Re: kafka log.retention.bytes

log.retention.bytes is a broker-level config that sets the maximum size of a topic partition on a broker, so it will apply to all topics… unless a topic has the retention.bytes property configured — this is a topic-level config and only applies to a single topic — in which case that takes

Re: Ensuring that the message is persisted after acknowledgement

Kunal, I recommend looking at the broker and topic parameters that include the term “flush” , such as https://kafka.apache.org/documentation/#topicconfigs_flush.messages Kafka lets you configure how often log messages are

Re: Under-replicated-partitions

Hi Sridhar, If your min.insync.replicas value is set to 3, then kafka won’t be able to move replicas until there are three replicas listed in the ISR. I would look into the health of broker 21 — it’s either down or unhealthy. It’s the only one not showing in the ISR list. — Peter Bukowinski

Re: Help with log.dirs please

advice would be great please as i've exhausted all my options. > > Many Thanks, > > Chris > > On 2022-09-20 21:43, Chris Peart wrote: > >> Thanks Peter, >> I'll give this a go tomorrow and let you know how I get on. >> Many Thanks, >> Chris >>

Re: Help with log.dirs please

; > > > Thanks again for your help on this and sorry for not spotting my mistake > sooner :-) > > > > Many Thanks > > Chris > > > > On 2022-09-21 22:24, Peter Bukowinski wrote: > >> Hmmm. Let’s start with some low level troubleshooting. >&g

Re: Help with log.dirs please

ny Thanks, > Chris > >> On 16 Sep 2022, at 1:56 am, Peter Bukowinski wrote: >> >> Hi Chris, >> >> Can you share the partition count and replication factor of your partitions? >> Also, do all four brokers contain topic-partition directories in &g

Re: Help with log.dirs please

Hi Chris, Can you share the partition count and replication factor of your partitions? Also, do all four brokers contain topic-partition directories in /data/1/kafka/data or just a single broker? Depending on your topic config, it may be entirely normal that his has happened. — Peter

Re: Help with log.dirs please

> Chris >> >> >> >> >> >>> On 2022-09-16 08:22, Chris Peart wrote: >>> >>> Thanks Peter, >>> I’ll check the logs next week and let you know my findings. >>> Many Thanks >>> Chris >>> >&

Re: Help with log.dirs please

>>> >>> I see no errors in server.log & controller.log. >>> >>> Any advice would be great please as i've exhausted all my options. >>> >>> Many Thanks, >>> >>> Chris >>> >>>> On 2022-09-20 21:43, C

Re: Consumer Lag-Apache_kafka_JMX metrics

Richard recently answered your query. A kafka cluster does not keep track of lag on behalf of external consumers and it therefore is not available in JMX. This is why tools like Burrow were written. The java kafka consumer published consumer lag metrics, and perhaps some other third-party

Re: Kafka cluster rolling restart

before all partitions have returned to healthy, and then you’ll have offline partitions because your minISR is 2. -- Peter Bukowinski > On Mar 6, 2023, at 7:04 AM, Luis Alves wrote: > > Hello, > > I'm doing some tests with rolling restarts in a Kafka cluster and I

Re: Some questions on Kafka on order of messages with mutiple partitions

t;topic":"md","partition":4,"replicas":[7,9,11]}, {"topic":"md","partition":5,"replicas":[3,11,1]} ] } You may want to adjust where the replicas sit to achieve a better balance across the cluster, but this arrangement only

Re: Some questions on Kafka on order of messages with mutiple partitions

Wed, 17 May 2023 at 00:21, Miguel A. Sotomayor > wrote: > >> Hi Mich, >> >> You can use the script `kafka-reassign-partitions.sh` to re-locate or >> change the number of replicas >> >> Regards >> Miguel >> >> El mar, 16 may 2023 a las 18:4