[jira] [Commented] (KAFKA-13331) Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events

[ https://issues.apache.org/jira/browse/KAFKA-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423761#comment-17423761 ] GEORGE LI commented on KAFKA-13331: --- besides slowness of the reassignments, when the EventQueue is

[jira] [Created] (KAFKA-13331) Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events

GEORGE LI created KAFKA-13331: - Summary: Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events Key: KAFKA-13331 URL:

[jira] [Commented] (KAFKA-12971) Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id

[ https://issues.apache.org/jira/browse/KAFKA-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367008#comment-17367008 ] GEORGE LI commented on KAFKA-12971: --- [~ijuma] The Flink 1.9 can use 2.x kafka-clients, while Flink

[jira] [Commented] (KAFKA-12971) Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id

[ https://issues.apache.org/jira/browse/KAFKA-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17366011#comment-17366011 ] GEORGE LI commented on KAFKA-12971: --- This issue is fixed in the {{1.1.x}} kafka client by back porting

[jira] [Created] (KAFKA-12971) Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id

GEORGE LI created KAFKA-12971: - Summary: Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id Key: KAFKA-12971 URL: https://issues.apache.org/jira/browse/KAFKA-12971

[jira] [Commented] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220243#comment-17220243 ] GEORGE LI commented on KAFKA-8733: -- for lossy clusters, setting unclean.leader.election.enable=true

[jira] [Commented] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17111858#comment-17111858 ] GEORGE LI commented on KAFKA-8638: -- [~hai_lin] some of the recent activities about KIP-491 is in

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103028#comment-17103028 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] hmm...That's weird. auto.leader.rebalance.enable

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086772#comment-17086772 ] GEORGE LI edited comment on KAFKA-4084 at 4/19/20, 7:28 AM: [~blodsbror] I

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086772#comment-17086772 ] GEORGE LI edited comment on KAFKA-4084 at 4/19/20, 7:20 AM: [~blodsbror] I

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086772#comment-17086772 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] I am not very familiar with 5.4 setup. Do you have

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17077958#comment-17077958 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Probably doing PLE with too many partitions at once is

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076124#comment-17076124 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] If this turns out to be positive in the testing. I

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076089#comment-17076089 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] I think it's still ok to set

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076006#comment-17076006 ] GEORGE LI edited comment on KAFKA-4084 at 4/6/20, 7:13 AM: --- [~blodsbror] Have

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076006#comment-17076006 ] GEORGE LI edited comment on KAFKA-4084 at 4/6/20, 2:26 AM: --- [~blodsbror] Have

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076006#comment-17076006 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Have some free time this weekend to troubleshoot and

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072259#comment-17072259 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Very busy with work and this COVID-19 Jira

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051509#comment-17051509 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] sure. I am working on porting the patch from 1.1.x.

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048116#comment-17048116 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] [~sriharsha] Hi Evan, I am so sorry I had been busy

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032598#comment-17032598 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Which kafka version you are running? Maybe I can

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032267#comment-17032267 ] GEORGE LI edited comment on KAFKA-4084 at 2/7/20 10:02 AM: --- [~blodsbror]

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032267#comment-17032267 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] [~junrao][~sriharsha] With

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030183#comment-17030183 ] GEORGE LI commented on KAFKA-4084: -- [~evanjpw] One solution we have in-house is a tooling to do

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029299#comment-17029299 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] We have implemented KIP-491 internally. Using a

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029299#comment-17029299 ] GEORGE LI edited comment on KAFKA-4084 at 2/3/20 9:26 PM: -- [~blodsbror] We have

[jira] [Updated] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

[ https://issues.apache.org/jira/browse/KAFKA-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8903: - Description: It very common (and sometimes frequent) that a broker has hardware failures (disk, memory,

[jira] [Updated] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

[ https://issues.apache.org/jira/browse/KAFKA-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8903: - Description: It very common (and sometimes frequent) that a broker has hardware failures (disk, memory,

[jira] [Commented] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

[ https://issues.apache.org/jira/browse/KAFKA-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928906#comment-16928906 ] GEORGE LI commented on KAFKA-8903: -- e.g. {code:java} /usr/lib/kafka/bin/kafka-configs.sh

[jira] [Created] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

GEORGE LI created KAFKA-8903: Summary: allow the new replica (offset 0) to catch up with current leader using latest offset Key: KAFKA-8903 URL: https://issues.apache.org/jira/browse/KAFKA-8903 Project:

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900461#comment-16900461 ] GEORGE LI commented on KAFKA-4084: -- [~jiaxinye] does the patch from

[jira] [Commented] (KAFKA-8725) Improve LogCleaner error handling when failing to grab the filthiest log

[ https://issues.apache.org/jira/browse/KAFKA-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899237#comment-16899237 ] GEORGE LI commented on KAFKA-8725: -- We saw similar issues. After the thread die, the "good" partitions

[jira] [Resolved] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[ https://issues.apache.org/jira/browse/KAFKA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI resolved KAFKA-8663. -- Resolution: Won't Fix looks like RAR + OAR is required for KIP-455 to preserve the targetReplicas

[jira] [Commented] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884787#comment-16884787 ] GEORGE LI commented on KAFKA-8638: -- Here is the KIP:

[jira] [Commented] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[ https://issues.apache.org/jira/browse/KAFKA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884526#comment-16884526 ] GEORGE LI commented on KAFKA-8663: -- As we can see from the original comments of the code: {code}

[jira] [Updated] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[ https://issues.apache.org/jira/browse/KAFKA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8663: - Description: >From my observation/experience during reassignment, the partition assignment >replica

[jira] [Created] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

GEORGE LI created KAFKA-8663: Summary: partition assignment would be better original_assignment + new_reassignment during reassignments Key: KAFKA-8663 URL: https://issues.apache.org/jira/browse/KAFKA-8663

[jira] [Commented] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881480#comment-16881480 ] GEORGE LI commented on KAFKA-8638: -- Hi Jose, Because a broker can have hundreds/thousands

[jira] [Updated] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8638: - Description: Currently, the kafka preferred leader election will pick the broker_id in the

[jira] [Created] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

GEORGE LI created KAFKA-8638: Summary: Preferred Leader Blacklist (deprioritized list) Key: KAFKA-8638 URL: https://issues.apache.org/jira/browse/KAFKA-8638 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-8527) add dynamic maintenance broker config

[ https://issues.apache.org/jira/browse/KAFKA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868914#comment-16868914 ] GEORGE LI commented on KAFKA-8527: -- We had this implemented in a separate admin service outside of

[jira] [Commented] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777257#comment-16777257 ] GEORGE LI commented on KAFKA-6794: -- Hi [~viktorsomogyi], I think this "Incremental Reassignment" is

[jira] [Comment Edited] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777149#comment-16777149 ] GEORGE LI edited comment on KAFKA-6794 at 2/25/19 7:56 PM: --- Hi,

[jira] [Commented] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777149#comment-16777149 ] GEORGE LI commented on KAFKA-6794: -- Hi, [~viktorsomogyi], When I rebalance the whole cluster, I

[jira] [Comment Edited] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775598#comment-16775598 ] GEORGE LI edited comment on KAFKA-6794 at 2/22/19 9:21 PM: --- I also have seen

[jira] [Commented] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775598#comment-16775598 ] GEORGE LI commented on KAFKA-6794: -- I also have seen this issue.  When more than one broker is in the

[jira] [Commented] (KAFKA-7854) Behavior change in controller picking up partition reassignment tasks since 1.1.0

[ https://issues.apache.org/jira/browse/KAFKA-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773395#comment-16773395 ] GEORGE LI commented on KAFKA-7854: --

[jira] [Commented] (KAFKA-6304) The controller should allow updating the partition reassignment for the partitions being reassigned

[ https://issues.apache.org/jira/browse/KAFKA-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773394#comment-16773394 ] GEORGE LI commented on KAFKA-6304: --

[jira] [Commented] (KAFKA-6359) Work for KIP-236

[ https://issues.apache.org/jira/browse/KAFKA-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773351#comment-16773351 ] GEORGE LI commented on KAFKA-6359: -- New Pull Request:   https://github.com/apache/kafka/pull/6296 >