[jira] [Updated] (KAFKA-7152) replica should be in-sync if its LEO equals leader's LEO

2018-07-21 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7152: Fix Version/s: 2.1.0 > replica should be in-sync if its LEO equals leader's LEO >

[jira] [Updated] (KAFKA-7147) Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

2018-07-21 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7147: Fix Version/s: 2.1.0 > Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client >

[jira] [Assigned] (KAFKA-7177) Update 2.0 documentation to reflect changed quota behaviors by KIP-219

2018-07-20 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reassigned KAFKA-7177: --- Assignee: Jon Lee > Update 2.0 documentation to reflect changed quota behaviors by KIP-219 >

[jira] [Commented] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-20 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551239#comment-16551239 ] Dong Lin commented on KAFKA-7126: - The simply patch in the attachment mostly fixes the issue and there is

[jira] [Resolved] (KAFKA-7098) Improve accuracy of the log cleaner throttle rate

2018-07-20 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-7098. - Resolution: Fixed > Improve accuracy of the log cleaner throttle rate >

[jira] [Commented] (KAFKA-6488) Prevent log corruption in case of OOM

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542271#comment-16542271 ] Dong Lin commented on KAFKA-6488: - One reasonable solution is to add the JVM flag ExitOnOutOfMemoryError

[jira] [Resolved] (KAFKA-6488) Prevent log corruption in case of OOM

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6488. - Resolution: Won't Do > Prevent log corruption in case of OOM > - > >

[jira] [Resolved] (KAFKA-3089) VerifiableProducer should do a clean shutdown in stop_node()

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-3089. - Resolution: Fixed > VerifiableProducer should do a clean shutdown in stop_node() >

[jira] [Resolved] (KAFKA-4820) ConsumerNetworkClient.send() should not require global lock

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-4820. - Resolution: Fixed > ConsumerNetworkClient.send() should not require global lock >

[jira] [Resolved] (KAFKA-3145) CPU Usage Spike to 100% when network connection is to error port

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-3145. - Resolution: Cannot Reproduce > CPU Usage Spike to 100% when network connection is to error port >

[jira] [Assigned] (KAFKA-7098) Improve accuracy of the log cleaner throttle rate

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reassigned KAFKA-7098: --- Assignee: Zhanxiang (Patrick) Huang (was: Dong Lin) > Improve accuracy of the log cleaner throttle

[jira] [Resolved] (KAFKA-6617) Improve controller performance by batching reassignment znode write operation

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6617. - Resolution: Fixed > Improve controller performance by batching reassignment znode write operation >

[jira] [Updated] (KAFKA-6082) consider fencing zookeeper updates with controller epoch zkVersion

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6082: Description:   Kafka controller may fail to function properly (even after repeated controller movement)

[jira] [Resolved] (KAFKA-5812) Represent logDir with case class where absolute path is required

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-5812. - Resolution: Won't Do > Represent logDir with case class where absolute path is required >

[jira] [Updated] (KAFKA-6082) consider fencing zookeeper updates with controller epoch zkVersion

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6082: Description:   Kafka controller may fail to function properly (even after repeated controller movement)

[jira] [Resolved] (KAFKA-6618) Prevent two controllers from updating znodes concurrently

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6618. - Resolution: Duplicate Duplicate of https://issues.apache.org/jira/browse/KAFKA-6082 > Prevent two

[jira] [Resolved] (KAFKA-6636) ReplicaFetcherThread should not die if hw < 0

2018-07-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6636. - Resolution: Fixed This is no longer an issue after KAFKA-3978; Ensure high watermark is always

[jira] [Comment Edited] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540687#comment-16540687 ] Dong Lin edited comment on KAFKA-7126 at 7/11/18 9:31 PM: -- Suppose there is no

[jira] [Commented] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540687#comment-16540687 ] Dong Lin commented on KAFKA-7126: - Suppose there is no error such as network disconnection or timeout,

[jira] [Updated] (KAFKA-7147) Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

2018-07-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7147: Summary: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

[jira] [Created] (KAFKA-7152) replica should be in-sync if its LEO equals leader's LEO

2018-07-11 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-7152: --- Summary: replica should be in-sync if its LEO equals leader's LEO Key: KAFKA-7152 URL: https://issues.apache.org/jira/browse/KAFKA-7152 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-7147) ReassignPartitionsCommand should be able to connect to broker over SSL

2018-07-10 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7147: Description: Currently both ReassignPartitionsCommand and LogDirsCommand instantiates AdminClient using

[jira] [Created] (KAFKA-7147) ReassignPartitionsCommand should be able to connect to broker over SSL

2018-07-10 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-7147: --- Summary: ReassignPartitionsCommand should be able to connect to broker over SSL Key: KAFKA-7147 URL: https://issues.apache.org/jira/browse/KAFKA-7147 Project: Kafka

[jira] [Commented] (KAFKA-7136) PushHttpMetricsReporter may deadlock when processing metrics changes

2018-07-08 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16536430#comment-16536430 ] Dong Lin commented on KAFKA-7136: - [~ijuma] Excellent. I didn't realize we have periodic system tests for

[jira] [Commented] (KAFKA-7136) PushHttpMetricsReporter may deadlock when processing metrics changes

2018-07-07 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535907#comment-16535907 ] Dong Lin commented on KAFKA-7136: - [~guozhang] [~ijuma] [~rsivaram] Not sure if it is required to re-run

[jira] [Commented] (KAFKA-7136) PushHttpMetricsReporter may deadlock when processing metrics changes

2018-07-07 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535674#comment-16535674 ] Dong Lin commented on KAFKA-7136: - [~guozhang] Thank you! Could you please help start system tests for

[jira] [Commented] (KAFKA-7136) PushHttpMetricsReporter may deadlock when processing metrics changes

2018-07-06 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535542#comment-16535542 ] Dong Lin commented on KAFKA-7136: - [~rsivaram] I see. Yeah we should include this fix in 1.1.1 release

[jira] [Commented] (KAFKA-7136) PushHttpMetricsReporter may deadlock when processing metrics changes

2018-07-05 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534191#comment-16534191 ] Dong Lin commented on KAFKA-7136: - [~rsivaram] Yes, this probably does not need to be blocker since it is

[jira] [Commented] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-01 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529384#comment-16529384 ] Dong Lin commented on KAFKA-7126: - [~yuzhih...@gmail.com] Thanks much for your information. I was looking

[jira] [Updated] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-01 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7126: Description: For a group of 200 MirrorMaker consumers with patten-based topic subscription, a single

[jira] [Created] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-01 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-7126: --- Summary: Reduce number of rebalance period for large consumer groups after a topic is created Key: KAFKA-7126 URL: https://issues.apache.org/jira/browse/KAFKA-7126 Project:

[jira] [Updated] (KAFKA-7126) Reduce number of rebalance period for large consumer groups after a topic is created

2018-07-01 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7126: Description: For a group of 200 MirrorMaker consumers with patten-based topic subscription, a single

[jira] [Commented] (KAFKA-7012) Performance issue upgrading to kafka 1.0.1 or 1.1

2018-06-26 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523303#comment-16523303 ] Dong Lin commented on KAFKA-7012: - Great to know that the fix works!   > Performance issue upgrading to

[jira] [Commented] (KAFKA-6949) alterReplicaLogDirs() should grab partition lock when accessing log of the future replica

2018-06-26 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523297#comment-16523297 ] Dong Lin commented on KAFKA-6949: - [~rsivaram] The patch has been merged into trunk. I opened PR

[jira] [Updated] (KAFKA-6949) alterReplicaLogDirs() should grab partition lock when accessing log of the future replica

2018-06-26 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6949: Fix Version/s: 2.0.0 > alterReplicaLogDirs() should grab partition lock when accessing log of the >

[jira] [Created] (KAFKA-7098) Improve accuracy of the log cleaner throttle rate

2018-06-25 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-7098: --- Summary: Improve accuracy of the log cleaner throttle rate Key: KAFKA-7098 URL: https://issues.apache.org/jira/browse/KAFKA-7098 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-7012) Performance issue upgrading to kafka 1.0.1 or 1.1

2018-06-20 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518751#comment-16518751 ] Dong Lin commented on KAFKA-7012: - [~mjsax] It is not included in RC0 for 1.1.1. I just updated the "fix

[jira] [Updated] (KAFKA-7012) Performance issue upgrading to kafka 1.0.1 or 1.1

2018-06-20 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7012: Fix Version/s: (was: 1.1.1) 1.1.2 > Performance issue upgrading to kafka 1.0.1 or

[jira] [Updated] (KAFKA-5975) No response when deleting topics and delete.topic.enable=false

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-5975: Fix Version/s: (was: 1.1.1) 1.1.2 > No response when deleting topics and

[jira] [Commented] (KAFKA-5975) No response when deleting topics and delete.topic.enable=false

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517623#comment-16517623 ] Dong Lin commented on KAFKA-5975: - Moving from 1.1.1 to 1.1.2 since it is marked as Priority Minor and

[jira] [Comment Edited] (KAFKA-5975) No response when deleting topics and delete.topic.enable=false

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517623#comment-16517623 ] Dong Lin edited comment on KAFKA-5975 at 6/19/18 11:23 PM: --- Moving from 1.1.1

[jira] [Updated] (KAFKA-6836) Upgrade metrics library

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6836: Fix Version/s: (was: 1.1.1) 1.1.2 > Upgrade metrics library >

[jira] [Updated] (KAFKA-6809) connections-created metric does not behave as expected

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6809: Fix Version/s: (was: 1.1.1) 1.1.2 > connections-created metric does not behave as

[jira] [Commented] (KAFKA-6809) connections-created metric does not behave as expected

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517621#comment-16517621 ] Dong Lin commented on KAFKA-6809: - Moving from 1.1.1 to 1.1.2 as there is no open PR for 1.1.1.   >

[jira] [Commented] (KAFKA-6836) Upgrade metrics library

2018-06-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517619#comment-16517619 ] Dong Lin commented on KAFKA-6836: - Moving from 1.1.1 to 1.1.2 as there is no open PR for 1.1.1. >

[jira] [Updated] (KAFKA-7019) Reduction the contention between metadata update and metadata read operation

2018-06-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7019: Description: Currently MetadataCache.updateCache() grabs a write lock in order to process the

[jira] [Updated] (KAFKA-7051) Improve the efficiency of the ReplicaManager when there are many partitions

2018-06-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7051: Fix Version/s: 2.0.0 > Improve the efficiency of the ReplicaManager when there are many partitions >

[jira] [Commented] (KAFKA-6806) Unable to validate sink connectors without "topics" component which is not required

2018-06-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509256#comment-16509256 ] Dong Lin commented on KAFKA-6806: - Hey [~rhauch], this issue has been marked as blocking issue for 1.1.1

[jira] [Commented] (KAFKA-7040) The replica fetcher thread may truncate accepted messages during multiple fast leadership transitions

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508847#comment-16508847 ] Dong Lin commented on KAFKA-7040: - [~luwang] Nice catch. It is indeed problematic (with some probability

[jira] [Comment Edited] (KAFKA-7040) The replica fetcher thread may truncate accepted messages during multiple fast leadership transitions

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508763#comment-16508763 ] Dong Lin edited comment on KAFKA-7040 at 6/11/18 9:15 PM: -- [~luwang] Say message

[jira] [Commented] (KAFKA-7040) The replica fetcher thread may truncate accepted messages during multiple fast leadership transitions

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508763#comment-16508763 ] Dong Lin commented on KAFKA-7040: - [~luwang] Say message m0 is truncated in step 5. Is that message acked

[jira] [Comment Edited] (KAFKA-7040) The replica fetcher thread may truncate accepted messages during multiple fast leadership transitions

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508763#comment-16508763 ] Dong Lin edited comment on KAFKA-7040 at 6/11/18 9:14 PM: -- [~luwang] Say message

[jira] [Comment Edited] (KAFKA-7040) The replica fetcher thread may truncate accepted messages during multiple fast leadership transitions

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508763#comment-16508763 ] Dong Lin edited comment on KAFKA-7040 at 6/11/18 9:14 PM: -- [~luwang] Say message

[jira] [Updated] (KAFKA-6697) JBOD configured broker should not die if log directory is invalid

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6697: Issue Type: Bug (was: Improvement) > JBOD configured broker should not die if log directory is invalid >

[jira] [Resolved] (KAFKA-6946) Keep the session id for incremental fetch when fetch responses are throttled

2018-06-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6946. - Resolution: Fixed > Keep the session id for incremental fetch when fetch responses are throttled >

[jira] [Comment Edited] (KAFKA-4381) Add per partition lag metric to KafkaConsumer.

2018-06-08 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506710#comment-16506710 ] Dong Lin edited comment on KAFKA-4381 at 6/8/18 11:51 PM: -- The metric added in

[jira] [Commented] (KAFKA-4381) Add per partition lag metric to KafkaConsumer.

2018-06-08 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506710#comment-16506710 ] Dong Lin commented on KAFKA-4381: - The metric added in this patch includes topic partition in the metric

[jira] [Updated] (KAFKA-7019) Reduction the contention between metadata update and metadata read operation

2018-06-07 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7019: Component/s: core > Reduction the contention between metadata update and metadata read operation >

[jira] [Updated] (KAFKA-7019) Reduction the contention between metadata update and metadata read operation

2018-06-07 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-7019: Fix Version/s: 2.1.0 > Reduction the contention between metadata update and metadata read operation >

[jira] [Created] (KAFKA-7019) Reduction the contention between metadata update and metadata read operation

2018-06-07 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-7019: --- Summary: Reduction the contention between metadata update and metadata read operation Key: KAFKA-7019 URL: https://issues.apache.org/jira/browse/KAFKA-7019 Project: Kafka

[jira] [Updated] (KAFKA-6949) alterReplicaLogDirs() should grab partition lock when accessing log of the future replica

2018-05-28 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6949: Fix Version/s: 2.0.0 > alterReplicaLogDirs() should grab partition lock when accessing log of the >

[jira] [Updated] (KAFKA-6949) alterReplicaLogDirs() should grab partition lock when accessing log of the future replica

2018-05-25 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6949: Summary: alterReplicaLogDirs() should grab partition lock when accessing log of the future replica (was:

[jira] [Commented] (KAFKA-6949) NoSuchElementException in ReplicaManager

2018-05-25 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491053#comment-16491053 ] Dong Lin commented on KAFKA-6949: - [~hachikuji] Certainly. I will look into this. >

[jira] [Assigned] (KAFKA-6949) NoSuchElementException in ReplicaManager

2018-05-25 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reassigned KAFKA-6949: --- Assignee: Dong Lin > NoSuchElementException in ReplicaManager >

[jira] [Commented] (KAFKA-3473) KIP-237: More Controller Health Metrics

2018-05-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481810#comment-16481810 ] Dong Lin commented on KAFKA-3473: - Sure. Opened [https://github.com/apache/kafka/pull/5043] to add docs.

[jira] [Commented] (KAFKA-3473) KIP-237: More Controller Health Metrics

2018-05-19 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481742#comment-16481742 ] Dong Lin commented on KAFKA-3473: - [~wushujames] Could you tell me which docs you are referring to? >

[jira] [Comment Edited] (KAFKA-6188) Broker fails with FATAL Shutdown - log dirs have failed

2018-05-18 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480983#comment-16480983 ] Dong Lin edited comment on KAFKA-6188 at 5/18/18 5:59 PM: -- Kafka currently

[jira] [Commented] (KAFKA-6188) Broker fails with FATAL Shutdown - log dirs have failed

2018-05-18 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480983#comment-16480983 ] Dong Lin commented on KAFKA-6188: - Kafka currently handles disk failure on a per-disk basis, i.e. if there

[jira] [Commented] (KAFKA-6705) producer.send() should not block due to metadata not available

2018-04-24 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450350#comment-16450350 ] Dong Lin commented on KAFKA-6705: - I am going to close this ticket. Thinking about this more, the benefit

[jira] [Resolved] (KAFKA-6705) producer.send() should not block due to metadata not available

2018-04-24 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6705. - Resolution: Won't Fix > producer.send() should not block due to metadata not available >

[jira] [Commented] (KAFKA-6795) Add unit test for ReplicaAlterLogDirsThread

2018-04-24 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449370#comment-16449370 ] Dong Lin commented on KAFKA-6795: - Thanks [~apovzner]! Nice catch. You are right that the right behavior 

[jira] [Commented] (KAFKA-6795) Add unit test for ReplicaAlterLogDirsThread

2018-04-16 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439991#comment-16439991 ] Dong Lin commented on KAFKA-6795: - Thanks for adding more tests [~apovzner]. Currently 

[jira] [Commented] (KAFKA-6705) producer.send() should not block due to metadata not available

2018-04-11 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434896#comment-16434896 ] Dong Lin commented on KAFKA-6705: - [~omkreddy] [~ijuma] Thanks for the reference to the previous

[jira] [Updated] (KAFKA-6705) producer.send() should not block due to metadata not available

2018-03-23 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6705: Summary: producer.send() should not block due to metadata not available (was: producer.send() should be

[jira] [Commented] (KAFKA-6705) producer.send() should be non-blocking

2018-03-23 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410876#comment-16410876 ] Dong Lin commented on KAFKA-6705: - [~ijuma] Thanks for the information. I will search for the previous

[jira] [Updated] (KAFKA-6705) producer.send() should be non-blocking

2018-03-23 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6705: Description: Currently producer.send() may block on metadata for up to max.block.ms. This behavior is well

[jira] [Commented] (KAFKA-6705) producer.send() should be non-blocking

2018-03-22 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410672#comment-16410672 ] Dong Lin commented on KAFKA-6705: - Ping [~ijuma] and [~jasong35] for comments. > producer.send() should

[jira] [Created] (KAFKA-6705) producer.send() should be non-blocking

2018-03-22 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-6705: --- Summary: producer.send() should be non-blocking Key: KAFKA-6705 URL: https://issues.apache.org/jira/browse/KAFKA-6705 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-6697) JBOD configured broker should not die if log directory is invalid

2018-03-20 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-6697: --- Summary: JBOD configured broker should not die if log directory is invalid Key: KAFKA-6697 URL: https://issues.apache.org/jira/browse/KAFKA-6697 Project: Kafka Issue

[jira] [Updated] (KAFKA-6617) Improve controller performance by batching reassignment znode write operation

2018-03-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6617: Summary: Improve controller performance by batching reassignment znode write operation (was: KAFKA-6617;

[jira] [Updated] (KAFKA-6617) KAFKA-6617; Improve controller performance by batching reassignment znode write operation

2018-03-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6617: Summary: KAFKA-6617; Improve controller performance by batching reassignment znode write operation (was:

[jira] [Updated] (KAFKA-6638) Leader should remove unassigned replica from ISR

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6638: Description: During partition reassignment, controller may remove a replica from the replica set while

[jira] [Updated] (KAFKA-6638) Leader should remove unassigned replica from ISR

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6638: Summary: Leader should remove unassigned replica from ISR (was: Controller should remove replica from ISR

[jira] [Updated] (KAFKA-6640) Improve efficiency of KafkaAdminClient.describeTopics()

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6640: Description: Currently in KafkaAdminClient.describeTopics(), for each topic in the request, an complete map

[jira] [Updated] (KAFKA-6640) Improve efficiency of KafkaAdminClient.describeTopics()

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6640: Description: Currently in KafkaAdminClient.describeTopics(), for each topic in the request, an complete map

[jira] [Updated] (KAFKA-6640) Improve efficiency of KafkaAdminClient.describeTopics()

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6640: Description: Currently in KafkaAdminClient.describeTopics(), for each topic in the request, an complete map

[jira] [Updated] (KAFKA-6640) Improve efficiency of KafkaAdminClient.describeTopics()

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6640: Description: Currently in KafkaAdminClient.describeTopics(), for each topic in the request, an complete map

[jira] [Created] (KAFKA-6640) Improve efficiency of KafkaAdminClient.describeTopics()

2018-03-12 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-6640: --- Summary: Improve efficiency of KafkaAdminClient.describeTopics() Key: KAFKA-6640 URL: https://issues.apache.org/jira/browse/KAFKA-6640 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-6638) Controller should remove replica from ISR if the replica is removed from the replica set

2018-03-12 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-6638: --- Summary: Controller should remove replica from ISR if the replica is removed from the replica set Key: KAFKA-6638 URL: https://issues.apache.org/jira/browse/KAFKA-6638

[jira] [Updated] (KAFKA-6636) ReplicaFetcherThread should not die if hw < 0

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6636: Description: ReplicaFetcherThread can die in the following scenario:   1) Partition P1 has replica set

[jira] [Created] (KAFKA-6636) ReplicaFetcherThread should not die if hw < 0

2018-03-12 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-6636: --- Summary: ReplicaFetcherThread should not die if hw < 0 Key: KAFKA-6636 URL: https://issues.apache.org/jira/browse/KAFKA-6636 Project: Kafka Issue Type: Improvement

[jira] [Assigned] (KAFKA-6624) log segment deletion could cause a disk to be marked offline incorrectly

2018-03-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reassigned KAFKA-6624: --- Assignee: Dong Lin > log segment deletion could cause a disk to be marked offline incorrectly >

[jira] [Commented] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

2018-03-10 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394107#comment-16394107 ] Dong Lin commented on KAFKA-3978: - Just realized that the problem I have been investigating is different

[jira] [Comment Edited] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

2018-03-10 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394103#comment-16394103 ] Dong Lin edited comment on KAFKA-3978 at 3/10/18 8:35 AM: -- Finally confirmed the

[jira] [Commented] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

2018-03-10 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394103#comment-16394103 ] Dong Lin commented on KAFKA-3978: - Finally confirmed the root cause and is able to reproduce this using a

[jira] [Assigned] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

2018-03-08 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reassigned KAFKA-3978: --- Assignee: Dong Lin > Cannot truncate to a negative offset (-1) exception at broker startup >

[jira] [Commented] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

2018-03-08 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391863#comment-16391863 ] Dong Lin commented on KAFKA-3978: - [~junrao] [~hachikuji] [~ijuma] I have a good theory now. I will submit

[jira] [Commented] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

2018-03-08 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391698#comment-16391698 ] Dong Lin commented on KAFKA-3978: - I also encountered this issue and I have been looking at this all day

[jira] [Commented] (KAFKA-6624) log segment deletion could cause a disk to be marked offline incorrectly

2018-03-07 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390753#comment-16390753 ] Dong Lin commented on KAFKA-6624: - [~junrao] It may be reasonable to grab the lock of the log during

[jira] [Updated] (KAFKA-6618) Prevent two controllers from updating znodes concurrently

2018-03-06 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-6618: Issue Type: Bug (was: Improvement) > Prevent two controllers from updating znodes concurrently >

<    1   2   3   4   5   6   7   >