[jira] [Commented] (KAFKA-2722) Improve ISR change propagation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989554#comment-14989554
 ] 

ASF GitHub Bot commented on KAFKA-2722:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/402


> Improve ISR change propagation
> --
>
> Key: KAFKA-2722
> URL: https://issues.apache.org/jira/browse/KAFKA-2722
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> Currently the ISR change propagation interval is hard coded to 5 seconds, 
> this might still create a lot of ISR change propagation for a large cluster 
> in cases such as rolling bounce. The patch uses a dynamic propagation 
> interval and fixed a performance bug in IsrChangeNotificationListener on 
> controller.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2722) Improve ISR change propagation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985686#comment-14985686
 ] 

ASF GitHub Bot commented on KAFKA-2722:
---

Github user becketqin closed the pull request at:

https://github.com/apache/kafka/pull/402


> Improve ISR change propagation
> --
>
> Key: KAFKA-2722
> URL: https://issues.apache.org/jira/browse/KAFKA-2722
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> Currently the ISR change propagation interval is hard coded to 5 seconds, 
> this might still create a lot of ISR change propagation for a large cluster 
> in cases such as rolling bounce. The patch uses a dynamic propagation 
> interval and fixed a performance bug in IsrChangeNotificationListener on 
> controller.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2722) Improve ISR change propagation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985687#comment-14985687
 ] 

ASF GitHub Bot commented on KAFKA-2722:
---

GitHub user becketqin reopened a pull request:

https://github.com/apache/kafka/pull/402

KAFKA-2722: Improve ISR change propagation.

The patch has two changes:
1. fixed a bug in controller that it sends UpdateMetadataRequest of all the 
partitions in the cluster.
2. Uses the following rules to propagate ISR change: 1) if there are ISR 
changes pending propagation and the last ISR change is more than five seconds 
ago, propagate the changes. 2) if there is ISR change at T in the recent five 
seconds, delay the propagation until T + 5s. 3) if the last propagation is more 
than 1 min ago, ignore rule No.2 and propagate ISR change if there are changes 
pending propagation.

This algorithm avoids a fixed configuration of ISR propagation interval as 
we discussed about in KIP-29.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/becketqin/kafka KAFKA-2722

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/402.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #402


commit 13892856d806183536657f0c3ea2aa63b1f1c4f2
Author: Jiangjie Qin 
Date:   2015-11-02T01:26:27Z

Improve ISR change propagation.




> Improve ISR change propagation
> --
>
> Key: KAFKA-2722
> URL: https://issues.apache.org/jira/browse/KAFKA-2722
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> Currently the ISR change propagation interval is hard coded to 5 seconds, 
> this might still create a lot of ISR change propagation for a large cluster 
> in cases such as rolling bounce. The patch uses a dynamic propagation 
> interval and fixed a performance bug in IsrChangeNotificationListener on 
> controller.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)