Re: Auto-rebalance not triggering in 2.10-0.8.1.1

2015-05-14 Thread Stephen Armstrong
Darn, I was looking at the defaults for 0.8.2, which is why I thought it
was enabled. Thanks for the help. Works fine now that I enabled it.

Steve

On Wed, May 13, 2015 at 3:25 PM, Jiangjie Qin j...@linkedin.com.invalid
wrote:

 Automatic preferred leader election hasn¹t been turned on in 0.8.1.1. It¹s
 been turned on in latest trunk though.

 The config name is ³auto.leader.rebalance.enable.

 Jiangjie (Becket) Qin

 On 5/13/15, 10:50 AM, Stephen Armstrong stephen.armstr...@linqia.com
 wrote:

 Does anyone have any insight into this? Am I correct that 0.8.1.1 should
 be
 running the leader election automatically? If this is a known issue, is
 there any reason not to have a cron script that runs the leader election
 regularly?
 
 Thanks
 Steve
 
 On Thu, May 7, 2015 at 2:47 PM, Stephen Armstrong 
 stephen.armstr...@linqia.com wrote:
 
  I'm running 2.10-0.8.1.1, and rebalance will not trigger on it's own.
 From
 
 
 http://grokbase.com/t/kafka/users/14bj5ps9hp/partition-auto-rebalance#201
 41118rf39q8cs4sjh6vzjgdw92e37cw
  I think the leader imbalance means: For a single broker, add up all the
  partitions it is leading (Y), and count the ones for which it's not the
  preferred broker (X). The ratio of X:Y is the one being used.
 
  I have about 10 topics spread between the 3 brokers, each with 4 or 8
  partitions. If I restart broker A, wait 5 min, then restart B,
 leadership
  ends up entirely on C (even though A was in ISR when B went down).
 Nothing
  triggers on it's own. Triggering it manually works (with
  bin/kafka-preferred-replica-election.sh).
 
  Is there something I should be checking, or is there a downside to just
  adding a cron job to trigger replica election once an hour?
 
  Thanks
  Steve
 
 
 
 
 
 
 --
 Stephen Armstrong
 Senior Software Engineer
 Linqia, Inc - Matching Marketers with Storytellers
 
 www.linqia.com
 Like http://www.facebook.com/linqia | Follow
 https://twitter.com/#%21/linqia | Blog http://www.linqia.com/blog/
 
 Email. stephen.armstr...@linqia.com
 
 The information in this e-mail and in any attachments is confidential and
 solely for the attention and use of the named addressee(s). You are hereby
 notified that any dissemination, distribution or copy of this
 communication
 is prohibited without the prior written consent of Linqia, Inc. If you
 have
 received this communication in error, please, notify the sender
 immediately
 by reply e-mail and delete this e-mail from your system as well as any
 files transmitted with it.




-- 
Stephen Armstrong
Senior Software Engineer
Linqia, Inc - Matching Marketers with Storytellers

www.linqia.com
Like http://www.facebook.com/linqia | Follow
https://twitter.com/#%21/linqia | Blog http://www.linqia.com/blog/

Email. stephen.armstr...@linqia.com

The information in this e-mail and in any attachments is confidential and
solely for the attention and use of the named addressee(s). You are hereby
notified that any dissemination, distribution or copy of this communication
is prohibited without the prior written consent of Linqia, Inc. If you have
received this communication in error, please, notify the sender immediately
by reply e-mail and delete this e-mail from your system as well as any
files transmitted with it.


Re: Auto-rebalance not triggering in 2.10-0.8.1.1

2015-05-13 Thread Stephen Armstrong
Does anyone have any insight into this? Am I correct that 0.8.1.1 should be
running the leader election automatically? If this is a known issue, is
there any reason not to have a cron script that runs the leader election
regularly?

Thanks
Steve

On Thu, May 7, 2015 at 2:47 PM, Stephen Armstrong 
stephen.armstr...@linqia.com wrote:

 I'm running 2.10-0.8.1.1, and rebalance will not trigger on it's own. From
 http://grokbase.com/t/kafka/users/14bj5ps9hp/partition-auto-rebalance#20141118rf39q8cs4sjh6vzjgdw92e37cw
 I think the leader imbalance means: For a single broker, add up all the
 partitions it is leading (Y), and count the ones for which it's not the
 preferred broker (X). The ratio of X:Y is the one being used.

 I have about 10 topics spread between the 3 brokers, each with 4 or 8
 partitions. If I restart broker A, wait 5 min, then restart B, leadership
 ends up entirely on C (even though A was in ISR when B went down). Nothing
 triggers on it's own. Triggering it manually works (with
 bin/kafka-preferred-replica-election.sh).

 Is there something I should be checking, or is there a downside to just
 adding a cron job to trigger replica election once an hour?

 Thanks
 Steve






-- 
Stephen Armstrong
Senior Software Engineer
Linqia, Inc - Matching Marketers with Storytellers

www.linqia.com
Like http://www.facebook.com/linqia | Follow
https://twitter.com/#%21/linqia | Blog http://www.linqia.com/blog/

Email. stephen.armstr...@linqia.com

The information in this e-mail and in any attachments is confidential and
solely for the attention and use of the named addressee(s). You are hereby
notified that any dissemination, distribution or copy of this communication
is prohibited without the prior written consent of Linqia, Inc. If you have
received this communication in error, please, notify the sender immediately
by reply e-mail and delete this e-mail from your system as well as any
files transmitted with it.


Auto-rebalance not triggering in 2.10-0.8.1.1

2015-05-07 Thread Stephen Armstrong
I'm running 2.10-0.8.1.1, and rebalance will not trigger on it's own. From
http://grokbase.com/t/kafka/users/14bj5ps9hp/partition-auto-rebalance#20141118rf39q8cs4sjh6vzjgdw92e37cw
I think the leader imbalance means: For a single broker, add up all the
partitions it is leading (Y), and count the ones for which it's not the
preferred broker (X). The ratio of X:Y is the one being used.

I have about 10 topics spread between the 3 brokers, each with 4 or 8
partitions. If I restart broker A, wait 5 min, then restart B, leadership
ends up entirely on C (even though A was in ISR when B went down). Nothing
triggers on it's own. Triggering it manually works (with
bin/kafka-preferred-replica-election.sh).

Is there something I should be checking, or is there a downside to just
adding a cron job to trigger replica election once an hour?

Thanks
Steve