Re: topic's partition have no leader and isr

2014-07-09 Thread 鞠大升
@Jun Rao, Kafka version: 0.8.1.1 @Guozhang Wang, I can not found the original controller log, but I can give the controller log after execute ./bin/kafka-reassign-partitions.sh and ./bin/kafka-preferred-replica-election.sh Now I do not known how to recover leader for partition 25 and 31, any

Re: topic's partition have no leader and isr

2014-07-09 Thread 鞠大升
I think @chenlax has encounter the same problem with me in us...@kafka.apache.org with titile How recover leader when broker restart. cc to us...@kafka.apache.org. On Wed, Jul 9, 2014 at 3:10 PM, 鞠大升 dashen...@gmail.com wrote: @Jun Rao, Kafka version: 0.8.1.1 @Guozhang Wang, I can not

[jira] [Created] (KAFKA-1531) zookeeper.connection.timeout.ms is set to 10000000 in configuration file in Kafka tarball

2014-07-09 Thread JIRA
Michał Michalski created KAFKA-1531: --- Summary: zookeeper.connection.timeout.ms is set to 1000 in configuration file in Kafka tarball Key: KAFKA-1531 URL: https://issues.apache.org/jira/browse/KAFKA-1531

[jira] [Updated] (KAFKA-1531) zookeeper.connection.timeout.ms is set to 10000000 in configuration file in Kafka tarball

2014-07-09 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Michalski updated KAFKA-1531: Description: I've noticed that Kafka tarball comes with

[jira] [Commented] (KAFKA-1530) howto update continuously

2014-07-09 Thread Stanislav Gilmulin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056049#comment-14056049 ] Stanislav Gilmulin commented on KAFKA-1530: --- Thank you, i'd like to ask some

[jira] [Updated] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-09 Thread Alexey Ozeritskiy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Ozeritskiy updated KAFKA-1414: - Attachment: parallel-dir-loading-trunk-threadpool.patch Speedup broker startup after

[jira] [Commented] (KAFKA-1406) Fix scaladoc/javadoc warnings

2014-07-09 Thread Alan Lee (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056147#comment-14056147 ] Alan Lee commented on KAFKA-1406: - Would it be okay if I work on this? Fix

[jira] [Updated] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1325: --- Attachment: (was: KAFKA-1325.patch) Fix inconsistent per topic log configs

Re: topic's partition have no leader and isr

2014-07-09 Thread Jun Rao
It's weird that you have replication factor 1, but two of the partitions 25 and 31 have 2 assigned replicas. What's the command you used for reassignment? Thanks, Jun On Wed, Jul 9, 2014 at 12:10 AM, 鞠大升 dashen...@gmail.com wrote: @Jun Rao, Kafka version: 0.8.1.1 @Guozhang Wang, I can

Review Request 23362: Patch for KAFKA-1325

2014-07-09 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23362/ --- Review request for kafka. Bugs: KAFKA-1325

[jira] [Updated] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1325: --- Attachment: KAFKA-1325.patch Fix inconsistent per topic log configs

[jira] [Commented] (KAFKA-1406) Fix scaladoc/javadoc warnings

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056299#comment-14056299 ] Jun Rao commented on KAFKA-1406: Alan, That would be great. Thanks for your help. Fix

[jira] [Commented] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056300#comment-14056300 ] Manikumar Reddy commented on KAFKA-1325: Created reviewboard

[jira] [Updated] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1325: --- Attachment: (was: KAFKA-1325.patch) Fix inconsistent per topic log configs

[jira] [Commented] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056319#comment-14056319 ] Jun Rao commented on KAFKA-1325: Thanks for the patch. It seems to me that ms granularity

[jira] [Commented] (KAFKA-1528) Normalize all the line endings

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056323#comment-14056323 ] Jun Rao commented on KAFKA-1528: Thanks for the patch. It doesn't apply to trunk though.

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056338#comment-14056338 ] Jun Rao commented on KAFKA-1414: Thanks for the patch. A few comments. 1. If there are

[jira] [Updated] (KAFKA-1406) Fix scaladoc/javadoc warnings

2014-07-09 Thread Alan Lee (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Lee updated KAFKA-1406: Attachment: kafka-1406-v1.patch I've attached a patch file for the fix. There are still few (non variable

[jira] [Commented] (KAFKA-1530) howto update continuously

2014-07-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056339#comment-14056339 ] Guozhang Wang commented on KAFKA-1530: -- Not sure I understand your question, what do

Re: topic's partition have no leader and isr

2014-07-09 Thread Guozhang Wang
It seems your broker 1 is in a bad state, besides these two partitions you also have partition 10 whose Isr/Leader is not part of the replicas list: Topic: org.mobile_nginx Partition: 10 Leader: 2 Replicas: 1 Isr: 2 Maybe you can go to broker 1 and check its logs first. Guozhang On

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-09 Thread Alexey Ozeritskiy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056358#comment-14056358 ] Alexey Ozeritskiy commented on KAFKA-1414: -- 1. Ok 2. If any thread fails we get

Review Request 23363: Patch for KAFKA-1325

2014-07-09 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23363/ --- Review request for kafka. Bugs: KAFKA-1325

[jira] [Updated] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1325: --- Attachment: KAFKA-1325.patch Fix inconsistent per topic log configs

[jira] [Commented] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056402#comment-14056402 ] Manikumar Reddy commented on KAFKA-1325: Created reviewboard

[jira] [Commented] (KAFKA-1325) Fix inconsistent per topic log configs

2014-07-09 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056410#comment-14056410 ] Manikumar Reddy commented on KAFKA-1325: I added log.retention.ms and log.roll.ms

[jira] [Commented] (KAFKA-1530) howto update continuously

2014-07-09 Thread Stanislav Gilmulin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056411#comment-14056411 ] Stanislav Gilmulin commented on KAFKA-1530: --- It means we have the risks. You're

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056417#comment-14056417 ] Jun Rao commented on KAFKA-1414: 2. Yes, that probably right. Could you verify that the

[jira] [Commented] (KAFKA-1530) howto update continuously

2014-07-09 Thread Oleg (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056422#comment-14056422 ] Oleg commented on KAFKA-1530: - Another situation which happened in our production: We have

[jira] [Commented] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056424#comment-14056424 ] Jun Rao commented on KAFKA-1507: Thanks for the patch. I am currently working on

[jira] [Comment Edited] (KAFKA-1530) howto update continuously

2014-07-09 Thread Oleg (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056422#comment-14056422 ] Oleg edited comment on KAFKA-1530 at 7/9/14 4:33 PM: - Another

[jira] [Resolved] (KAFKA-1515) Wake-up Sender upon blocked on fetching leader metadata

2014-07-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1515. -- Resolution: Fixed Wake-up Sender upon blocked on fetching leader metadata

[jira] [Commented] (KAFKA-1019) kafka-preferred-replica-election.sh will fail without clear error message if /brokers/topics/[topic]/partitions does not exist

2014-07-09 Thread Mickael Hemri (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056440#comment-14056440 ] Mickael Hemri commented on KAFKA-1019: -- Hi Jun, We use ZK 3.4.6, can it cause this

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056441#comment-14056441 ] Jay Kreps commented on KAFKA-1532: -- This is a good idea, but unfortunately would break

[jira] [Commented] (KAFKA-1530) howto update continuously

2014-07-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056448#comment-14056448 ] Guozhang Wang commented on KAFKA-1530: -- Hi Stanislav/Oleg, Kafka server has a config

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056444#comment-14056444 ] Jay Kreps commented on KAFKA-1532: -- Also since Kafka itself fully materializes messages

[jira] [Commented] (KAFKA-1019) kafka-preferred-replica-election.sh will fail without clear error message if /brokers/topics/[topic]/partitions does not exist

2014-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056456#comment-14056456 ] Jun Rao commented on KAFKA-1019: I am not sure how reliable ZK 3.4.6 is. If you check the

[jira] [Commented] (KAFKA-1528) Normalize all the line endings

2014-07-09 Thread Evgeny Vereshchagin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056521#comment-14056521 ] Evgeny Vereshchagin commented on KAFKA-1528: [~junrao], what command do you

Re: topic's partition have no leader and isr

2014-07-09 Thread DashengJu
that's because the topic have replication factor 2 at first, then I want to reassign all partitions 2 replicas to 1 replicas, so all partitions changed success except 25 and 31. I think before the reassign operation partitions 25 and 31 became to have no leader, cause the reassign operation

Re: topic's partition have no leader and isr

2014-07-09 Thread DashengJu
thanks for your found, it is weird. i checked my broker 1, it is working fine, no error or warn log. I checked the data folder, and found replication 10's replicas, isr and leader is actually on broker 2, works fine. just now I execute a reassignment operation to move partition 10's replicas to

[jira] [Commented] (KAFKA-1528) Normalize all the line endings

2014-07-09 Thread Evgeny Vereshchagin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056720#comment-14056720 ] Evgeny Vereshchagin commented on KAFKA-1528: kafka-patch-review.py squashes

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Julian Morrison (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056792#comment-14056792 ] Julian Morrison commented on KAFKA-1532: Is Kafka itself fully materializes

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056802#comment-14056802 ] Jay Kreps commented on KAFKA-1532: -- It's not unavoidable, but it is pretty hard. To make

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056806#comment-14056806 ] Jay Kreps commented on KAFKA-1532: -- Your approach where you buffer the message with a

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Julian Morrison (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056816#comment-14056816 ] Julian Morrison commented on KAFKA-1532: Streaming it to a scratch file (never

[jira] [Commented] (KAFKA-1532) Move CRC32 to AFTER the payload

2014-07-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056833#comment-14056833 ] Jay Kreps commented on KAFKA-1532: -- Yeah agreed. I think that is doable. Move CRC32 to