Why producer did no use zookeeper

Hello all, I read the kafka source code, and I found the consumer use zookeeper. but the producer not user. I think the design it not better. do we will change the code? chenjinp...@conew.com

[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup

[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569586#comment-14569586 ] Adrian Muraru commented on KAFKA-2237: -- Do you know by chance the JIRA id where this

[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup

[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569583#comment-14569583 ] Adrian Muraru commented on KAFKA-2237: -- [~nehanarkhede] I'm on 0.8.2 branch

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Quick follow-up on the discussion on KIP-19. For partitionsFor() I think the question was whether to use max.enqueue.block.ms or request.timeout.ms to control the timeout. The proposed plan was to use request.timeout.ms. Alternately we could change the config max.enqueue.block.ms to max.block.ms

[jira] [Updated] (KAFKA-2236) offset request reply racing with segment rolling

[ https://issues.apache.org/jira/browse/KAFKA-2236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-2236: - Assignee: Jason Gustafson offset request reply racing with segment rolling

notes from Kafka KIP hangout Jun. 2

The following are my notes. 1. Security: Harsh will upload a new SSL patch in the next day or two. 2. KIP-21 (dynamic configuration): How to serve DescribeConfigRequest? Currently, topic config is only cached on brokers storing the topic. We will need to either cache the config on every broker

[jira] [Commented] (KAFKA-2187) Introduce merge-kafka-pr.py script

[ https://issues.apache.org/jira/browse/KAFKA-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569499#comment-14569499 ] Neha Narkhede commented on KAFKA-2187: -- [~ijuma] As I went through the patch again, I

[jira] [Updated] (KAFKA-2187) Introduce merge-kafka-pr.py script

[ https://issues.apache.org/jira/browse/KAFKA-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2187: --- Attachment: KAFKA-2187_2015-06-02_20:05:50.patch Introduce merge-kafka-pr.py script

[jira] [Commented] (KAFKA-2187) Introduce merge-kafka-pr.py script

[ https://issues.apache.org/jira/browse/KAFKA-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569592#comment-14569592 ] Ismael Juma commented on KAFKA-2187: Thanks [~nehanarkhede], I changed a couple of

Re: Review Request 34502: Patch for KAFKA-2187

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34502/ --- (Updated June 2, 2015, 7:06 p.m.) Review request for kafka. Bugs: KAFKA-2187

[jira] [Commented] (KAFKA-2187) Introduce merge-kafka-pr.py script

[ https://issues.apache.org/jira/browse/KAFKA-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569579#comment-14569579 ] Ismael Juma commented on KAFKA-2187: Updated reviewboard

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Thanks Jay. max.block.ms looks good. I will update the wiki page. Jiangjie (Becket) Qin On 6/2/15, 11:26 AM, Jay Kreps jay.kr...@gmail.com wrote: Quick follow-up on the discussion on KIP-19. For partitionsFor() I think the question was whether to use max.enqueue.block.ms or request.timeout.ms

Re: notes from Kafka KIP hangout Jun. 2

Jun, Thanks for the notes! @Jun/ Joel, An inline question on ISR info in TMR. On Tuesday, June 2, 2015, Jun Rao j...@confluent.io wrote: The following are my notes. 1. Security: Harsh will upload a new SSL patch in the next day or two. 2. KIP-21 (dynamic configuration): How to serve

[jira] [Created] (KAFKA-2239) Allow coordinator to aggresively remove group when it is no longer owning the group

Guozhang Wang created KAFKA-2239: Summary: Allow coordinator to aggresively remove group when it is no longer owning the group Key: KAFKA-2239 URL: https://issues.apache.org/jira/browse/KAFKA-2239

[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup

[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569752#comment-14569752 ] Adrian Muraru commented on KAFKA-2237: -- False alarm, yeah on 0.8.2 it seems to work

[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup

[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569672#comment-14569672 ] Ewen Cheslack-Postava commented on KAFKA-2237: -- KAFKA-404 which is pretty old

Re: Review Request 33620: Patch for KAFKA-1690

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review86250 ---

[VOTE] KIP-19 Add a request timeout to NetworkClient

Hi folks, Thanks a lot for all the input and help. It looks we do not have further concerns on KIP-19. I’ve updated the wiki page, let’s vote! https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient Thanks, Jiangjie (Becket) Qin

[jira] [Created] (KAFKA-2238) KafkaMetricsConfig not documented in KafkaConfig

Aditya Auradkar created KAFKA-2238: -- Summary: KafkaMetricsConfig not documented in KafkaConfig Key: KAFKA-2238 URL: https://issues.apache.org/jira/browse/KAFKA-2238 Project: Kafka Issue

[jira] [Resolved] (KAFKA-2237) zookeeper chroot path should be created on broker startup

[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2237. -- Resolution: Duplicate zookeeper chroot path should be created on broker

[jira] [Commented] (KAFKA-2208) Consumer should detect coordinator failures

[ https://issues.apache.org/jira/browse/KAFKA-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569768#comment-14569768 ] Guozhang Wang commented on KAFKA-2208: -- Updated reviewboard

Re: Review Request 34524: Fix KAFKA-2208

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34524/ --- (Updated June 2, 2015, 9:07 p.m.) Review request for kafka. Bugs: KAFKA-2208

[jira] [Commented] (KAFKA-2238) KafkaMetricsConfig not documented in KafkaConfig

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570048#comment-14570048 ] Aditya A Auradkar commented on KAFKA-2238: -- Created reviewboard

[jira] [Updated] (KAFKA-2238) KafkaMetricsConfig not documented in KafkaConfig

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2238: - Attachment: KAFKA-2238.patch KafkaMetricsConfig not documented in KafkaConfig

Review Request 34966: Patch for KAFKA-2238

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34966/ --- Review request for kafka. Bugs: KAFKA-2238

[jira] [Updated] (KAFKA-2238) KafkaMetricsConfig not documented in KafkaConfig

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2238: --- Description: All metrics config values are not included in KafkaConfig and consequently do

[jira] [Updated] (KAFKA-2238) KafkaMetricsConfig cannot be configured in broker (KafkaConfig)

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2238: --- Summary: KafkaMetricsConfig cannot be configured in broker (KafkaConfig) (was:

[jira] [Updated] (KAFKA-2238) KafkaMetricsConfig cannot be configured in broker (KafkaConfig)

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2238: --- Description: All metrics config values are not included in KafkaConfig and consequently

Re: Review Request 33620: Patch for KAFKA-1690

On May 22, 2015, 12:33 a.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, lines 240-242 https://reviews.apache.org/r/33620/diff/8/?file=966806#file966806line240 Would it be better to just throw the exception in the NOT_HANDSHAKING

RE: [VOTE] KIP-19 Add a request timeout to NetworkClient

+1 (non-binding). One minor comment. Should we rename max.block.ms to max.send.block.ms? Something that indicates that it is being applied to the send API methods exposed by KafkaProducer. Aditya From: Joel Koshy [jjkosh...@gmail.com] Sent: Tuesday,

[jira] [Commented] (KAFKA-2170) 10 LogTest cases failed for file.renameTo failed under windows

[ https://issues.apache.org/jira/browse/KAFKA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570031#comment-14570031 ] Honghai Chen commented on KAFKA-2170: - Need make sure all test cases passed in

[jira] [Updated] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2168: --- Attachment: KAFKA-2168_2015-06-02_17:09:37.patch New consumer poll() can block other calls

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated June 3, 2015, 12:10 a.m.) Review request for kafka, Joel Koshy and

Re: Review Request 34789: Patch for KAFKA-2168

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 3, 2015, 12:10 a.m.) Review request for kafka. Bugs:

Re: [VOTE] KIP-19 Add a request timeout to NetworkClient

+1 (non-binding). Thanks Jiangjie. Thanks, Mayuresh On Tue, Jun 2, 2015 at 4:49 PM, Ewen Cheslack-Postava e...@confluent.io wrote: +1 non-binding. Thanks for all the work on this Jiangjie! Aditya -- see discussion of partitionsFor() API, which needs to also use one of those timeouts, and

Re: [VOTE] KIP-19 Add a request timeout to NetworkClient

+1 non-binding. Thanks for all the work on this Jiangjie! Aditya -- see discussion of partitionsFor() API, which needs to also use one of those timeouts, and is a fetch rather than a send since it's just getting metadata. -Ewen On Tue, Jun 2, 2015 at 4:44 PM, Aditya Auradkar

Re: [VOTE] KIP-19 Add a request timeout to NetworkClient

+1 On Tue, Jun 02, 2015 at 10:13:11PM +, Jiangjie Qin wrote: Ah, yes. Just changed it to ³request.timeout.ms². Thanks. Jiangjie (Becket) Qin On 6/2/15, 2:31 PM, Jun Rao j...@confluent.io wrote: The wiki references network.request.timeout.ms in ProducerConfig. It should be

[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569988#comment-14569988 ] Dong Lin commented on KAFKA-2241: - Oh, I thought the first issue creation has failed.

[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570010#comment-14570010 ] Dong Lin commented on KAFKA-2241: - Created reviewboard

[jira] [Updated] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2241: Attachment: KAFKA-2241.patch AbstractFetcherThread.shutdown() should not block on

[jira] [Updated] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2241: Status: Patch Available (was: Open) AbstractFetcherThread.shutdown() should not block on

Review Request 34965: Patch for KAFKA-2241

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34965/ --- Review request for kafka. Bugs: KAFKA-2241

[jira] [Commented] (KAFKA-2170) 10 LogTest cases failed for file.renameTo failed under windows

[ https://issues.apache.org/jira/browse/KAFKA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570023#comment-14570023 ] Honghai Chen commented on KAFKA-2170: - [~junrao] The logic has been triggered (add

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated June 3, 2015, 12:09 a.m.) Review request for kafka, Joel Koshy and

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570036#comment-14570036 ] Jason Gustafson commented on KAFKA-2168: Updated reviewboard

[jira] [Commented] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570034#comment-14570034 ] Aditya A Auradkar commented on KAFKA-2084: -- Updated reviewboard

[jira] [Updated] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2084: - Attachment: KAFKA-2084_2015-06-02_17:09:28.patch byte rate metrics per client ID

[jira] [Resolved] (KAFKA-2240) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-2240. - Resolution: Duplicate AbstractFetcherThread.shutdown() should not block on

[jira] [Closed] (KAFKA-2240) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin closed KAFKA-2240. --- Duplicate of KAFKA-2241. AbstractFetcherThread.shutdown() should not block on

[jira] [Created] (KAFKA-2240) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

Dong Lin created KAFKA-2240: --- Summary: AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer) Key: KAFKA-2240 URL: https://issues.apache.org/jira/browse/KAFKA-2240 Project:

[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569934#comment-14569934 ] Ismael Juma commented on KAFKA-2241: [~lindong], it looks like you filed two issues

Re: Review Request 34524: Fix KAFKA-2208

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34524/#review86304 --- Ship it! I brought up a local vagrant cluster of 2 brokers, 1 zk

[jira] [Updated] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2241: Attachment: client.java server.java Run server before client. Then check if channel.read()

Re: notes from Kafka KIP hangout Jun. 2

We have a proposal to propagate ISR changes to every broker in KAFKA-1367. Thanks, Jun On Tue, Jun 2, 2015 at 12:55 PM, Ashish Singh asi...@cloudera.com wrote: Jun, Thanks for the notes! @Jun/ Joel, An inline question on ISR info in TMR. On Tuesday, June 2, 2015, Jun Rao

Re: [VOTE] KIP-19 Add a request timeout to NetworkClient

+1. Thanks Jiangjie! On Tue, Jun 2, 2015 at 2:06 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi folks, Thanks a lot for all the input and help. It looks we do not have further concerns on KIP-19. I’ve updated the wiki page, let’s vote!

Re: [VOTE] KIP-19 Add a request timeout to NetworkClient

The wiki references network.request.timeout.ms in ProducerConfig. It should be request.timeout.ms, right? Thanks, Jun On Tue, Jun 2, 2015 at 2:06 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi folks, Thanks a lot for all the input and help. It looks we do not have further concerns on

[jira] [Created] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

Dong Lin created KAFKA-2241: --- Summary: AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer) Key: KAFKA-2241 URL: https://issues.apache.org/jira/browse/KAFKA-2241 Project:

[jira] [Commented] (KAFKA-2242) Fix license headers (remove copyright notices) in a few test sources

[ https://issues.apache.org/jira/browse/KAFKA-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570230#comment-14570230 ] Joel Koshy commented on KAFKA-2242: --- Thanks - sorry I missed the detail on KAFKA-2161

[jira] [Commented] (KAFKA-2161) Fix a few copyrights

[ https://issues.apache.org/jira/browse/KAFKA-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570235#comment-14570235 ] Joel Koshy commented on KAFKA-2161: --- Pushed to trunk. Ping [~charmalloc] on above

Re: [VOTE] KIP-19 Add a request timeout to NetworkClient

+1 By the way thanks Becket for following this all the way through. I know it seems crazy to put this much thought into configuration but the reality is that people really struggle to configure systems like this, and when they fail then things break. So from the user's point of view confusing is

[jira] [Created] (KAFKA-2242) Fix license headers (remove copyright notices) in a few test sources

Joel Koshy created KAFKA-2242: - Summary: Fix license headers (remove copyright notices) in a few test sources Key: KAFKA-2242 URL: https://issues.apache.org/jira/browse/KAFKA-2242 Project: Kafka

[jira] [Resolved] (KAFKA-2242) Fix license headers (remove copyright notices) in a few test sources

[ https://issues.apache.org/jira/browse/KAFKA-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2242. -- Resolution: Duplicate [~jjkoshy] Yup, already caught this and it's filed with a

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Option 3 seems a lot better than previous options, especially from the user's perspective. I think it gives reasonable balance between control and fewer options, and the only implementation details it's exposing are that there is a buffer and there is a network request. Making the request timeout

[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper

[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569420#comment-14569420 ] Joel Koshy commented on KAFKA-1367: --- You may still want to switch back to a replica in

[jira] [Created] (KAFKA-2237) zookeeper chroot path should be created on broker startup

Adrian Muraru created KAFKA-2237: Summary: zookeeper chroot path should be created on broker startup Key: KAFKA-2237 URL: https://issues.apache.org/jira/browse/KAFKA-2237 Project: Kafka

Re: Review Request 34524: Fix KAFKA-2208

On June 2, 2015, 12:20 a.m., Onur Karaman wrote: It'd rather avoid mixing coordinator failover optimization logic with this rb. Can you undo the changes in ConsumerCoordinator.scala from line 214 down to the bottom of ConsumerCoordinator.scala? OK agreed. Revert the remove-group logic

[jira] [Updated] (KAFKA-2208) Consumer should detect coordinator failures

[ https://issues.apache.org/jira/browse/KAFKA-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2208: - Attachment: KAFKA-2208_2015-06-02_10:45:34.patch Consumer should detect coordinator failures

Re: Review Request 34524: Fix KAFKA-2208

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34524/ --- (Updated June 2, 2015, 5:45 p.m.) Review request for kafka. Bugs: KAFKA-2208

[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper

[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569484#comment-14569484 ] Jun Rao commented on KAFKA-1367: [~joel koshy], yes, that's a good point. If we want to