Re: Review Request 33196: Patch for KAFKA-2123

2015-06-04 Thread Jason Gustafson
On May 31, 2015, 9:10 p.m., Guozhang Wang wrote: Thanks for the explanation Ewen. I agree that a delayed scheduler would be a good fit here, but was originally more concerned about the complexity we introduced by adding two queues (one for delayed actions and another for handling

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-04 Thread Jason Gustafson
419541011d652becf0cda7a5e62ce813cddb1732 clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-04 Thread Jason Gustafson
: https://reviews.apache.org/r/34789/#review86651 --- On June 4, 2015, 9:36 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-04 Thread Jason Gustafson
, 2015, 4:07 a.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 4, 2015, 4:07 a.m

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-05 Thread Jason Gustafson
ecc78cedf59a994fcf084fa7a458fe9ed5386b00 clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-05 Thread Jason Gustafson
/HeartbeatTest.java ecc78cedf59a994fcf084fa7a458fe9ed5386b00 clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-03 Thread Jason Gustafson
:10 a.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 3, 2015, 12:10 a.m

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-02 Thread Jason Gustafson
/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732 clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-09 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/#review87257 --- On June 5, 2015, 7:45 p.m., Jason Gustafson wrote

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-03 Thread Jason Gustafson
/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-03 Thread Jason Gustafson
/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: [Discussion] New Consumer API / Protocol

2015-06-09 Thread Jason Gustafson
Just a minor correction, but #2 is KAFKA-2168 and #3 is KAFKA-2123. For #1, I think there should be some minimal effort to making the poll respect the timeout (with the understanding that spurious wakeups can happen). I think this really just means calling NetworkClient.poll() in a loop and

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-09 Thread Jason Gustafson
: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-23 Thread Jason Gustafson
--- On June 23, 2015, 4:39 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-23 Thread Jason Gustafson
9be8fbc648369ad9db1a7eea94bc1b9edbfdbfd7 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 35791: Patch for KAFKA-2298

2015-06-23 Thread Jason Gustafson
/Selector.java (line 282) https://reviews.apache.org/r/35791/#comment141609 Do you think we should just move the disconnected.add() into the close method? - Jason Gustafson On June 23, 2015, 5:41 p.m., Dong Lin wrote

Re: Review Request 35791: Patch for KAFKA-2298

2015-06-23 Thread Jason Gustafson
On June 23, 2015, 5:59 p.m., Jason Gustafson wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, line 282 https://reviews.apache.org/r/35791/diff/1/?file=990592#file990592line282 Do you think we should just move the disconnected.add() into the close

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Jason Gustafson
/test/java/org/apache/kafka/common/utils/UtilsTest.java 2ebe3c21f611dc133a2dbb8c7dfb0845f8c21498 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Jason Gustafson
/#review88690 --- On June 19, 2015, 4:48 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35655

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Jason Gustafson
the test case? - Jason --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35655/#review88877 --- On June 19, 2015, 4:48 p.m., Jason Gustafson

Review Request 35655: Patch for KAFKA-2271

2015-06-19 Thread Jason Gustafson
, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-19 Thread Jason Gustafson
2ebe3c21f611dc133a2dbb8c7dfb0845f8c21498 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-18 Thread Jason Gustafson
, 2015, 9:40 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 18, 2015, 9:40 p.m

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-18 Thread Jason Gustafson
: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-18 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/#review88451 --- On June 18, 2015, 9:40 p.m., Jason Gustafson wrote

Re: Review Request 35791: Patch for KAFKA-2298

2015-06-24 Thread Jason Gustafson
On June 23, 2015, 5:59 p.m., Jason Gustafson wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, line 282 https://reviews.apache.org/r/35791/diff/1/?file=990592#file990592line282 Do you think we should just move the disconnected.add() into the close

Re: Review Request 35791: Patch for KAFKA-2298

2015-06-24 Thread Jason Gustafson
On June 23, 2015, 5:59 p.m., Jason Gustafson wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, line 282 https://reviews.apache.org/r/35791/diff/1/?file=990592#file990592line282 Do you think we should just move the disconnected.add() into the close

Re: Review Request 35791: Patch for KAFKA-2298

2015-06-24 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35791/#review89241 --- Ship it! Ship It! - Jason Gustafson On June 24, 2015, 8 p.m

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-10 Thread Jason Gustafson
that we could just return when wakeup is called. Jason Gustafson wrote: You might be waking up from a synchronous commit, for example. In that case, all we can do is raise an exception. We could alternatively say that wakeup only applies to the poll() method and cannot be used

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-10 Thread Jason Gustafson
the wakeup check. Jason Gustafson wrote: I might be wrong, but I think we can still use NetworkClient.wakeup to interrupt a poll call which is waiting forever. Jun Rao wrote: Yes, you are right. I missed that. Jun Rao wrote: Actually, this seems to be still a problem

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-11 Thread Jason Gustafson
, at the end of the call, we expect the fetch offset to be set to the beginning. This is now changed to async, which doesn't match the intended behavior. We need to think through if this matters or not. Ditto for seekToEnd(). Jason Gustafson wrote: Since we always update fetch

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-11 Thread Jason Gustafson
because the protocol responses use that terminology. Future? Result? Jason Gustafson wrote: Agreed. In fact, they were XResult initially. I changed them because BrokerResult and CoordinatorResult didn't seems to suggest as clearly what they were for as BrokerResponse

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-11 Thread Jason Gustafson
e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-16 Thread Jason Gustafson
--- On June 11, 2015, 9:10 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789

Re: Review Request 35231: Fix KAFKA-1740

2015-06-17 Thread Jason Gustafson
285) https://reviews.apache.org/r/35231/#comment140756 Same as above, wrong ticket. - Jason Gustafson On June 8, 2015, 11:12 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 35371: Patch for KAFKA-2266

2015-06-12 Thread Jason Gustafson
effb1e63081ed2c1fff6d08d4ecdf8a3cb43e40a clients/src/test/java/org/apache/kafka/common/network/SelectorTest.java d23b4b6a7060eeefa9f47f292fd818c881d789c1 Diff: https://reviews.apache.org/r/35371/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 35371: Patch for KAFKA-2266

2015-06-12 Thread Jason Gustafson
generated e-mail. To reply, visit: https://reviews.apache.org/r/35371/#review87667 --- On June 12, 2015, 4:19 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail

Review Request 35371: Patch for KAFKA-2266

2015-06-11 Thread Jason Gustafson
/apache/kafka/common/network/SelectorTest.java d23b4b6a7060eeefa9f47f292fd818c881d789c1 Diff: https://reviews.apache.org/r/35371/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-03 Thread Jason Gustafson
of the big ones and using named nested classes might help improve clarity, although it does separate the request initiating code from the response handling code. Jason Gustafson wrote: Yeah, that's right. The offsetBefore method was the tricky one to deal with. I tried to keep

Review Request 34789: Patch for KAFKA-2168

2015-05-28 Thread Jason Gustafson
ef9dd5238fbc771496029866ece1d85db6d7b7a5 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34789: Patch for KAFKA-2168

2015-05-28 Thread Jason Gustafson
28, 2015, 10:58 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated May 28, 2015, 10:58

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-01 Thread Jason Gustafson
/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732 clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-27 Thread Jason Gustafson
d5b306b026e788b4e5479f3419805aa49ae889f3 clients/src/test/java/org/apache/kafka/test/MockSelector.java ea89b06a4c9e5bb351201299cd3037f5226f0e6c Diff: https://reviews.apache.org/r/34608/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-27 Thread Jason Gustafson
GC issues might become an issue. Jason Gustafson wrote: I'll check it out. We could also reuse the same PollResult object, but that sort of defeats the point of the change. I had hoped that by making the underlying objects safer for threaded usage, some of the complexity

Re: Review Request 35231: Address Onur and Jason's comments

2015-07-01 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35231/#review90155 --- Ship it! Ship It! - Jason Gustafson On June 30, 2015, 1:44 a.m

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-25 Thread Jason Gustafson
--- On May 25, 2015, 5:46 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34608/ --- (Updated May 25

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-25 Thread Jason Gustafson
, Jason Gustafson

Review Request 34608: Patch for KAFKA-2217

2015-05-22 Thread Jason Gustafson
/java/org/apache/kafka/common/network/SelectorTest.java d5b306b026e788b4e5479f3419805aa49ae889f3 clients/src/test/java/org/apache/kafka/test/MockSelector.java ea89b06a4c9e5bb351201299cd3037f5226f0e6c Diff: https://reviews.apache.org/r/34608/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-26 Thread Jason Gustafson
/kafka/test/MockSelector.java ea89b06a4c9e5bb351201299cd3037f5226f0e6c Diff: https://reviews.apache.org/r/34608/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-26 Thread Jason Gustafson
GC issues might become an issue. Jason Gustafson wrote: I'll check it out. We could also reuse the same PollResult object, but that sort of defeats the point of the change. I had hoped that by making the underlying objects safer for threaded usage, some of the complexity

Re: Kafka Consumer thoughts

2015-07-31 Thread Jason Gustafson
guess that with the new setAssignment() api, we will also be getting rid of pause() and resume()? Thanks, Jun On Fri, Jul 31, 2015 at 11:29 AM, Jason Gustafson ja...@confluent.io wrote: I was thinking a little bit this morning about the subscription API and I have a few ideas on how

Re: Review Request 36858: Patch for KAFKA-2120

2015-08-12 Thread Jason Gustafson
On Aug. 11, 2015, 8:49 p.m., Jason Gustafson wrote: clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java, line 302 https://reviews.apache.org/r/36858/diff/4/?file=1037078#file1037078line302 Can we make this value greater than sessionTimeoutMs (which is 30s

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-17 Thread Jason Gustafson
validate by running a few tests on the patch and I suggest we wait for that. Thanks, Neha On Sat, Aug 15, 2015 at 9:14 AM, Jason Gustafson ja...@confluent.io wrote: Hey Jiangjie, I was thinking about the same problem. When metadata is changing frequently

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-18 Thread Jason Gustafson
as we are not changing topic more than 5 times per sec, there shouldn't be constant churns, right? Thanks, Jun On Tue, Aug 11, 2015 at 1:19 PM, Jason Gustafson ja...@confluent.io wrote: Hi Kafka Devs, One of the nagging issues in the current design of the new consumer has been

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-18 Thread Jason Gustafson
for consumers. Would that be an issue? Thanks, Jiangjie (Becket) Qin On Tue, Aug 18, 2015 at 10:22 AM, Jason Gustafson ja...@confluent.io wrote: Hi Jun, Answers below: 1. When there are multiple common protocols in the JoinGroupRequest, which one would the coordinator pick? I

Re: Review Request 35231: Address Onur and Jason's comments

2015-06-30 Thread Jason Gustafson
On June 18, 2015, 12:50 a.m., Jason Gustafson wrote: core/src/main/scala/kafka/coordinator/ConsumerCoordinator.scala, lines 229-236 https://reviews.apache.org/r/35231/diff/1/?file=980918#file980918line229 In manual group management, would we expect consumerId and generationId

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-30 Thread Jason Gustafson
/integration/kafka/api/ConsumerBounceTest.scala f56096b826bdbf760411a54ba067a6a83eca8a10 Diff: https://reviews.apache.org/r/34789/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 35231: Address Onur and Jason's comments

2015-06-29 Thread Jason Gustafson
On June 18, 2015, 12:50 a.m., Jason Gustafson wrote: core/src/main/scala/kafka/coordinator/ConsumerCoordinator.scala, lines 229-236 https://reviews.apache.org/r/35231/diff/1/?file=980918#file980918line229 In manual group management, would we expect consumerId and generationId

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-30 Thread Jason Gustafson
://reviews.apache.org/r/34789/#review89971 --- On June 30, 2015, 5:55 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789

Re: Kafka Consumer thoughts

2015-07-29 Thread Jason Gustafson
, but this is the first kafka consumer being used by the Databus team. So I don't really think the feedback received was about the simpler times or wanting additional higher-level clients. - Onur On Mon, Jul 27, 2015 at 1:41 PM, Jason Gustafson ja...@confluent.io wrote: I think if we recommend

Re: Kafka Consumer thoughts

2015-07-31 Thread Jason Gustafson
team had prior experience with the old SimpleConsumer, but this is the first kafka consumer being used by the Databus team. So I don't really think the feedback received was about the simpler times or wanting additional higher-level clients. - Onur On Mon, Jul 27, 2015 at 1:41 PM, Jason

Re: Review Request 36871: Patch for KAFKA-2381

2015-07-28 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36871/#review93321 --- Ship it! Ship It! - Jason Gustafson On July 28, 2015, 4:17 p.m

Re: New Consumer API and Range Consumption with Fail-over

2015-08-05 Thread Jason Gustafson
are correct we need pause(TopicPartition tp), resume(TopicPartition tp), and/or API to set to end offset for each partition. Please do let us know your preference to support above simple use-case. Thanks, Bhavesh On Thu, Jul 30, 2015 at 1:23 PM, Jason Gustafson ja...@confluent.io wrote: Hi

Re: Review Request 36858: Patch for KAFKA-2120

2015-07-30 Thread Jason Gustafson
On July 27, 2015, 10:55 p.m., Jason Gustafson wrote: clients/src/main/java/org/apache/kafka/clients/ClientRequest.java, line 26 https://reviews.apache.org/r/36858/diff/1/?file=1022752#file1022752line26 Should ClientResponse.requestLatencyMs be updated to use sendMs instead

[DISCUSS] Client-side Assignment for New Consumer

2015-08-11 Thread Jason Gustafson
Hi Kafka Devs, One of the nagging issues in the current design of the new consumer has been the need to support a variety of assignment strategies. We've encountered this in particular in the design of copycat and the processing framework (KIP-28). From what I understand, Samza also has a number

Re: Review Request 36858: Patch for KAFKA-2120

2015-08-11 Thread Jason Gustafson
of the box. - Jason Gustafson On Aug. 11, 2015, 2:55 a.m., Mayuresh Gharat wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-13 Thread Jason Gustafson
the risk. Thanks, Jiangjie (Becket) Qin On Thu, Aug 13, 2015 at 12:41 PM, Jason Gustafson ja...@confluent.io wrote: Hey Onur and Jiangjie, I've updated that wiki with a proposal to add regex subscriptions to the consumer metadata. Can you have a look to see if it addresses your

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-13 Thread Jason Gustafson
where there are either a large number of consumers or topics. I wonder if you guys can share your thoughts about your scaling expectations? -Jason On Wed, Aug 12, 2015 at 12:28 PM, Jason Gustafson ja...@confluent.io wrote: Hey Jiangjie, That's a great point. In the worst case (the mirror

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-14 Thread Jason Gustafson
this to solve the topic renaming case but this is a completely different discussion. Thanks, Jiangjie (Becket) Qin On Thu, Aug 13, 2015 at 2:14 PM, Jason Gustafson ja...@confluent.io wrote: Thanks Jiangjie, that information helps. I agree the protocol must

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-14 Thread Jason Gustafson
, Jiangjie (Becket) Qin On Thu, Aug 13, 2015 at 2:14 PM, Jason Gustafson ja...@confluent.io wrote: Thanks Jiangjie, that information helps. I agree the protocol must consider scalability. My point was that the synchronization barrier

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-15 Thread Jason Gustafson
, There is a discussion thread about this to solve the topic renaming case but this is a completely different discussion. Thanks, Jiangjie (Becket) Qin On Thu, Aug 13, 2015 at 2:14 PM, Jason Gustafson ja...@confluent.io wrote

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Jason Gustafson
/#comment146812 I think convention is to leave off the braces on one-line if statements. - Jason Gustafson On July 22, 2015, 6:32 a.m., Ashish Singh wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: [DISCUSS] Partitioning in Kafka

2015-07-27 Thread Jason Gustafson
For a little background, the difference between this partitioner and the default one is that it breaks the deterministic mapping from key to partition. Instead, messages for a given key can end up in either of two partitions. This means that the consumer generally won't see all messages for a

Re: Kafka Consumer thoughts

2015-07-27 Thread Jason Gustafson
I think if we recommend a longer session timeout, then we should expose the heartbeat frequency in configuration since this generally controls how long normal rebalances will take. I think it's currently hard-coded to 3 heartbeats per session timeout. It could also be nice to have an explicit

Re: Review Request 36858: Patch for KAFKA-2120

2015-07-27 Thread Jason Gustafson
/MockClient.java (line 94) https://reviews.apache.org/r/36858/#comment147498 Do we need to set sendMs in this method for consistency? - Jason Gustafson On July 27, 2015, 10:32 p.m., Mayuresh Gharat wrote: --- This is an automatically

Re: Review Request 36871: Patch for KAFKA-2381

2015-07-27 Thread Jason Gustafson
/integration/kafka/api/ConsumerTest.scala (line 220) https://reviews.apache.org/r/36871/#comment147533 Could we catch this issue more directly with a unit test for SubscriptionState? - Jason Gustafson On July 28, 2015, 12:56 a.m., Ashish Singh wrote

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Jason Gustafson
. clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java (line 173) https://reviews.apache.org/r/36681/#comment146940 Why do we need to do this in a separate thread? - Jason Gustafson On July 22, 2015, 11:09 p.m., Ashish Singh wrote

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-23 Thread Jason Gustafson
On July 23, 2015, 7:19 a.m., Ashish Singh wrote: clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java, line 173 https://reviews.apache.org/r/36681/diff/2/?file=1019197#file1019197line173 getAllTopics is a blocking call and we need to send a response

Re: Build failed in Jenkins: KafkaPreCommit #147

2015-07-14 Thread Jason Gustafson
Looks like there is already KAFKA-2055 for the transient ConsumerBounceTest failure. I'll have a look at it. -Jason On Tue, Jul 14, 2015 at 9:02 AM, Guozhang Wang wangg...@gmail.com wrote: I think there is no JIRA ticket created for ConsumerBounceTest yet. We can create a new one for it. On

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-14 Thread Jason Gustafson
/r/36333/#review91646 --- On July 14, 2015, 8:21 p.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36333

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-14 Thread Jason Gustafson
clients/src/test/java/org/apache/kafka/clients/consumer/internals/RequestFutureTest.java PRE-CREATION core/src/test/scala/integration/kafka/api/ConsumerTest.scala 92ffb91b5e039dc0d4cd0e072ca46db32f280cf9 Diff: https://reviews.apache.org/r/36333/diff/ Testing --- Thanks, Jason

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-14 Thread Jason Gustafson
--- Thanks, Jason Gustafson

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-15 Thread Jason Gustafson
, visit: https://reviews.apache.org/r/36333/#review91810 --- On July 15, 2015, 1:21 a.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-18 Thread Jason Gustafson
to be a way to request the metadata we're interested in directly instead. It would involve a change to NetworkClient, but it might be worth looking into, at least to see the level of effort. - Jason Gustafson On July 18, 2015, 4:39 a.m., Ashish Singh wrote

Re: Review Request 36244: Patch for KAFKA-2312

2015-07-13 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36244/#review91488 --- Ship it! LGTM - Jason Gustafson On July 7, 2015, 5 a.m., Tim

Re: Review Request 36341: Patch for KAFKA-2311

2015-07-13 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36341/#review91556 --- Ship it! Ship It! - Jason Gustafson On July 9, 2015, 1:04 a.m

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-13 Thread Jason Gustafson
--- On July 12, 2015, 12:34 a.m., Jason Gustafson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36333

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-13 Thread Jason Gustafson
catch. - Jason --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36333/#review91564 --- On July 12, 2015, 12:34 a.m., Jason Gustafson

Re: Review Request 36333: Patch for KAFKA-2123

2015-07-13 Thread Jason Gustafson
/consumer/internals/RequestFutureTest.java PRE-CREATION core/src/test/scala/integration/kafka/api/ConsumerTest.scala 92ffb91b5e039dc0d4cd0e072ca46db32f280cf9 Diff: https://reviews.apache.org/r/36333/diff/ Testing --- Thanks, Jason Gustafson

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Jason Gustafson
/kafka/clients/consumer/KafkaConsumer.java (lines 1074 - 1077) https://reviews.apache.org/r/36590/#comment146361 Is it an actual problem if we return this topic to the user? - Jason Gustafson On July 20, 2015, 5:44 p.m., Ashish Singh wrote

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Jason Gustafson
On July 19, 2015, 1:11 a.m., Jason Gustafson wrote: Ashish Singh wrote: Jason, thanks for your review! I looked into ConsumerNetworkClient/ NetwrokClient, Metadata and Cluster classes. On receiving metadataUpdate, cluster instance in metadata is updated. However, when a topic

[DISCUSS] KIP-40 - Extend GroupMetadata request to return group and member status

2015-10-25 Thread Jason Gustafson
Hi Kafka Devs, Currently, the new consumer provides no way to view a group's status except by inspecting coordinator and consumer logs. This includes listing the members of the group and their partition assignments. For the old consumer, tools could read this information directly from Zookeeper,

Re: [DISCUSS] KIP-40 - Extend GroupMetadata request to return group and member status

2015-10-29 Thread Jason Gustafson
On Thu, Oct 29, 2015 at 8:37 AM, Neha Narkhede <n...@confluent.io> wrote: > I like Jun's suggestion of adding ListGroups and DescribeGroup. It keeps > the abstractions simple and avoids overloading existing requests. > > On Wed, Oct 28, 2015 at 8:08 PM, Jason Gustafson <ja...@

[VOTE] KIP-40 ListGroups and DescribeGroup

2015-10-29 Thread Jason Gustafson
Since we're crunching a little on the 0.9 release, I'd like to begin voting for KIP-40. If you agree overall, but have minor objections with the schemas, we can discuss them on the JIRA. Summary: Add ListGroups and DescribeGroup APIs for admin tooling to view consumer groups and status. KIP:

Re: [DISCUSS] KIP-40 - Extend GroupMetadata request to return group and member status

2015-10-29 Thread Jason Gustafson
Forgot to include a link to the changed KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-40%3A+ListGroups+and+DescribeGroup -Jason On Thu, Oct 29, 2015 at 9:43 AM, Jason Gustafson <ja...@confluent.io> wrote: > Hi All, > > I've updated the KIP for ListGroups an

Re: [DISCUSS] KIP-40 - Extend GroupMetadata request to return group and member status

2015-10-28 Thread Jason Gustafson
ed, Oct 28, 2015 at 3:37 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > On Wed, Oct 28, 2015 at 10:25 PM, Jason Gustafson <ja...@confluent.io> > > wrote: > > > > > Hey Ashish, > > > > > > Yeah, that's fine with me too. I thought people ki

Re: [DISCUSS] KIP-40 - Extend GroupMetadata request to return group and member status

2015-10-28 Thread Jason Gustafson
n Wed, Oct 28, 2015 at 12:59 PM, Gwen Shapira <g...@confluent.io> wrote: > > > Looks awesome to me :) > > > > This will allow to both list all groups and to retrieve offsets for > > specific groups. > > > > Since 3 days passed with no comments, would you like

Re: [DISCUSS] KIP-40 - Extend GroupMetadata request to return group and member status

2015-10-28 Thread Jason Gustafson
s awesome to me :) > > > > > > This will allow to both list all groups and to retrieve offsets for > > > specific groups. > > > > > > Since 3 days passed with no comments, would you like to start a vote? > > > > > > On Sun, Oct 25, 2015

Re: 0.9.0 release branch

2015-11-02 Thread Jason Gustafson
I added KAFKA-2691 as well, which improves client handling of authorization errors. -Jason On Mon, Nov 2, 2015 at 10:25 AM, Becket Qin wrote: > Hi Jun, > > I added KAFKA-2722 as a blocker for 0.9. It fixes the ISR propagation > scalability issue we saw. > > Thanks, > >

Re: [DISCUSS] KIP-37 - Add namespaces in Kafka

2015-10-14 Thread Jason Gustafson
Hey Ashish, thanks for the write-up. I think having a namespace capability is a useful feature for Kafka, in particular with the addition of the authorization layer. I probably prefer Jay's hierarchical approach if we're going to embed the namespace in the topic name since it seems more general.

  1   2   3   4   5   6   7   8   9   10   >