[jira] [Updated] (KAFKA-2136) Client side protocol changes to return quota delays

2015-04-20 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2136: --- Reviewer: Joel Koshy Client side protocol changes to return quota delays

[jira] [Created] (KAFKA-2136) Client side protocol changes to return quota delays

2015-04-20 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2136: -- Summary: Client side protocol changes to return quota delays Key: KAFKA-2136 URL: https://issues.apache.org/jira/browse/KAFKA-2136 Project: Kafka Issue

Re: Review Request 33378: Patch for KAFKA-2136

2015-04-20 Thread Aditya Auradkar
9186c90de5a983a73b042fcb42987bfabae14fcf core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing (updated) --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-04-20 Thread Aditya Auradkar
/DelayedOperationTest.scala 9186c90de5a983a73b042fcb42987bfabae14fcf core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Review Request 33378: Patch for KAFKA-2136

2015-04-20 Thread Aditya Auradkar
/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- Thanks, Aditya Auradkar

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-16 Thread Aditya Auradkar
such cases clearly for quote actions mixed with ack = -1. Guozhang On Wed, Apr 15, 2015 at 4:23 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Thanks for the review Guozhang. 1. Agreed. 2. This proposal actually waits for the maximum of the 2 timeouts. This reduces

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-15 Thread Aditya Auradkar
immediately, 3) after X ms elapsed, set timeout-error-code and return. Guozhang On Tue, Apr 14, 2015 at 5:01 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: This is an implementation proposal for delaying requests in quotas using the current purgatory. I'll discuss the usage for produce

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-14 Thread Aditya Auradkar
...@gmail.com] Sent: Monday, April 13, 2015 7:27 PM To: dev@kafka.apache.org Subject: Re: [KIP-DISCUSSION] KIP-13 Quotas I think KAFKA-2063 (bounding fetch response) is still under discussion, and may not be got it in time with KAFKA-1927. On Thu, Apr 9, 2015 at 4:49 PM, Aditya Auradkar aaurad

Re: Review Request 33049: Patch for KAFKA-2084

2015-04-10 Thread Aditya Auradkar
69b772c1941865fbe15b34bb2784c511f8ce519a core/src/main/scala/kafka/server/KafkaServer.scala c63f4ba9d622817ea8636d4e6135fba917ce085a core/src/test/scala/unit/kafka/server/ClientQuotaMetricsTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/33049/diff/ Testing --- Thanks, Aditya

Re: Review Request 33049: Patch for KAFKA-2084

2015-04-10 Thread Aditya Auradkar
69b772c1941865fbe15b34bb2784c511f8ce519a core/src/main/scala/kafka/server/KafkaServer.scala c63f4ba9d622817ea8636d4e6135fba917ce085a core/src/test/scala/unit/kafka/server/ClientQuotaMetricsTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/33049/diff/ Testing --- Thanks, Aditya Auradkar

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-08 Thread Aditya Auradkar
, Aditya Auradkar wrote: Hey everyone, Following up after today's hangout. After discussing the client side metrics piece internally, we've incorporated that section into the KIP. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Since there appears to be sufficient consensus, I'm

[jira] [Resolved] (KAFKA-766) Isr shrink/expand check is fragile

2015-04-08 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar resolved KAFKA-766. --- Resolution: Fixed Fixed in KAFKA-1546 Isr shrink/expand check is fragile

RE: [VOTE] KIP-13 Quotas

2015-04-08 Thread Aditya Auradkar
good to me. Thanks, Jun On Tue, Apr 7, 2015 at 10:47 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Please vote. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Thanks, Aditya

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-07 Thread Aditya Auradkar
changes. What changes are you proposing in the clients for monitoring and how would that be used? -Jay On Mon, Apr 6, 2015 at 10:36 AM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Hi Jay, 2. At this time, the proposed response format changes are only

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-07 Thread Aditya Auradkar
really like to understand the details of the use case before we make protocol changes. What changes are you proposing in the clients for monitoring and how would that be used? -Jay On Mon, Apr 6, 2015 at 10:36 AM, Aditya Auradkar aaurad...@linkedin.com.invalid

[VOTE] KIP-13 Quotas

2015-04-07 Thread Aditya Auradkar
Please vote. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Thanks, Aditya

[jira] [Created] (KAFKA-2097) Implement request delays for quota violations

2015-04-06 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2097: -- Summary: Implement request delays for quota violations Key: KAFKA-2097 URL: https://issues.apache.org/jira/browse/KAFKA-2097 Project: Kafka Issue Type

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-06 Thread Aditya Auradkar
a small delay. I haven't worked out the efficiency implications of this choice, so as long as we do that I'm happy. -Jay On Fri, Apr 3, 2015 at 1:10 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Some responses to Jay's points. 1. Using commas - Cool. 2. Adding return flag - I'm

[jira] [Commented] (KAFKA-2087) TopicConfigManager javadoc references incorrect paths

2015-04-04 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14396083#comment-14396083 ] Aditya Auradkar commented on KAFKA-2087: [~jkreps] Yeah, I've discarded that patch

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-03 Thread Aditya Auradkar
: Aditya Auradkar Sent: Friday, April 03, 2015 10:10 AM To: dev@kafka.apache.org Subject: RE: [KIP-DISCUSSION] KIP-13 Quotas Thanks Jun. Some thoughts: 10) I think it is better we throttle regardless of the produce/fetch version. This is a nice feature where clients can tell if they are being throttled

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-03 Thread Aditya Auradkar
On Wed, Apr 1, 2015 at 3:27 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Hey everyone, I've made changes to the KIP to capture our discussions over the last couple of weeks. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas I'll start a voting thread after people

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-03 Thread Aditya Auradkar
to enforce quotas. That may be simpler, but would be incur a slight perf penalty if too many clients are being throttled. Thanks, Joel -Jay On Fri, Apr 3, 2015 at 10:45 AM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Update, I added a proposal on doing dynamic client based

[jira] [Created] (KAFKA-2087) TopicConfigManager javadoc references incorrect paths

2015-04-02 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2087: -- Summary: TopicConfigManager javadoc references incorrect paths Key: KAFKA-2087 URL: https://issues.apache.org/jira/browse/KAFKA-2087 Project: Kafka

Review Request 32778: Patch for KAFKA-2087

2015-04-02 Thread Aditya Auradkar
/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/32778/diff/ Testing --- Thanks, Aditya Auradkar

Review Request 32781: Patch for KAFKA-2087

2015-04-02 Thread Aditya Auradkar
/browse/KAFKA-2087 Repository: kafka Description --- Fixing KAFKA-2087 Diffs - core/src/main/scala/kafka/server/TopicConfigManager.scala 47295d4 Diff: https://reviews.apache.org/r/32781/diff/ Testing --- Thanks, Aditya Auradkar

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-01 Thread Aditya Auradkar
, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Hey everyone, Thanks for the great discussion. There are currently a few points on this KIP that need addressing and I want to make sure we are on the same page about those

[jira] [Created] (KAFKA-2083) Quotas for Kafka

2015-04-01 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2083: -- Summary: Quotas for Kafka Key: KAFKA-2083 URL: https://issues.apache.org/jira/browse/KAFKA-2083 Project: Kafka Issue Type: New Feature

[jira] [Created] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

2015-04-01 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2084: -- Summary: byte rate metrics per client ID (producer and consumer) Key: KAFKA-2084 URL: https://issues.apache.org/jira/browse/KAFKA-2084 Project: Kafka

[jira] [Created] (KAFKA-2085) Return delay time in QuotaViolationException

2015-04-01 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2085: -- Summary: Return delay time in QuotaViolationException Key: KAFKA-2085 URL: https://issues.apache.org/jira/browse/KAFKA-2085 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-1546) Automate replica lag tuning

2015-03-31 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1546: --- Attachment: documentation.diff Automate replica lag tuning

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-31 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389441#comment-14389441 ] Aditya Auradkar commented on KAFKA-1546: Good point Jun. I've added those metrics

RE: Metrics package discussion

2015-03-30 Thread Aditya Auradkar
to do this anyway since the new clients are on KM metrics. Thanks, Joel On Fri, Mar 27, 2015 at 06:48:48PM +, Aditya Auradkar wrote: Adding to what Jay said. The library maintains 1k samples by default. The UniformSample has a long array so about 8k overhead per

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-27 Thread Aditya Auradkar
: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

RE: Metrics package discussion

2015-03-27 Thread Aditya Auradkar
these issues, and ripped them out. Whether they are really useful or not, I don't know. Averages can be pretty misleading so it can be nice but I don't know that it is critical. -Jay On Thu, Mar 26, 2015 at 1:58 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: From what I can tell

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384734#comment-14384734 ] Aditya Auradkar commented on KAFKA-1546: [~junrao] I found references

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384620#comment-14384620 ] Aditya Auradkar commented on KAFKA-1546: [~jjkoshy] Where can I find the html

[jira] [Updated] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1546: --- Attachment: documentation.diff Automate replica lag tuning

Re: Review Request 31366: Patch for KAFKA-1461

2015-03-27 Thread Aditya Auradkar
/AbstractFetcherThread.scala https://reviews.apache.org/r/31366/#comment126602 Can you add some documentation for this class? - Aditya Auradkar On March 27, 2015, 10:31 p.m., Sriharsha Chintalapani wrote: --- This is an automatically

[jira] [Updated] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1546: --- Status: Patch Available (was: Open) Automate replica lag tuning

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384759#comment-14384759 ] Aditya Auradkar commented on KAFKA-1546: Added a patch for the HTML changes

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-26 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31967/#review77799 --- On March 25, 2015, 8:27 p.m., Aditya Auradkar wrote

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-26 Thread Aditya Auradkar
/31967/diff/ Testing --- Thanks, Aditya Auradkar

RE: Metrics package discussion

2015-03-26 Thread Aditya Auradkar
/GC performance. IIRC, when I was looking into codahale at LinkedIn, I remember it having quite a few memory management and GC issues while using histograms. In comparison, histograms in the new metrics package aren't very well tested. Thanks, Neha On Wed, Mar 25, 2015 at 8:25 AM, Aditya

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-26 Thread Aditya Auradkar
/#review77810 --- On March 25, 2015, 8:27 p.m., Aditya Auradkar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31967

Metrics package discussion

2015-03-25 Thread Aditya Auradkar
Hey everyone, Picking up this discussion after yesterdays KIP hangout. For anyone who did not join the meeting, we have 2 different metrics packages being used by the clients (custom package) and the server (codahale). We are discussing whether to migrate the server to the new package. What

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-25 Thread Aditya Auradkar
core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-19 Thread Aditya Auradkar
that this KIP doesn't need to block on the migration of the metrics package. Thanks, Jun On Wed, Mar 18, 2015 at 4:02 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Hey everyone, Thanks for the great discussion. There are currently a few points on this KIP that need addressing and I want

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-19 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370193#comment-14370193 ] Aditya Auradkar commented on KAFKA-1546: [~nehanarkhede][~junrao] I've

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Aditya Auradkar
. Thanks, Jun On Wed, Mar 4, 2015 at 3:51 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Posted a KIP for quotas in kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Appreciate any

RE: [DISCUSS] KIP-5 - Broker Configuration Management

2015-03-18 Thread Aditya Auradkar
Hi Andrii, Thanks for the writeup. IMO, we should be able to support dynamically changing certain configs. For example: the Quota proposal relies on such a mechanism for changing quotas on the fly. Aditya From: Andrii Biletskyi

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Aditya Auradkar
, 2015 at 3:51 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Posted a KIP for quotas in kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Appreciate any feedback. Aditya

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-17 Thread Aditya Auradkar
, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-17 Thread Aditya Auradkar
://reviews.apache.org/r/31967/#comment124449 Renamed. We can use the raw offset directly, but in general the code is passing the LogOffsetMetadata objects and it made sense to remain consistent. - Aditya Auradkar On March 16, 2015, 6:32 p.m., Aditya Auradkar wrote

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-16 Thread Aditya Auradkar
PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Posted a KIP for quotas in kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Appreciate any feedback. Aditya

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
://reviews.apache.org/r/31967/#comment124158 Interesting point. I thought that it would be enough to simply check the lag value. But yes, this will cause the HW to be inconsistent. - Aditya Auradkar On March 12, 2015, 8:42 p.m., Aditya Auradkar wrote

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

RE: [VOTE] KIP-16: Replica Lag Tuning

2015-03-15 Thread Aditya Auradkar
on the approach. Much better than before. Thanks, Jun On Fri, Mar 13, 2015 at 9:54 AM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Details in the KIP, Jira and RB. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning https://issues.apache.org/jira

RE: [VOTE] KIP-16: Replica Lag Tuning

2015-03-15 Thread Aditya Auradkar
Rather, 6 binding +1's. Aditya From: Aditya Auradkar Sent: Sunday, March 15, 2015 10:39 PM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-16: Replica Lag Tuning Thanks. We have 5 binding +1s. I'll update the KIP with your comments and mark as accepted

Re: Review Request 32063: Patch for KAFKA-2019

2015-03-13 Thread Aditya Auradkar
? - Aditya Auradkar On March 14, 2015, 12:13 a.m., Joseph Holsten wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32063

[VOTE] KIP-16: Replica Lag Tuning

2015-03-13 Thread Aditya Auradkar
Details in the KIP, Jira and RB. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning https://issues.apache.org/jira/browse/KAFKA-1546 https://reviews.apache.org/r/31967/ Aditya

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-12 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359925#comment-14359925 ] Aditya Auradkar commented on KAFKA-1546: I ran a bunch of tests on my patch

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-12 Thread Aditya Auradkar
doing a good job of the windowed metrics, but we'll want to integrate the backoff calculation with that algorithm (assuming that is what we are using). Cheers, -Jay On Wed, Mar 4, 2015 at 3:51 PM, Aditya Auradkar aaurad

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-12 Thread Aditya Auradkar
c124c8df5b5079e5ffbd0c4ea359562a66aaf317 core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya

RE: [DISCUSS] KIP 16 - Replica lag tuning

2015-03-12 Thread Aditya Auradkar
- - - - - - - - - - - - - - - - - On Thu, Mar 12, 2015 at 4:36 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: I wrote a KIP for this after some discussion on KAFKA-1546. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning The RB is here: https://reviews.apache.org/r/31967

[DISCUSS] KIP 16 - Replica lag tuning

2015-03-12 Thread Aditya Auradkar
I wrote a KIP for this after some discussion on KAFKA-1546. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning The RB is here: https://reviews.apache.org/r/31967/ Thanks, Aditya

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-12 Thread Aditya Auradkar
/ Testing --- Thanks, Aditya Auradkar

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-12 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359371#comment-14359371 ] Aditya Auradkar commented on KAFKA-1546: [~jkreps]I'm going to test it like you

Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-11 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358074#comment-14358074 ] Aditya Auradkar commented on KAFKA-1546: I'll write a short KIP

[jira] [Updated] (KAFKA-1930) Move server over to new metrics library

2015-03-11 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1930: --- Assignee: Aditya Auradkar Move server over to new metrics library

[jira] [Commented] (KAFKA-1930) Move server over to new metrics library

2015-03-11 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358077#comment-14358077 ] Aditya Auradkar commented on KAFKA-1930: I plan to work on this ticket since

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
c124c8df5b5079e5ffbd0c4ea359562a66aaf317 core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-09 Thread Aditya Auradkar
to integrate the backoff calculation with that algorithm (assuming that is what we are using). Cheers, -Jay On Wed, Mar 4, 2015 at 3:51 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Posted a KIP for quotas in kafka. https://cwiki.apache.org/confluence/display/KAFKA

RE: [DISCUSS] KIP-6 - New reassignment partition logic for re-balancing

2015-03-05 Thread Aditya Auradkar
Thanks for the writeup and RB Joe/Dmitry. Is it possible to describe the proposed partition reassignment algorithm in more detail on the KIP? In fact, it would be really easy to understand if we had some concrete examples comparing partition assignment with the old algorithm and the new.

[KIP-DISCUSSION] KIP-13 Quotas

2015-03-04 Thread Aditya Auradkar
Posted a KIP for quotas in kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Appreciate any feedback. Aditya

RE: Announcing the Confluent Platform built on Apache Kafka

2015-02-25 Thread Aditya Auradkar
Congrats! From: Andrew Otto [ao...@wikimedia.org] Sent: Wednesday, February 25, 2015 12:06 PM To: us...@kafka.apache.org Cc: dev@kafka.apache.org Subject: Re: Announcing the Confluent Platform built on Apache Kafka Wow, .deb packages. I love you. On

[jira] [Updated] (KAFKA-1986) Producer request failure rate should not include InvalidMessageSizeException and OffsetOutOfRangeException

2015-02-25 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1986: --- Status: Patch Available (was: Open) Producer request failure rate should not include

Review Request 31449: Patch for KAFKA-1986

2015-02-25 Thread Aditya Auradkar
/browse/KAFKA-1986 Repository: kafka Description --- Patch for KAFKA-1986 Diffs - core/src/main/scala/kafka/server/ReplicaManager.scala 586cf4caa95f58d5b2e6c7429732d25d2b3635c8 Diff: https://reviews.apache.org/r/31449/diff/ Testing --- Thanks, Aditya Auradkar

[jira] [Created] (KAFKA-1986) Producer request failure rate should not include InvalidMessageSizeException and OffsetOutOfRangeException

2015-02-25 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-1986: -- Summary: Producer request failure rate should not include InvalidMessageSizeException and OffsetOutOfRangeException Key: KAFKA-1986 URL: https://issues.apache.org/jira/browse

Re: Review Request 30196: Patch for KAFKA-1886

2015-02-23 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30196/#review71556 --- On Feb. 2, 2015, 9:57 p.m., Aditya Auradkar wrote: --- This is an automatically

[jira] [Commented] (KAFKA-1974) NPE in SelectorTest on trunk

2015-02-23 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14333735#comment-14333735 ] Aditya Auradkar commented on KAFKA-1974: [~nehanarkhede] I agree that a null check

[jira] [Commented] (KAFKA-1974) NPE in SelectorTest on trunk

2015-02-20 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330026#comment-14330026 ] Aditya Auradkar commented on KAFKA-1974: Neha, do you have the entire stack trace

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-02-19 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328260#comment-14328260 ] Aditya Auradkar commented on KAFKA-1546: I do have a concern about the heuristic

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-02-19 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328427#comment-14328427 ] Aditya Auradkar commented on KAFKA-1546: I agree we should model this in terms

[jira] [Comment Edited] (KAFKA-1546) Automate replica lag tuning

2015-02-19 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328427#comment-14328427 ] Aditya Auradkar edited comment on KAFKA-1546 at 2/20/15 1:30 AM

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
/SimpleFetchTest.scala 525c835803b048c952667567cd205b4b06824391 Diff: https://reviews.apache.org/r/31168/diff/ Testing (updated) --- All unit tests now pass. BUILD SUCCESSFUL Total time: 9 mins 35.296 secs Thanks, Aditya Auradkar

[jira] [Assigned] (KAFKA-1546) Automate replica lag tuning

2015-02-18 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar reassigned KAFKA-1546: -- Assignee: Aditya Auradkar (was: nicu marasoiu) Automate replica lag tuning

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-02-18 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326773#comment-14326773 ] Aditya Auradkar commented on KAFKA-1546: [~nmarasoi] I'm going to assign

Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
525c835803b048c952667567cd205b4b06824391 Diff: https://reviews.apache.org/r/31168/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
/SimpleFetchTest.scala 525c835803b048c952667567cd205b4b06824391 Diff: https://reviews.apache.org/r/31168/diff/ Testing (updated) --- All unit tests now pass. Thanks, Aditya Auradkar

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30570/#review72817 --- On Feb. 17, 2015, 11:46 p.m., Aditya Auradkar wrote

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Aditya Auradkar
/ Testing --- I've added asserts to the SimpleFetchTest to count the number of fetch requests. I'm going to file an additional jira to add unit tests for all the BrokerTopicMetrics updated via ReplicaManager Thanks, Aditya Auradkar

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Aditya Auradkar
://reviews.apache.org/r/30570/#review72845 --- On Feb. 17, 2015, 11:46 p.m., Aditya Auradkar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r

[jira] [Updated] (KAFKA-1943) Producer request failure rate should not include MessageSetSizeTooLarge and MessageSizeTooLargeException

2015-02-11 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1943: --- Issue Type: Sub-task (was: Bug) Parent: KAFKA-1946 Producer request failure rate

[jira] [Created] (KAFKA-1946) Improve BrokerTopicMetrics reporting

2015-02-11 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-1946: -- Summary: Improve BrokerTopicMetrics reporting Key: KAFKA-1946 URL: https://issues.apache.org/jira/browse/KAFKA-1946 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-1914) Count TotalProduceRequestRate and TotalFetchRequestRate in BrokerTopicMetrics

2015-02-11 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1914: --- Issue Type: Sub-task (was: Bug) Parent: KAFKA-1946 Count TotalProduceRequestRate

[jira] [Updated] (KAFKA-1920) Add a metric to count client side errors in BrokerTopicMetrics

2015-02-11 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1920: --- Issue Type: Sub-task (was: Improvement) Parent: KAFKA-1946 Add a metric to count

<    1   2   3   4   5   >