date:20150824


[ 
https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709609#comment-14709609
 ] 

Jay Kreps commented on KAFKA-2389:
--

I think this may be the first time Ewen has ever agreed with me. :-)

 CommitType seems not necessary in commit().
 ---

 Key: KAFKA-2389
 URL: https://issues.apache.org/jira/browse/KAFKA-2389
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Jiangjie Qin

 The CommitType does not seem to be necessary in for commit(), it can be 
 inferred from whether user passed in a callback or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

2015-08-24 Thread Jiangjie Qin (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709620#comment-14709620
 ] 

Jiangjie Qin commented on KAFKA-2389:
-

I am also thinking about having only no parameters  map + callback. Passing in 
a null map is less common in API but it does make the API cleaner. I'll follow 
this approach and submit another patch. And also glad Jay and Ewen agreed for 
the first time :P

 CommitType seems not necessary in commit().
 ---

 Key: KAFKA-2389
 URL: https://issues.apache.org/jira/browse/KAFKA-2389
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Jiangjie Qin

 The CommitType does not seem to be necessary in for commit(), it can be 
 inferred from whether user passed in a callback or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

2015-08-24 Thread Jason Gustafson (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709631#comment-14709631
 ] 

Jason Gustafson commented on KAFKA-2389:


[~ewencp] If I understand correctly, you are proposing to drop 
commitAsync(callback). Is that right? But I can imagine the user wanting to 
commit the current offsets (whatever they are), and getting some notification 
for when it actually happens (and whether it failed). I would probably support 
all three variants: no parameters, callback, map + callback.

 CommitType seems not necessary in commit().
 ---

 Key: KAFKA-2389
 URL: https://issues.apache.org/jira/browse/KAFKA-2389
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Jiangjie Qin

 The CommitType does not seem to be necessary in for commit(), it can be 
 inferred from whether user passed in a callback or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-24 Thread Aditya Auradkar



 On Aug. 22, 2015, 12:45 a.m., Joel Koshy wrote:
  clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java, 
  line 107
  https://reviews.apache.org/r/33378/diff/12-13/?file=1043780#file1043780line107
 
  This will probably need a versionId as well (as is done in the Scala 
  response) - i.e., when we move the broker over to use these protocol 
  schemas.

Makes sense. Do you want me to tackle this in this patch or should it be 
tackled in the patch that migrates the broker to use these schemas?


- Aditya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96100
---


On Aug. 24, 2015, 5:33 p.m., Aditya Auradkar wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/33378/
 ---
 
 (Updated Aug. 24, 2015, 5:33 p.m.)
 
 
 Review request for kafka, Joel Koshy and Jun Rao.
 
 
 Bugs: KAFKA-2136
 https://issues.apache.org/jira/browse/KAFKA-2136
 
 
 Repository: kafka
 
 
 Description
 ---
 
 Changes are
 - Addressing Joel's comments
 - protocol changes to the fetch request and response to return the 
 throttle_time_ms to clients
 - New producer/consumer metrics to expose the avg and max delay time for a 
 client
 - Test cases.
 - Addressed Joel and Juns comments
 
 
 Diffs
 -
 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java
  9dc669728e6b052f5c6686fcf1b5696a50538ab4 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 0baf16e55046a2f49f6431e01d52c323c95eddf0 
   clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 
 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f 
   clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 
 df073a0e76cc5cc731861b9604d0e19a928970e0 
   clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java 
 eb8951fba48c335095cc43fc3672de1c733e07ff 
   clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 
 715504b32950666e9aa5a260fa99d5f897b2007a 
   clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 
 febfc70dabc23671fd8a85cf5c5b274dff1e10fb 
   
 clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java
  a7c83cac47d41138d47d7590a3787432d675c1b0 
   
 clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java
  8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4 
   
 clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java
  8b2aca85fa738180e5420985fddc39a4bf9681ea 
   core/src/main/scala/kafka/api/FetchRequest.scala 
 5b38f8554898e54800abd65a7415dd0ac41fd958 
   core/src/main/scala/kafka/api/FetchResponse.scala 
 b9efec2efbd3ea0ee12b911f453c47e66ad34894 
   core/src/main/scala/kafka/api/ProducerRequest.scala 
 c866180d3680da03e48d374415f10220f6ca68c4 
   core/src/main/scala/kafka/api/ProducerResponse.scala 
 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330 
   core/src/main/scala/kafka/consumer/SimpleConsumer.scala 
 7ebc0405d1f309bed9943e7116051d1d8276f200 
   core/src/main/scala/kafka/server/AbstractFetcherThread.scala 
 f84306143c43049e3aa44e42beaffe7eb2783163 
   core/src/main/scala/kafka/server/ClientQuotaManager.scala 
 9f8473f1c64d10c04cf8cc91967688e29e54ae2e 
   core/src/main/scala/kafka/server/KafkaApis.scala 
 67f0cad802f901f255825aa2158545d7f5e7cc3d 
   core/src/main/scala/kafka/server/ReplicaFetcherThread.scala 
 fae22d2af8daccd528ac24614290f46ae8f6c797 
   core/src/main/scala/kafka/server/ReplicaManager.scala 
 d829e180c3943a90861a12ec184f9b4e4bbafe7d 
   core/src/main/scala/kafka/server/ThrottledResponse.scala 
 1f80d5480ccf7c411a02dd90296a7046ede0fae2 
   core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 
 b4c2a228c3c9872e5817ac58d3022e4903e317b7 
   core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 
 caf98e8f2e09d39ab8234b9f4b9478686865e908 
   core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 
 c4b5803917e700965677d53624f1460c1a52bf52 
 
 Diff: https://reviews.apache.org/r/33378/diff/
 
 
 Testing
 ---
 
 New tests added
 
 
 Thanks,
 
 Aditya Auradkar

Re: KAFKA-2364 migrate docs from SVN to git

2015-08-24 Thread Manikumar Reddy

Hi,

Infra team created git repo for kafka site docs.

Gwen/Guozhang,
Need your help to create a branch asf-site and copy the exiting
svn contents to that branch.

git repo: https://git-wip-us.apache.org/repos/asf/kafka-site.git

https://issues.apache.org/jira/browse/INFRA-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709630#comment-14709630

Kumar

On Fri, Aug 21, 2015 at 6:16 PM, Ismael Juma ism...@juma.me.uk wrote:

My preference would be to do `2` because it reduces the number of tools we
need to know. If we want to clone the repo for the generated site, we can
use the same tools as we do for the code repo and we can watch for changes
on GitHub, etc.

Ismael

On Fri, Aug 21, 2015 at 1:34 PM, Manikumar Reddy ku...@nmsworks.co.in
wrote:

Hi All,

Can we finalize the approach? So that we can proceed further.

1. Gwen's suggestion + existing svn repo
2. Gwen's suggestion + new git repo for docs

kumar

On Thu, Aug 20, 2015 at 11:48 PM, Manikumar Reddy ku...@nmsworks.co.in
wrote:

Also can we migrate svn repo to git repo? This will help us to fix
occasional doc changes/bug fixes through github PR.

On Thu, Aug 20, 2015 at 4:04 AM, Guozhang Wang wangg...@gmail.com
wrote:

Gwen: I remembered it wrong. We would not need another round of
voting.

On Wed, Aug 19, 2015 at 3:06 PM, Gwen Shapira g...@confluent.io
wrote:

Looking back at this thread, the +1 mention same repo, so I'm not
sure a
new vote is required.

On Wed, Aug 19, 2015 at 3:00 PM, Guozhang Wang wangg...@gmail.com
wrote:

So I think we have two different approaches here. The original
proposal
from Aseem is to move website from SVN to a separate Git repo, and
hence
have separate commits on code / doc changes. For that we have
accumulated
enough binging +1s to move on; Gwen's proposal is to move website
into
the
same repo under a different folder. If people feel they prefer
this
over
the previous approach I would like to call for another round of
voting.

Guozhang

On Wed, Aug 19, 2015 at 10:24 AM, Ashish paliwalash...@gmail.com

wrote:

+1 to what Gwen has suggested. This is what we follow in Flume.

All the latest doc changes are in git, once ready you move
changes
to
svn to update website.
The only catch is, when you need to update specific changes to
website
outside release cycle, need to be a bit careful :)

On Wed, Aug 19, 2015 at 9:06 AM, Gwen Shapira
g...@confluent.io
wrote:
Yeah, so the way this works in few other projects I worked on
is:

* The code repo has a /docs directory with the latest revision
of
the
docs
(not multiple versions, just one that matches the latest state
of
code)
* When you submit a patch that requires doc modification, you
modify
all
relevant files in same patch and they get reviewed and
committed
together
(ideally)
* When we release, we copy the docs matching the release and
commit
to
SVN
website. We also do this occasionally to fix bugs in earlier
docs.
* Release artifacts include a copy of the docs

Nice to have:
* Docs are in Asciidoc and build generates the HTML. Asciidoc
is
easier
to
edit and review.

I suggest a similar process for Kafka.

On Wed, Aug 19, 2015 at 8:53 AM, Ismael Juma
ism...@juma.me.uk

wrote:

I should clarify: it's not possible unless we add an
additional
step
that
moves the docs from the code repo to the website repo.

Ismael

On Wed, Aug 19, 2015 at 4:42 PM, Ismael Juma
ism...@juma.me.uk
wrote:

Hi all,

It looks like it's not feasible to update the code and
website
in
the
same
commit given existing limitations of the Apache infra:

https://issues.apache.org/jira/browse/INFRA-10143?focusedCommentId=14703175page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14703175

Best,
Ismael

On Wed, Aug 12, 2015 at 10:00 AM, Ismael Juma
ism...@juma.me.uk
wrote:

Hi Gwen,

I filed KAFKA-2425 as KAFKA-2364 is about improving the
website
documentation. Aseem Bansal seemed interested in helping
us
with
the
move
so I pinged him in the issue.

Best,
Ismael

On Wed, Aug 12, 2015 at 1:51 AM, Gwen Shapira
g...@confluent.io

wrote:

Ah, there is already a JIRA in the title. Never mind :)

On Tue, Aug 11, 2015 at 5:51 PM, Gwen Shapira
g...@confluent.io

[jira] [Commented] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings

[
https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709685#comment-14709685
]

Edward Ribeiro commented on KAFKA-2338:
---

Updated reviewboard https://reviews.apache.org/r/36578/diff/
against branch origin/trunk

Warn users if they change max.message.bytes that they also need to update
broker and consumer settings
--

Key: KAFKA-2338
URL: https://issues.apache.org/jira/browse/KAFKA-2338
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8.2.1
Reporter: Ewen Cheslack-Postava
Assignee: Edward Ribeiro
Fix For: 0.8.3

Attachments: KAFKA-2338.patch, KAFKA-2338_2015-07-18_00:37:31.patch,
KAFKA-2338_2015-07-21_13:21:19.patch, KAFKA-2338_2015-08-24_14:32:38.patch

We already have KAFKA-1756 filed to more completely address this issue, but
it is waiting for some other major changes to configs to completely protect
users from this problem.
This JIRA should address the low hanging fruit to at least warn users of the
potential problems. Currently the only warning is in our documentation.
1. Generate a warning in the kafka-topics.sh tool when they change this
setting on a topic to be larger than the default. This needs to be very
obvious in the output.
2. Currently, the broker's replica fetcher isn't logging any useful error
messages when replication can't succeed because a message size is too large.
Logging an error here would allow users that get into a bad state to find out
why it is happening more easily. (Consumers should already be logging a
useful error message.)

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings

[
https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edward Ribeiro updated KAFKA-2338:
--
Attachment: KAFKA-2338_2015-08-24_14:32:38.patch

Warn users if they change max.message.bytes that they also need to update
broker and consumer settings
--

Attachments: KAFKA-2338.patch, KAFKA-2338_2015-07-18_00:37:31.patch,
KAFKA-2338_2015-07-21_13:21:19.patch, KAFKA-2338_2015-08-24_14:32:38.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 36578: Patch for KAFKA-2338

2015-08-24 Thread Edward Ribeiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36578/
---

(Updated Aug. 24, 2015, 5:36 p.m.)


Review request for kafka.


Bugs: KAFKA-2338
https://issues.apache.org/jira/browse/KAFKA-2338


Repository: kafka


Description
---

KAFKA-2338 Warn users if they change max.message.bytes that they also need to 
update broker and consumer settings


Diffs (updated)
-

  core/src/main/scala/kafka/admin/TopicCommand.scala 
f1405a5b2961bc826caa22507db8ba39ce1cf4d3 
  core/src/main/scala/kafka/server/AbstractFetcherThread.scala 
f84306143c43049e3aa44e42beaffe7eb2783163 

Diff: https://reviews.apache.org/r/36578/diff/


Testing
---


Thanks,

Edward Ribeiro

[jira] [Commented] (KAFKA-2390) Seek() should take a callback.


[ 
https://issues.apache.org/jira/browse/KAFKA-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709607#comment-14709607
 ] 

Jay Kreps commented on KAFKA-2390:
--

Thanks for the clear problem statement, I agree with the problem. I thought we 
already discussed the fix for this and agreed on just adding the partition(s) 
and offset(s) that are out of range to the existing OffsetOutOfRangeException?

The key point is that it is actually quite ambiguous whether the seek caused 
the out of range fetch or not. The seek may actually be valid at seek time, but 
become invalid later when the fetch occurs. I agree that you need to know you 
are out of range but figuring out whether this was caused by the seek or not 
is actually not necessarily doable and I don't think actually helps.

I propose the fix as just taking the fetch offset we already have per-partition 
and including this with the partition in the exception that is thrown. Thoughts?

 Seek() should take a callback.
 --

 Key: KAFKA-2390
 URL: https://issues.apache.org/jira/browse/KAFKA-2390
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Dong Lin

 Currently seek is an async call. To have the same interface as other calls 
 like commit(), seek() should take a callback. This callback will be invoked 
 if the position to seek triggers OFFSET_OUT_OF_RANGE exception from broker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2136) Client side protocol changes to return quota delays

2015-08-24 Thread Aditya A Auradkar (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aditya A Auradkar updated KAFKA-2136:
-
Attachment: KAFKA-2136_2015-08-24_10:33:10.patch

 Client side protocol changes to return quota delays
 ---

 Key: KAFKA-2136
 URL: https://issues.apache.org/jira/browse/KAFKA-2136
 Project: Kafka
  Issue Type: Sub-task
Reporter: Aditya Auradkar
Assignee: Aditya Auradkar
  Labels: quotas
 Attachments: KAFKA-2136.patch, KAFKA-2136_2015-05-06_18:32:48.patch, 
 KAFKA-2136_2015-05-06_18:35:54.patch, KAFKA-2136_2015-05-11_14:50:56.patch, 
 KAFKA-2136_2015-05-12_14:40:44.patch, KAFKA-2136_2015-06-09_10:07:13.patch, 
 KAFKA-2136_2015-06-09_10:10:25.patch, KAFKA-2136_2015-06-30_19:43:55.patch, 
 KAFKA-2136_2015-07-13_13:34:03.patch, KAFKA-2136_2015-08-18_13:19:57.patch, 
 KAFKA-2136_2015-08-18_13:24:00.patch, KAFKA-2136_2015-08-21_16:29:17.patch, 
 KAFKA-2136_2015-08-24_10:33:10.patch


 As described in KIP-13, evolve the protocol to return a throttle_time_ms in 
 the Fetch and the ProduceResponse objects. Add client side metrics on the new 
 producer and consumer to expose the delay time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2136) Client side protocol changes to return quota delays

2015-08-24 Thread Aditya A Auradkar (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709677#comment-14709677
 ] 

Aditya A Auradkar commented on KAFKA-2136:
--

Updated reviewboard https://reviews.apache.org/r/33378/diff/
 against branch origin/trunk

 Client side protocol changes to return quota delays
 ---

 Key: KAFKA-2136
 URL: https://issues.apache.org/jira/browse/KAFKA-2136
 Project: Kafka
  Issue Type: Sub-task
Reporter: Aditya Auradkar
Assignee: Aditya Auradkar
  Labels: quotas
 Attachments: KAFKA-2136.patch, KAFKA-2136_2015-05-06_18:32:48.patch, 
 KAFKA-2136_2015-05-06_18:35:54.patch, KAFKA-2136_2015-05-11_14:50:56.patch, 
 KAFKA-2136_2015-05-12_14:40:44.patch, KAFKA-2136_2015-06-09_10:07:13.patch, 
 KAFKA-2136_2015-06-09_10:10:25.patch, KAFKA-2136_2015-06-30_19:43:55.patch, 
 KAFKA-2136_2015-07-13_13:34:03.patch, KAFKA-2136_2015-08-18_13:19:57.patch, 
 KAFKA-2136_2015-08-18_13:24:00.patch, KAFKA-2136_2015-08-21_16:29:17.patch, 
 KAFKA-2136_2015-08-24_10:33:10.patch


 As described in KIP-13, evolve the protocol to return a throttle_time_ms in 
 the Fetch and the ProduceResponse objects. Add client side metrics on the new 
 producer and consumer to expose the delay time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-24 Thread Aditya Auradkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
---

(Updated Aug. 24, 2015, 5:33 p.m.)


Review request for kafka, Joel Koshy and Jun Rao.


Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136


Repository: kafka


Description (updated)
---

Changes are
- Addressing Joel's comments
- protocol changes to the fetch request and response to return the 
throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a 
client
- Test cases.
- Addressed Joel and Juns comments


Diffs
-

  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 
9dc669728e6b052f5c6686fcf1b5696a50538ab4 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
0baf16e55046a2f49f6431e01d52c323c95eddf0 
  clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 
3dc8b015afd2347a41c9a9dbc02b8e367da5f75f 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 
df073a0e76cc5cc731861b9604d0e19a928970e0 
  clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java 
eb8951fba48c335095cc43fc3672de1c733e07ff 
  clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 
715504b32950666e9aa5a260fa99d5f897b2007a 
  clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 
febfc70dabc23671fd8a85cf5c5b274dff1e10fb 
  
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java
 a7c83cac47d41138d47d7590a3787432d675c1b0 
  
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java
 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4 
  
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 
8b2aca85fa738180e5420985fddc39a4bf9681ea 
  core/src/main/scala/kafka/api/FetchRequest.scala 
5b38f8554898e54800abd65a7415dd0ac41fd958 
  core/src/main/scala/kafka/api/FetchResponse.scala 
b9efec2efbd3ea0ee12b911f453c47e66ad34894 
  core/src/main/scala/kafka/api/ProducerRequest.scala 
c866180d3680da03e48d374415f10220f6ca68c4 
  core/src/main/scala/kafka/api/ProducerResponse.scala 
5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330 
  core/src/main/scala/kafka/consumer/SimpleConsumer.scala 
7ebc0405d1f309bed9943e7116051d1d8276f200 
  core/src/main/scala/kafka/server/AbstractFetcherThread.scala 
f84306143c43049e3aa44e42beaffe7eb2783163 
  core/src/main/scala/kafka/server/ClientQuotaManager.scala 
9f8473f1c64d10c04cf8cc91967688e29e54ae2e 
  core/src/main/scala/kafka/server/KafkaApis.scala 
67f0cad802f901f255825aa2158545d7f5e7cc3d 
  core/src/main/scala/kafka/server/ReplicaFetcherThread.scala 
fae22d2af8daccd528ac24614290f46ae8f6c797 
  core/src/main/scala/kafka/server/ReplicaManager.scala 
d829e180c3943a90861a12ec184f9b4e4bbafe7d 
  core/src/main/scala/kafka/server/ThrottledResponse.scala 
1f80d5480ccf7c411a02dd90296a7046ede0fae2 
  core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 
b4c2a228c3c9872e5817ac58d3022e4903e317b7 
  core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 
caf98e8f2e09d39ab8234b9f4b9478686865e908 
  core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 
c4b5803917e700965677d53624f1460c1a52bf52 

Diff: https://reviews.apache.org/r/33378/diff/


Testing
---

New tests added


Thanks,

Aditya Auradkar

[jira] [Updated] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings

[
https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edward Ribeiro updated KAFKA-2338:
--
Status: Patch Available (was: In Progress)

Warn users if they change max.message.bytes that they also need to update
broker and consumer settings
--

Attachments: KAFKA-2338.patch, KAFKA-2338_2015-07-18_00:37:31.patch,
KAFKA-2338_2015-07-21_13:21:19.patch, KAFKA-2338_2015-08-24_14:32:38.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings


[ 
https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709707#comment-14709707
 ] 

Edward Ribeiro commented on KAFKA-2338:
---

Hi, [~gwenshap], thanks for the kind words. :) I am sorry for not being able to 
give the necessary love to this patch :( (much because of my inexperience with 
the code base, I guess). I hope I can dig more about max message size problems 
soon tough. I have just rebased the patch and it compiles successfully now with 
latest trunk.

Oh, one thing that has caught my attention is that some chunk of code (below) 
was removed from TopicCommand, specifically in the alterTopic() method, in the 
context of KAFKA-2198 (a7e0ac):. Seems to indicate that now topic configuration 
cannot be altered, right?

{code}
  val configs = AdminUtils.fetchTopicConfig(zkClient, topic)
  if(opts.options.has(opts.configOpt) || 
opts.options.has(opts.deleteConfigOpt)) {
val configsToBeAdded = parseTopicConfigsToBeAdded(opts)
val configsToBeDeleted = parseTopicConfigsToBeDeleted(opts)
// compile the final set of configs
configs.putAll(configsToBeAdded)
configsToBeDeleted.foreach(config = configs.remove(config))
AdminUtils.changeTopicConfig(zkClient, topic, configs)
println(Updated config for topic \%s\..format(topic))
  }
{code}

Sorry if my doubt is naive/stupid. And feel free to merge this patch, but take 
a look to see if I am doing it right. :) 

Thanks!
Edward

 Warn users if they change max.message.bytes that they also need to update 
 broker and consumer settings
 --

 Key: KAFKA-2338
 URL: https://issues.apache.org/jira/browse/KAFKA-2338
 Project: Kafka
  Issue Type: Bug
  Components: core
Affects Versions: 0.8.2.1
Reporter: Ewen Cheslack-Postava
Assignee: Edward Ribeiro
 Fix For: 0.8.3

 Attachments: KAFKA-2338.patch, KAFKA-2338_2015-07-18_00:37:31.patch, 
 KAFKA-2338_2015-07-21_13:21:19.patch, KAFKA-2338_2015-08-24_14:32:38.patch


 We already have KAFKA-1756 filed to more completely address this issue, but 
 it is waiting for some other major changes to configs to completely protect 
 users from this problem.
 This JIRA should address the low hanging fruit to at least warn users of the 
 potential problems. Currently the only warning is in our documentation.
 1. Generate a warning in the kafka-topics.sh tool when they change this 
 setting on a topic to be larger than the default. This needs to be very 
 obvious in the output.
 2. Currently, the broker's replica fetcher isn't logging any useful error 
 messages when replication can't succeed because a message size is too large. 
 Logging an error here would allow users that get into a bad state to find out 
 why it is happening more easily. (Consumers should already be logging a 
 useful error message.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 37723: Patch for KAFKA-1811

2015-08-24 Thread Edward Ribeiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37723/
---

Review request for kafka.


Bugs: KAFKA-1811
https://issues.apache.org/jira/browse/KAFKA-1811


Repository: kafka


Description
---

KAFKA-1811 - Ensuring registered broker host:port is unique


Diffs
-

  core/src/main/scala/kafka/utils/ZKLock.scala PRE-CREATION 
  core/src/main/scala/kafka/utils/ZkUtils.scala 
74b587e04cdd67386ba8ebccc9430be61c803ad7 
  core/src/test/scala/unit/kafka/server/ServerStartupTest.scala 
0adc0aa3942429639e3f5eabfd4e5a3a8eabe488 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
00fbb61077a0b5ad732c01c6c8d76862ceb5c3c0 
  core/src/test/scala/unit/kafka/zk/ZKLockTest.scala PRE-CREATION 

Diff: https://reviews.apache.org/r/37723/diff/


Testing
---


Thanks,

Edward Ribeiro

[jira] [Commented] (KAFKA-1811) ensuring registered broker host:port is unique


[ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709767#comment-14709767
 ] 

Edward Ribeiro commented on KAFKA-1811:
---

Created reviewboard https://reviews.apache.org/r/37723/diff/
 against branch origin/trunk

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811-2.patch, KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-1811) ensuring registered broker host:port is unique


 [ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Ribeiro updated KAFKA-1811:
--
Status: Patch Available  (was: Open)

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811-2.patch, KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-1811) ensuring registered broker host:port is unique


 [ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Ribeiro updated KAFKA-1811:
--
Attachment: KAFKA-1811.patch

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811-2.patch, KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-1811) ensuring registered broker host:port is unique


 [ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Ribeiro updated KAFKA-1811:
--
Attachment: (was: KAFKA-1811-2.patch)

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-1811) ensuring registered broker host:port is unique


[ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709794#comment-14709794
 ] 

Edward Ribeiro commented on KAFKA-1811:
---

Hi [~gwenshap] and [~nehanarkhede], I have just uploaded a first cut to address 
this issue. I looked for a ready ZK lock recipe, but couldn't find one in 
either ZkClient nor Kafka, so I rolled out one. Hopefully, as Curator is 
integrated into Kafka we will have the chance of replacing it by a superior 
implementation. :) If [~fpj] could take a look at my naive ZKLock, I would be 
really glad. ;) As this patch touches a critical code path, I had to adjust it 
as some unit tests failing (mainly the time sensitive ones), but I didn't have 
the opportunity to run all the test suite, so any feedback about this is 
welcome. Please, let me know if this patch is really worth. 

Thanks!

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-1811) ensuring registered broker host:port is unique


[ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709794#comment-14709794
 ] 

Edward Ribeiro edited comment on KAFKA-1811 at 8/24/15 6:42 PM:


Hi [~gwenshap] and [~nehanarkhede], I have just uploaded a first cut to address 
this issue. I looked for a ready ZK lock recipe, but couldn't find one in 
either ZkClient nor Kafka, so I rolled out one. Hopefully, as Curator is 
integrated into Kafka we will have the chance of replacing it by a superior 
implementation. :) If [~fpj] could take a look at my naive ZKLock, I would be 
really glad. ;) As this patch touches a critical code path, I had to adjust it 
as some unit tests failing (mainly the time sensitive ones), but I didn't have 
the opportunity to run all the test suite, so any feedback about this is 
welcome. Please, let me know if this patch is really worth. 

update: this is my last use of old review process, gonna switch to Github next. 
:)

Thanks!


was (Author: eribeiro):
Hi [~gwenshap] and [~nehanarkhede], I have just uploaded a first cut to address 
this issue. I looked for a ready ZK lock recipe, but couldn't find one in 
either ZkClient nor Kafka, so I rolled out one. Hopefully, as Curator is 
integrated into Kafka we will have the chance of replacing it by a superior 
implementation. :) If [~fpj] could take a look at my naive ZKLock, I would be 
really glad. ;) As this patch touches a critical code path, I had to adjust it 
as some unit tests failing (mainly the time sensitive ones), but I didn't have 
the opportunity to run all the test suite, so any feedback about this is 
welcome. Please, let me know if this patch is really worth. 

Thanks!

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (KAFKA-1811) ensuring registered broker host:port is unique


 [ 
https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Ribeiro updated KAFKA-1811:
--
Comment: was deleted

(was: A tentative (git) patch cut for addressing this issue.)

 ensuring registered broker host:port is unique
 --

 Key: KAFKA-1811
 URL: https://issues.apache.org/jira/browse/KAFKA-1811
 Project: Kafka
  Issue Type: Improvement
Reporter: Jun Rao
Assignee: Edward Ribeiro
  Labels: newbie
 Attachments: KAFKA-1811.patch, KAFKA_1811.patch


 Currently, we expect each of the registered broker to have a unique host:port 
 pair. However, we don't enforce that, which causes various weird problems. It 
 would be useful to ensure this during broker registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-2461) request logger no longer logs extra information in debug mode

Gwen Shapira created KAFKA-2461:
---

 Summary: request logger no longer logs extra information in debug 
mode
 Key: KAFKA-2461
 URL: https://issues.apache.org/jira/browse/KAFKA-2461
 Project: Kafka
  Issue Type: Bug
Reporter: Gwen Shapira


Currently request logging calls are identical for trace and debug:

{code}
if(requestLogger.isTraceEnabled)
requestLogger.trace(Completed request:%s from connection 
%s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
.format(requestDesc, connectionId, totalTime, requestQueueTime, 
apiLocalTime, apiRemoteTime, responseQueueTime, responseSendTime))
  else if(requestLogger.isDebugEnabled)
requestLogger.debug(Completed request:%s from connection 
%s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
  .format(requestDesc, connectionId, totalTime, requestQueueTime, 
apiLocalTime, apiRemoteTime, responseQueueTime, responseSendTime))
{code}

I think in the past (3 refactoring steps ago), we used to print more 
information about specific topics and partitions in debug mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2390) Seek() should take a callback.

2015-08-24 Thread Jiangjie Qin (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709854#comment-14709854
 ] 

Jiangjie Qin commented on KAFKA-2390:
-

Do you mean we have something like:
{code}
class OffsetOutOfRangeException {
  TopicPartition tp;
  long fetchPosition; 
}
{code}
I am thinking what user would do after receiving this exception? I guess most 
of them would want to do some state cleanup and then either reset offset to 
earliest or latest or throw out the exception. So it seems useful to provide 
the log starting or ending offset in the exception as well, right?

 Seek() should take a callback.
 --

 Key: KAFKA-2390
 URL: https://issues.apache.org/jira/browse/KAFKA-2390
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Dong Lin

 Currently seek is an async call. To have the same interface as other calls 
 like commit(), seek() should take a callback. This callback will be invoked 
 if the position to seek triggers OFFSET_OUT_OF_RANGE exception from broker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2461) request logger no longer logs extra information in debug mode

2015-08-24 Thread Ashish K Singh (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709853#comment-14709853
 ] 

Ashish K Singh commented on KAFKA-2461:
---

[~gwenshap] I too noticed this. Do you mind if I take a look at this?

 request logger no longer logs extra information in debug mode
 -

 Key: KAFKA-2461
 URL: https://issues.apache.org/jira/browse/KAFKA-2461
 Project: Kafka
  Issue Type: Bug
Reporter: Gwen Shapira

 Currently request logging calls are identical for trace and debug:
 {code}
 if(requestLogger.isTraceEnabled)
 requestLogger.trace(Completed request:%s from connection 
 %s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
 .format(requestDesc, connectionId, totalTime, 
 requestQueueTime, apiLocalTime, apiRemoteTime, responseQueueTime, 
 responseSendTime))
   else if(requestLogger.isDebugEnabled)
 requestLogger.debug(Completed request:%s from connection 
 %s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
   .format(requestDesc, connectionId, totalTime, requestQueueTime, 
 apiLocalTime, apiRemoteTime, responseQueueTime, responseSendTime))
 {code}
 I think in the past (3 refactoring steps ago), we used to print more 
 information about specific topics and partitions in debug mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2390) Seek() should take a callback.


[ 
https://issues.apache.org/jira/browse/KAFKA-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709867#comment-14709867
 ] 

Jay Kreps commented on KAFKA-2390:
--

I think there could be multiple out of range partitions so it would have to be 
something like
   class OffsetOutOfRangeException(val partitionOffsets: List[PartitionOffset])
(in scala syntax).

I think it could be nice to include the log start and end if we already have 
that on hand but if not I don't think we should do extra requests to get it.

 Seek() should take a callback.
 --

 Key: KAFKA-2390
 URL: https://issues.apache.org/jira/browse/KAFKA-2390
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Dong Lin

 Currently seek is an async call. To have the same interface as other calls 
 like commit(), seek() should take a callback. This callback will be invoked 
 if the position to seek triggers OFFSET_OUT_OF_RANGE exception from broker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2412) Documentation bug: Add information for key.serializer and value.serializer to New Producer Config sections

2015-08-24 Thread Grayson Chao (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Grayson Chao updated KAFKA-2412:

Attachment: KAFKA-2412-r1.diff

 Documentation bug: Add information for key.serializer and value.serializer to 
 New Producer Config sections
 --

 Key: KAFKA-2412
 URL: https://issues.apache.org/jira/browse/KAFKA-2412
 Project: Kafka
  Issue Type: Bug
Reporter: Jeremy Fields
Assignee: Grayson Chao
Priority: Minor
  Labels: newbie
 Attachments: KAFKA-2412-r1.diff, KAFKA-2412.diff


 As key.serializer and value.serializer are required options when using the 
 new producer, they should be mentioned in the documentation ( here and svn 
 http://kafka.apache.org/documentation.html#newproducerconfigs )
 Appropriate values for these options exist in javadoc and producer.java 
 examples; however, not everyone is reading those, as is the case for anyone 
 setting up a producer.config file for mirrormaker.
 A sensible default should be suggested, such as
 org.apache.kafka.common.serialization.StringSerializer
 Or at least a mention of the key.serializer and value.serializer options 
 along with a link to javadoc
 Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-1792) change behavior of --generate to produce assignment config with fair replica distribution and minimal number of reassignments

2015-08-24 Thread Allen Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709887#comment-14709887
 ] 

Allen Wang commented on KAFKA-1792:
---

I agree with [~88manpreet] that after the reassignment the leader count for 
each broker can be imbalanced. This is the result of the following code in 
AdminUtils.getReplicaReassignmentByPartitions: 

{code}
val result = new mutable.LinkedHashMap[TopicAndPartition, ListBuffer[Int]]()
reassignment.keys.foreach(brokerId = {
  val partitions = reassignment.getOrElse(brokerId, null)
  partitions.foreach(p = result.getOrElseUpdate(p, new 
ListBuffer[Int]).append(brokerId))
})
{code}

The brokers that appear early in the traversing order of reassignment.keys will 
likely to be the leaders of partitions. They will get more load after preferred 
leader election. It will be great to avoid that problem.



 change behavior of --generate to produce assignment config with fair replica 
 distribution and minimal number of reassignments
 -

 Key: KAFKA-1792
 URL: https://issues.apache.org/jira/browse/KAFKA-1792
 Project: Kafka
  Issue Type: Sub-task
  Components: tools
Reporter: Dmitry Pekar
Assignee: Dmitry Pekar
 Attachments: KAFKA-1792.patch, KAFKA-1792_2014-12-03_19:24:56.patch, 
 KAFKA-1792_2014-12-08_13:42:43.patch, KAFKA-1792_2014-12-19_16:48:12.patch, 
 KAFKA-1792_2015-01-14_12:54:52.patch, KAFKA-1792_2015-01-27_19:09:27.patch, 
 KAFKA-1792_2015-02-13_21:07:06.patch, KAFKA-1792_2015-02-26_16:58:23.patch, 
 generate_alg_tests.txt, rebalance_use_cases.txt


 Current implementation produces fair replica distribution between specified 
 list of brokers. Unfortunately, it doesn't take
 into account current replica assignment.
 So if we have, for instance, 3 brokers id=[0..2] and are going to add fourth 
 broker id=3, 
 generate will create an assignment config which will redistribute replicas 
 fairly across brokers [0..3] 
 in the same way as those partitions were created from scratch. It will not 
 take into consideration current replica 
 assignment and accordingly will not try to minimize number of replica moves 
 between brokers.
 As proposed by [~charmalloc] this should be improved. New output of improved 
 --generate algorithm should suite following requirements:
 - fairness of replica distribution - every broker will have R or R+1 replicas 
 assigned;
 - minimum of reassignments - number of replica moves between brokers will be 
 minimal;
 Example.
 Consider following replica distribution per brokers [0..3] (we just added 
 brokers 2 and 3):
 - broker - 0, 1, 2, 3 
 - replicas - 7, 6, 0, 0
 The new algorithm will produce following assignment:
 - broker - 0, 1, 2, 3 
 - replicas - 4, 3, 3, 3
 - moves - -3, -3, +3, +3
 It will be fair and number of moves will be 6, which is minimal for specified 
 initial distribution.
 The scope of this issue is:
 - design an algorithm matching the above requirements;
 - implement this algorithm and unit tests;
 - test it manually using different initial assignments;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2390) Seek() should take a callback.

2015-08-24 Thread Jiangjie Qin (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708826#comment-14708826
 ] 

Jiangjie Qin commented on KAFKA-2390:
-

[~jkreps] I share your concern of the adding complexity to the API. To explain 
the case current API does not support well, consider the following code:
{code}
...
seek(t0p0, 100);
poll();
seek(t1p0, 1000);
poll();
seek(t2p0, 1000);
poll(); // OffsetOutOfRangeException received.
...
{code}
If user configure the offset reset policy to None, with current API, user may 
not be able to tell which seek actually failed, so it is hard for user to take 
action.

The *synchronous* semantic here is not very clear. When we tell user seek() is 
a synchronous call, everyone we talked to thought it also verifies the offset. 
In fact it is synchronous in terms of setting the in memory fetch position, but 
it does not really verify the offset. The exception of seek() is *asynchronous* 
and will come later at some point.

I think the problem in above example is real. It would be nice if we can find a 
solution to that without complicating the current API. One alternative might be 
throwing a SeekException like below:
{code}
public class SeekException {
  TopicPartition topicPartition;
  long SeekedOffset;
  long LogStartingOffset;
  long LogEndOffset;
}
{code}
So user can decide what to do when got the exception.

 Seek() should take a callback.
 --

 Key: KAFKA-2390
 URL: https://issues.apache.org/jira/browse/KAFKA-2390
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Dong Lin

 Currently seek is an async call. To have the same interface as other calls 
 like commit(), seek() should take a callback. This callback will be invoked 
 if the position to seek triggers OFFSET_OUT_OF_RANGE exception from broker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[
https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708866#comment-14708866
]

Ewen Cheslack-Postava commented on KAFKA-2389:
--

If I'm understanding people's current positions, I think I agree with Jay that
we should take an all-or-nothing approach for async. People who want async
should know what they are doing. If they provide no args, they expect an async
commit with the current offsets; if they provide arguments, they might have to
provide a bit more information (OffsetAndMetadata rather than just offsets),
but that burden is perfectly acceptable for a relatively unusual use case.
Anyone using async commit should be comfortable passing in a callback (or null
if appropriate) for a callback to an async method. However, I still think the
commitAsync naming (or any similar differentiation between sync and async
commits) helps to make it clear to the user the semantics of the method they
are invoking. So I think 2 methods (no parameters map + callback) for the
async variants should work well enough.

CommitType seems not necessary in commit().
---

Key: KAFKA-2389
URL: https://issues.apache.org/jira/browse/KAFKA-2389
Project: Kafka
Issue Type: Sub-task
Reporter: Jiangjie Qin
Assignee: Jiangjie Qin

The CommitType does not seem to be necessary in for commit(), it can be
inferred from whether user passed in a callback or not.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2431) Test SSL/TLS impact on performance


[ 
https://issues.apache.org/jira/browse/KAFKA-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709914#comment-14709914
 ] 

Jay Kreps commented on KAFKA-2431:
--

It would be good to also do the same test(s) against 0.8.2 (I'm assuming these 
results are from trunk for both the ssl and no ssl case). There have been a TON 
of changes in the network layer over all so we need to sanity check that the no 
SSL number is actually the true baseline.

 Test SSL/TLS impact on performance
 --

 Key: KAFKA-2431
 URL: https://issues.apache.org/jira/browse/KAFKA-2431
 Project: Kafka
  Issue Type: Sub-task
  Components: security
Reporter: Ismael Juma
Assignee: Ben Stopford
 Fix For: 0.8.3


 Test new Producer and new Consumer performance with and without SSL/TLS once 
 the SSL/TLS branch is integrated.
 The ideal scenario is that SSL/TLS would not have an impact if disabled. When 
 enabled, there will be some overhead (encryption and the inability to use 
 `SendFile`) and it will be good to quantify it. The encryption overhead is 
 reduced if recent JDKs are used with CPUs that support AES-specific 
 instructions (https://en.wikipedia.org/wiki/AES_instruction_set).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-24 Thread Joel Koshy



 On Aug. 22, 2015, 12:45 a.m., Joel Koshy wrote:
  clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java, 
  line 107
  https://reviews.apache.org/r/33378/diff/12-13/?file=1043780#file1043780line107
 
  This will probably need a versionId as well (as is done in the Scala 
  response) - i.e., when we move the broker over to use these protocol 
  schemas.
 
 Aditya Auradkar wrote:
 Makes sense. Do you want me to tackle this in this patch or should it be 
 tackled in the patch that migrates the broker to use these schemas?

I think it would be safer to do it in this patch itself.


- Joel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96100
---


On Aug. 24, 2015, 5:33 p.m., Aditya Auradkar wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/33378/
 ---
 
 (Updated Aug. 24, 2015, 5:33 p.m.)
 
 
 Review request for kafka, Joel Koshy and Jun Rao.
 
 
 Bugs: KAFKA-2136
 https://issues.apache.org/jira/browse/KAFKA-2136
 
 
 Repository: kafka
 
 
 Description
 ---
 
 Changes are
 - Addressing Joel's comments
 - protocol changes to the fetch request and response to return the 
 throttle_time_ms to clients
 - New producer/consumer metrics to expose the avg and max delay time for a 
 client
 - Test cases.
 - Addressed Joel and Juns comments
 
 
 Diffs
 -
 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java
  9dc669728e6b052f5c6686fcf1b5696a50538ab4 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 0baf16e55046a2f49f6431e01d52c323c95eddf0 
   clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 
 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f 
   clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 
 df073a0e76cc5cc731861b9604d0e19a928970e0 
   clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java 
 eb8951fba48c335095cc43fc3672de1c733e07ff 
   clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 
 715504b32950666e9aa5a260fa99d5f897b2007a 
   clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 
 febfc70dabc23671fd8a85cf5c5b274dff1e10fb 
   
 clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java
  a7c83cac47d41138d47d7590a3787432d675c1b0 
   
 clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java
  8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4 
   
 clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java
  8b2aca85fa738180e5420985fddc39a4bf9681ea 
   core/src/main/scala/kafka/api/FetchRequest.scala 
 5b38f8554898e54800abd65a7415dd0ac41fd958 
   core/src/main/scala/kafka/api/FetchResponse.scala 
 b9efec2efbd3ea0ee12b911f453c47e66ad34894 
   core/src/main/scala/kafka/api/ProducerRequest.scala 
 c866180d3680da03e48d374415f10220f6ca68c4 
   core/src/main/scala/kafka/api/ProducerResponse.scala 
 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330 
   core/src/main/scala/kafka/consumer/SimpleConsumer.scala 
 7ebc0405d1f309bed9943e7116051d1d8276f200 
   core/src/main/scala/kafka/server/AbstractFetcherThread.scala 
 f84306143c43049e3aa44e42beaffe7eb2783163 
   core/src/main/scala/kafka/server/ClientQuotaManager.scala 
 9f8473f1c64d10c04cf8cc91967688e29e54ae2e 
   core/src/main/scala/kafka/server/KafkaApis.scala 
 67f0cad802f901f255825aa2158545d7f5e7cc3d 
   core/src/main/scala/kafka/server/ReplicaFetcherThread.scala 
 fae22d2af8daccd528ac24614290f46ae8f6c797 
   core/src/main/scala/kafka/server/ReplicaManager.scala 
 d829e180c3943a90861a12ec184f9b4e4bbafe7d 
   core/src/main/scala/kafka/server/ThrottledResponse.scala 
 1f80d5480ccf7c411a02dd90296a7046ede0fae2 
   core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 
 b4c2a228c3c9872e5817ac58d3022e4903e317b7 
   core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 
 caf98e8f2e09d39ab8234b9f4b9478686865e908 
   core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 
 c4b5803917e700965677d53624f1460c1a52bf52 
 
 Diff: https://reviews.apache.org/r/33378/diff/
 
 
 Testing
 ---
 
 New tests added
 
 
 Thanks,
 
 Aditya Auradkar

[jira] [Commented] (KAFKA-2461) request logger no longer logs extra information in debug mode


[ 
https://issues.apache.org/jira/browse/KAFKA-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709988#comment-14709988
 ] 

Gwen Shapira commented on KAFKA-2461:
-

Of course. Thanks [~singhashish]

 request logger no longer logs extra information in debug mode
 -

 Key: KAFKA-2461
 URL: https://issues.apache.org/jira/browse/KAFKA-2461
 Project: Kafka
  Issue Type: Bug
Reporter: Gwen Shapira

 Currently request logging calls are identical for trace and debug:
 {code}
 if(requestLogger.isTraceEnabled)
 requestLogger.trace(Completed request:%s from connection 
 %s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
 .format(requestDesc, connectionId, totalTime, 
 requestQueueTime, apiLocalTime, apiRemoteTime, responseQueueTime, 
 responseSendTime))
   else if(requestLogger.isDebugEnabled)
 requestLogger.debug(Completed request:%s from connection 
 %s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
   .format(requestDesc, connectionId, totalTime, requestQueueTime, 
 apiLocalTime, apiRemoteTime, responseQueueTime, responseSendTime))
 {code}
 I think in the past (3 refactoring steps ago), we used to print more 
 information about specific topics and partitions in debug mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2431) Test SSL/TLS impact on performance

2015-08-24 Thread Ismael Juma (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1470#comment-1470
 ] 

Ismael Juma commented on KAFKA-2431:


Yes, definitely. We discussed that offline, but I realise now that it wasn't 
clear in the ticket, so thanks for mentioning that.

 Test SSL/TLS impact on performance
 --

 Key: KAFKA-2431
 URL: https://issues.apache.org/jira/browse/KAFKA-2431
 Project: Kafka
  Issue Type: Sub-task
  Components: security
Reporter: Ismael Juma
Assignee: Ben Stopford
 Fix For: 0.8.3


 Test new Producer and new Consumer performance with and without SSL/TLS once 
 the SSL/TLS branch is integrated.
 The ideal scenario is that SSL/TLS would not have an impact if disabled. When 
 enabled, there will be some overhead (encryption and the inability to use 
 `SendFile`) and it will be good to quantify it. The encryption overhead is 
 reduced if recent JDKs are used with CPUs that support AES-specific 
 instructions (https://en.wikipedia.org/wiki/AES_instruction_set).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2412) Documentation bug: Add information for key.serializer and value.serializer to New Producer Config sections

2015-08-24 Thread Grayson Chao (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709975#comment-14709975
 ] 

Grayson Chao commented on KAFKA-2412:
-

Thanks [~wushujames]!

 Documentation bug: Add information for key.serializer and value.serializer to 
 New Producer Config sections
 --

 Key: KAFKA-2412
 URL: https://issues.apache.org/jira/browse/KAFKA-2412
 Project: Kafka
  Issue Type: Bug
Reporter: Jeremy Fields
Assignee: Grayson Chao
Priority: Minor
  Labels: newbie
 Attachments: KAFKA-2412-r1.diff, KAFKA-2412.diff


 As key.serializer and value.serializer are required options when using the 
 new producer, they should be mentioned in the documentation ( here and svn 
 http://kafka.apache.org/documentation.html#newproducerconfigs )
 Appropriate values for these options exist in javadoc and producer.java 
 examples; however, not everyone is reading those, as is the case for anyone 
 setting up a producer.config file for mirrormaker.
 A sensible default should be suggested, such as
 org.apache.kafka.common.serialization.StringSerializer
 Or at least a mention of the key.serializer and value.serializer options 
 along with a link to javadoc
 Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2412) Documentation bug: Add information for key.serializer and value.serializer to New Producer Config sections

2015-08-24 Thread James Cheng (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709974#comment-14709974
 ] 

James Cheng commented on KAFKA-2412:


[~gchao], here are some slides that describe how out-of-order arrival is 
possible: 
http://www.slideshare.net/JiangjieQin/no-data-loss-pipeline-with-apache-kafka-49753844

 Documentation bug: Add information for key.serializer and value.serializer to 
 New Producer Config sections
 --

 Key: KAFKA-2412
 URL: https://issues.apache.org/jira/browse/KAFKA-2412
 Project: Kafka
  Issue Type: Bug
Reporter: Jeremy Fields
Assignee: Grayson Chao
Priority: Minor
  Labels: newbie
 Attachments: KAFKA-2412-r1.diff, KAFKA-2412.diff


 As key.serializer and value.serializer are required options when using the 
 new producer, they should be mentioned in the documentation ( here and svn 
 http://kafka.apache.org/documentation.html#newproducerconfigs )
 Appropriate values for these options exist in javadoc and producer.java 
 examples; however, not everyone is reading those, as is the case for anyone 
 setting up a producer.config file for mirrormaker.
 A sensible default should be suggested, such as
 org.apache.kafka.common.serialization.StringSerializer
 Or at least a mention of the key.serializer and value.serializer options 
 along with a link to javadoc
 Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-2462) allow modifying soft limit for open files in Kafka startup script

Gwen Shapira created KAFKA-2462:
---

 Summary: allow modifying soft limit for open files in Kafka 
startup script
 Key: KAFKA-2462
 URL: https://issues.apache.org/jira/browse/KAFKA-2462
 Project: Kafka
  Issue Type: Bug
Reporter: Gwen Shapira
Assignee: Gwen Shapira


In some systems the hard limit for number of open files is set reasonably high, 
but the default soft limit for the user running Kafka is insufficient.

It would be nice if the Kafka startup script could set the soft limit of number 
of files for the Kafka process to a user-defined value before starting Kafka. 

Something like:
kafka-server-start --soft-file-limit 1 config/server.properties




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

no Kafka KIP meeting tomorrow

2015-08-24 Thread Jun Rao

Since there are no new KIP issues for discussion, there is no KIP meeting
tomorrow.

Thanks,

Jun

[jira] [Created] (KAFKA-2464) Client-side assignment and group generalization

2015-08-24 Thread Jason Gustafson (JIRA)

Jason Gustafson created KAFKA-2464:
--

 Summary: Client-side assignment and group generalization
 Key: KAFKA-2464
 URL: https://issues.apache.org/jira/browse/KAFKA-2464
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jason Gustafson
Assignee: Jason Gustafson


Add support for client-side assignment and generalization of join group 
protocol as documented here: 
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Client-side+Assignment+Proposal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

[
https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710216#comment-14710216
]

Ewen Cheslack-Postava commented on KAFKA-2367:
--

I posted an initial patch. It covers roughly the functionality of Avro's APIs.
The actual data API code (not tests) clocks in at 600 lines, substantially
smaller than the code from Avro (which didn't even have GenericRecord support
included...) Overall the code (including tests) shrunk by 1200 lines and now
has more functionality. The API broader than Avro in some ways (more primitive
types to cover types available in other serialization formats, supports typed
keys in maps, etc), in other ways it's omitted some things currently (enums,
unions).

Some notes and thoughts:

* I have not defined any logical types yet, but they can be implemented for any
type and identified by the name + version fields. I think most of the effort
here is in the specification. The implementation is just providing the schema
definitions, and occasionally including conversion utilities for built-in java
types that correspond to the type and are likely to be used commonly.

* The version field is a bit weird - it is a byte[]. This lets you, e.g., use a
network byte order integer directly if you want to. I considered just requiring
a string instead. The overhead of requiring encoding to a string seems pretty
minimal for versioning schemes I can think of (incrementing integers, hashes),
but the byte[] seemed like the right way to handle it. One thing the
documentation for version numbers doesn't currently define is how ordering
works. I'm not sure versions are going to be very useful without this (besides
uniquely identifying the schema, it would be nice to know whether a schema is
newer or older than another one), but I'm not sure everyone will have a
versioning scheme that allows for comparing versions without contacting an
external service.

* I provided support for byte[] and ByteBuffer. byte[] is really annoying since
equals/hashCode don't work for it; the situation is made worse because it can
be nested, arbitrarily deeply, in lists and maps. I haven't tried to address
the equals/hashCode problems in those cases. I think its better to recommend
using ByteBuffer when you need to be able to test for equality/compute
hashcodes. However, this also needs to be respected by deserializers/converters.

* Made schemas explicit in the Copycat APIs. Previously schemas were inferred
for primitive types. I think making this explicit might be a bit more effort
for connector developers, but it's ultimately the right thing to do. There are
too many weird edge cases you can get into without explicitly specifying the
schema, especially with list/map types (if it's empty, what are the key/value
schemas?) and optional values (if you encounter a null, what's the schema?).

* Caching of conversion of schemas. In the JSON implementation we're including,
we're probably being pretty wasteful right now since every record has to
translate both the schema and data to JSON. We should definitely be doing some
caching here. I think an LRU using an IdentityHashMap should be fine. However,
this does assume that connectors are good about reusing schemas (defining them
up front, or if they are dynamic they should have their own cache of schemas
and be able to detect when they can be reused).

* Many serialization formats don't support unions natively. They require a
separate struct that contains optional fields for all the types (and might
provide syntactic sugar, but it has no impact on the actual encoding). When you
make nullability part of every type, I think the need for unions pretty much
disappears -- creating the appropriate struct is trivial.

* I didn't include an IndexedRecord-like interface. I did allow setting a field
by passing in the Field object instead of the field name, which avoids having
to do a hash lookup of the fieldname, which has basically the same result as
setting via the index. I didn't want to expose something like put(int
fieldNumber, Object value) because using an array internally in Struct is
really an implementation detail to avoid object allocations.

* The fluent API for SchemaBuilder is different from how avro works. It prefers
a single class for building schemas, doesn't directly handle nesting (you can
just run another schemabuilder inline in the calls to the parent). This
actually is a substantial difference -- in Copycat the expectation is that most
schemas are created dynamically (most connectors do *not* have fixed schemas to
work with), whereas Avro's seems more targeted towards more static use cases
(not surprisingly). Avro's more complex implementation can catch certain types
of errors at compile time (there are lots of different *Builder classes with
various abilities/restrictions), but for

[GitHub] kafka pull request: KAFKA-2462: allow modifying soft limit for ope...

2015-08-24 Thread gwenshap

GitHub user gwenshap opened a pull request:

https://github.com/apache/kafka/pull/164

KAFKA-2462: allow modifying soft limit for open files in Kafka startup 
script



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gwenshap/kafka ulimit

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/164.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #164


commit 7a049dafddb362a15498772df64c341d80e52d9b
Author: Gwen Shapira csh...@gmail.com
Date:   2015-08-24T23:21:08Z

adding parameter for setting soft ulimit. tested on Linux




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (KAFKA-2461) request logger no longer logs extra information in debug mode

2015-08-24 Thread Ashish K Singh (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish K Singh reassigned KAFKA-2461:
-

Assignee: Ashish K Singh

 request logger no longer logs extra information in debug mode
 -

 Key: KAFKA-2461
 URL: https://issues.apache.org/jira/browse/KAFKA-2461
 Project: Kafka
  Issue Type: Bug
Reporter: Gwen Shapira
Assignee: Ashish K Singh

 Currently request logging calls are identical for trace and debug:
 {code}
 if(requestLogger.isTraceEnabled)
 requestLogger.trace(Completed request:%s from connection 
 %s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
 .format(requestDesc, connectionId, totalTime, 
 requestQueueTime, apiLocalTime, apiRemoteTime, responseQueueTime, 
 responseSendTime))
   else if(requestLogger.isDebugEnabled)
 requestLogger.debug(Completed request:%s from connection 
 %s;totalTime:%d,requestQueueTime:%d,localTime:%d,remoteTime:%d,responseQueueTime:%d,sendTime:%d
   .format(requestDesc, connectionId, totalTime, requestQueueTime, 
 apiLocalTime, apiRemoteTime, responseQueueTime, responseSendTime))
 {code}
 I think in the past (3 refactoring steps ago), we used to print more 
 information about specific topics and partitions in debug mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2462) allow modifying soft limit for open files in Kafka startup script


[ 
https://issues.apache.org/jira/browse/KAFKA-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710261#comment-14710261
 ] 

ASF GitHub Bot commented on KAFKA-2462:
---

GitHub user gwenshap opened a pull request:

https://github.com/apache/kafka/pull/164

KAFKA-2462: allow modifying soft limit for open files in Kafka startup 
script



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gwenshap/kafka ulimit

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/164.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #164


commit 7a049dafddb362a15498772df64c341d80e52d9b
Author: Gwen Shapira csh...@gmail.com
Date:   2015-08-24T23:21:08Z

adding parameter for setting soft ulimit. tested on Linux




 allow modifying soft limit for open files in Kafka startup script
 -

 Key: KAFKA-2462
 URL: https://issues.apache.org/jira/browse/KAFKA-2462
 Project: Kafka
  Issue Type: Bug
Reporter: Gwen Shapira
Assignee: Gwen Shapira

 In some systems the hard limit for number of open files is set reasonably 
 high, but the default soft limit for the user running Kafka is insufficient.
 It would be nice if the Kafka startup script could set the soft limit of 
 number of files for the Kafka process to a user-defined value before starting 
 Kafka. 
 Something like:
 kafka-server-start --soft-file-limit 1 config/server.properties



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-2465) Need to document replica.fetcher.backoff.ms

Gwen Shapira created KAFKA-2465:
---

 Summary: Need to document replica.fetcher.backoff.ms
 Key: KAFKA-2465
 URL: https://issues.apache.org/jira/browse/KAFKA-2465
 Project: Kafka
  Issue Type: Improvement
Reporter: Gwen Shapira


We added this parameter in KAFKA-1461, it changes existing behavior and is 
configurable by users. 

We should document the new behavior and the parameter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2465) Need to document replica.fetcher.backoff.ms


[ 
https://issues.apache.org/jira/browse/KAFKA-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710457#comment-14710457
 ] 

Gwen Shapira commented on KAFKA-2465:
-

[~harsha_ch] - since you added the configuration, can you pick this one up too?

 Need to document replica.fetcher.backoff.ms
 ---

 Key: KAFKA-2465
 URL: https://issues.apache.org/jira/browse/KAFKA-2465
 Project: Kafka
  Issue Type: Improvement
Reporter: Gwen Shapira

 We added this parameter in KAFKA-1461, it changes existing behavior and is 
 configurable by users. 
 We should document the new behavior and the parameter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (KAFKA-2463) Our shell scripts should use getopts for argument parsing.


 [ 
https://issues.apache.org/jira/browse/KAFKA-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gwen Shapira resolved KAFKA-2463.
-
Resolution: Won't Fix

Never mind, getopts doesn't support long options, which means we will break 
-daemon.

Lets stick with middle-ages for now :)

 Our shell scripts should use getopts for argument parsing.
 --

 Key: KAFKA-2463
 URL: https://issues.apache.org/jira/browse/KAFKA-2463
 Project: Kafka
  Issue Type: Improvement
Reporter: Gwen Shapira

 It is 2015 and we are parsing arguments manually.
 Other than causing people to wonder if our network protocol was optimized for 
 circuit switching telephony, it also prevents us from adding new arguments 
 cleanly.
 Some examples of arguments we may decide to add:
 --debug port - for attaching a debugger
 --ulimit type=value
 There are probably more. Lets use getopts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-2367) Add Copycat runtime data API

[
https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710216#comment-14710216
]

Ewen Cheslack-Postava edited comment on KAFKA-2367 at 8/24/15 10:56 PM:

Some notes and thoughts:

[jira] [Commented] (KAFKA-2106) Partition balance tool between borkers

2015-08-24 Thread Allen Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710364#comment-14710364
 ] 

Allen Wang commented on KAFKA-2106:
---

I am curious in the following code why there is a restriction that partitions 
should have no more than 3 replicas and every partition has the same number or 
replicas. It is possible that different topic has different number of replicas 
given different requirement of availability.

{code}
  def filterValidTopicAssignment() = {
val groupedByTopic = allTopicsAssignment.groupBy(tp = tp._1.topic)
/**
 * check replicas:
 * replicas amount should be more than 0 and less than 3
 * all partitions should have the same amount of replicas
 *
 */
var validTopicAssignment = groupedByTopic.filter(
  t = {
t._2.head._2.size  0  t._2.head._2.size  3  t._2.values.map(seq 
= seq.length).toSet.size == 1
  }
)

if(includeTopicSet.size != 0) {
  validTopicAssignment = validTopicAssignment.filter(topicInfo = 
includeTopicSet.contains(topicInfo._1))
}

if(excludeTopicSet.size != 0) {
  validTopicAssignment = validTopicAssignment.filter(topicInfo = (! 
excludeTopicSet.contains(topicInfo._1)))
}

validTopicAssignment
  }
{code}

 Partition balance tool between borkers
 --

 Key: KAFKA-2106
 URL: https://issues.apache.org/jira/browse/KAFKA-2106
 Project: Kafka
  Issue Type: New Feature
  Components: admin
Affects Versions: 0.8.3
Reporter: chenshangan
 Attachments: KAFKA-2106.3, KAFKA-2106.patch, KAFKA-2106.patch.2


 The default partition assignment algorithm can work well in a static kafka 
 cluster(number of brokers seldom change). Actually, in production env, number 
 of brokers is always increasing according to the business data. When new 
 brokers added to the cluster, it's better to provide a tool that can help to 
 move existing data to new brokers. Currently, users need to choose topic or 
 partitions manually and use the Reassign Partitions Tool 
 (kafka-reassign-partitions.sh) to achieve the goal. It's a time-consuming 
 task when there's a lot of topics in the cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-2463) Our shell scripts should use getopts for argument parsing.

Gwen Shapira created KAFKA-2463:
---

 Summary: Our shell scripts should use getopts for argument parsing.
 Key: KAFKA-2463
 URL: https://issues.apache.org/jira/browse/KAFKA-2463
 Project: Kafka
  Issue Type: Improvement
Reporter: Gwen Shapira


It is 2015 and we are parsing arguments manually.

Other than causing people to wonder if our network protocol was optimized for 
circuit switching telephony, it also prevents us from adding new arguments 
cleanly.
Some examples of arguments we may decide to add:
--debug port - for attaching a debugger
--ulimit type=value

There are probably more. Lets use getopts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 36652: Patch for KAFKA-2351

2015-08-24 Thread Mayuresh Gharat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36652/
---

(Updated Aug. 24, 2015, 10:50 p.m.)


Review request for kafka.


Bugs: KAFKA-2351
https://issues.apache.org/jira/browse/KAFKA-2351


Repository: kafka


Description (updated)
---

Addressed Joel's comments


Diffs (updated)
-

  core/src/main/scala/kafka/network/SocketServer.scala 
649812d9f8014edbd9e99113a0f9eaf186360bcc 

Diff: https://reviews.apache.org/r/36652/diff/


Testing
---


Thanks,

Mayuresh Gharat

[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly

2015-08-24 Thread Mayuresh Gharat (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710205#comment-14710205
 ] 

Mayuresh Gharat commented on KAFKA-2351:


Updated reviewboard https://reviews.apache.org/r/36652/diff/
 against branch origin/trunk

 Brokers are having a problem shutting down correctly
 

 Key: KAFKA-2351
 URL: https://issues.apache.org/jira/browse/KAFKA-2351
 Project: Kafka
  Issue Type: Bug
Reporter: Mayuresh Gharat
Assignee: Mayuresh Gharat
 Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, 
 KAFKA-2351_2015-07-23_21:36:52.patch, KAFKA-2351_2015-08-13_13:10:05.patch, 
 KAFKA-2351_2015-08-24_15:50:41.patch


 The run() in Acceptor during shutdown might throw an exception that is not 
 caught and it never reaches shutdownComplete due to which the latch is not 
 counted down and the broker will not be able to shutdown.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2351) Brokers are having a problem shutting down correctly

2015-08-24 Thread Mayuresh Gharat (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mayuresh Gharat updated KAFKA-2351:
---
Attachment: KAFKA-2351_2015-08-24_15:50:41.patch

 Brokers are having a problem shutting down correctly
 

 Key: KAFKA-2351
 URL: https://issues.apache.org/jira/browse/KAFKA-2351
 Project: Kafka
  Issue Type: Bug
Reporter: Mayuresh Gharat
Assignee: Mayuresh Gharat
 Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, 
 KAFKA-2351_2015-07-23_21:36:52.patch, KAFKA-2351_2015-08-13_13:10:05.patch, 
 KAFKA-2351_2015-08-24_15:50:41.patch


 The run() in Acceptor during shutdown might throw an exception that is not 
 caught and it never reaches shutdownComplete due to which the latch is not 
 counted down and the broker will not be able to shutdown.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: KAFKA-2367: Add Copycat runtime data API.

2015-08-24 Thread ewencp

GitHub user ewencp opened a pull request:

https://github.com/apache/kafka/pull/163

KAFKA-2367: Add Copycat runtime data API.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ewencp/kafka 
kafka-2367-copycat-runtime-data-api

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/163.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #163


commit b90049e06a060f473878f6df1de9f4b6f2b38bc5
Author: Ewen Cheslack-Postava m...@ewencp.org
Date:   2015-08-21T01:06:56Z

KAFKA-2367: Add Copycat runtime data API.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API


[ 
https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710212#comment-14710212
 ] 

ASF GitHub Bot commented on KAFKA-2367:
---

GitHub user ewencp opened a pull request:

https://github.com/apache/kafka/pull/163

KAFKA-2367: Add Copycat runtime data API.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ewencp/kafka 
kafka-2367-copycat-runtime-data-api

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/163.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #163


commit b90049e06a060f473878f6df1de9f4b6f2b38bc5
Author: Ewen Cheslack-Postava m...@ewencp.org
Date:   2015-08-21T01:06:56Z

KAFKA-2367: Add Copycat runtime data API.




 Add Copycat runtime data API
 

 Key: KAFKA-2367
 URL: https://issues.apache.org/jira/browse/KAFKA-2367
 Project: Kafka
  Issue Type: Sub-task
  Components: copycat
Reporter: Ewen Cheslack-Postava
Assignee: Ewen Cheslack-Postava
 Fix For: 0.8.3


 Design the API used for runtime data in Copycat. This API is used to 
 construct schemas and records that Copycat processes. This needs to be a 
 fairly general data model (think Avro, JSON, Protobufs, Thrift) in order to 
 support complex, varied data types that may be input from/output to many data 
 systems.
 This should issue should also address the serialization interfaces used 
 within Copycat, which translate the runtime data into serialized byte[] form. 
 It is important that these be considered together because the data format can 
 be used in multiple ways (records, partition IDs, partition offsets), so it 
 and the corresponding serializers must be sufficient for all these use cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2464) Client-side assignment and group generalization


[ 
https://issues.apache.org/jira/browse/KAFKA-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710573#comment-14710573
 ] 

ASF GitHub Bot commented on KAFKA-2464:
---

GitHub user hachikuji opened a pull request:

https://github.com/apache/kafka/pull/165

KAFKA-2464; client-side assignment for new consumer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hachikuji/kafka KAFKA-2464

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/165.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #165


commit f6349cb9a3954464277ea3bd5ecfb80dcb8f4345
Author: Jason Gustafson ja...@confluent.io
Date:   2015-08-13T20:33:05Z

KAFKA-2464; client-side assignment for new consumer




 Client-side assignment and group generalization
 ---

 Key: KAFKA-2464
 URL: https://issues.apache.org/jira/browse/KAFKA-2464
 Project: Kafka
  Issue Type: Sub-task
  Components: consumer
Reporter: Jason Gustafson
Assignee: Jason Gustafson
 Fix For: 0.8.3


 Add support for client-side assignment and generalization of join group 
 protocol as documented here: 
 https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Client-side+Assignment+Proposal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents


[ 
https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710622#comment-14710622
 ] 

David Jacot commented on KAFKA-2071:


I have merged trunk in the PR and adapted it to the latest changes. It is ready 
for review when you have time.

 Replace Produce Request/Response with their org.apache.kafka.common.requests 
 equivalents
 

 Key: KAFKA-2071
 URL: https://issues.apache.org/jira/browse/KAFKA-2071
 Project: Kafka
  Issue Type: Sub-task
Reporter: Gwen Shapira
Assignee: David Jacot
 Fix For: 0.8.3


 Replace Produce Request/Response with their org.apache.kafka.common.requests 
 equivalents



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (KAFKA-2465) Need to document replica.fetcher.backoff.ms

2015-08-24 Thread Sriharsha Chintalapani (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani reassigned KAFKA-2465:
-

Assignee: Sriharsha Chintalapani

 Need to document replica.fetcher.backoff.ms
 ---

 Key: KAFKA-2465
 URL: https://issues.apache.org/jira/browse/KAFKA-2465
 Project: Kafka
  Issue Type: Improvement
Reporter: Gwen Shapira
Assignee: Sriharsha Chintalapani

 We added this parameter in KAFKA-1461, it changes existing behavior and is 
 configurable by users. 
 We should document the new behavior and the parameter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: KAFKA-2464; client-side assignment for new con...

2015-08-24 Thread hachikuji

GitHub user hachikuji opened a pull request:

https://github.com/apache/kafka/pull/165

KAFKA-2464; client-side assignment for new consumer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hachikuji/kafka KAFKA-2464

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/165.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #165


commit f6349cb9a3954464277ea3bd5ecfb80dcb8f4345
Author: Jason Gustafson ja...@confluent.io
Date:   2015-08-13T20:33:05Z

KAFKA-2464; client-side assignment for new consumer




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-2072) Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module


[ 
https://issues.apache.org/jira/browse/KAFKA-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710626#comment-14710626
 ] 

ASF GitHub Bot commented on KAFKA-2072:
---

Github user dajac closed the pull request at:

https://github.com/apache/kafka/pull/141


 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module
 --

 Key: KAFKA-2072
 URL: https://issues.apache.org/jira/browse/KAFKA-2072
 Project: Kafka
  Issue Type: Sub-task
Reporter: Gwen Shapira
Assignee: David Jacot
 Fix For: 0.8.3


 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: KAFKA-2072 [WIP]: Add StopReplica request/resp...

2015-08-24 Thread dajac

Github user dajac closed the pull request at:

https://github.com/apache/kafka/pull/141


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (KAFKA-2466) ConsoleConsumer throws ConcurrentModificationException on termination

2015-08-24 Thread Ashish K Singh (JIRA)

Ashish K Singh created KAFKA-2466:
-

 Summary: ConsoleConsumer throws ConcurrentModificationException on 
termination
 Key: KAFKA-2466
 URL: https://issues.apache.org/jira/browse/KAFKA-2466
 Project: Kafka
  Issue Type: Bug
  Components: tools
Reporter: Ashish K Singh
Assignee: Ashish K Singh


ConsoleConsumer throws ConcurrentModificationException on termination.

ST:
{code}
Exception in thread Thread-1 java.util.ConcurrentModificationException: 
KafkaConsumer is not safe for multi-threaded access
at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1169)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1087)
at kafka.consumer.NewShinyConsumer.close(BaseConsumer.scala:50)
at kafka.tools.ConsoleConsumer$$anon$1.run(ConsoleConsumer.scala:74)
{code}

Other thread which constantly tries to consume is
{code}
main prio=10 tid=0x7f3aa800c000 nid=0x1314 runnable [0x7f3aae37d000]
   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked 0xdd1df130 (a sun.nio.ch.Util$2)
- locked 0xdd1df120 (a java.util.Collections$UnmodifiableSet)
- locked 0xdd0af720 (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at org.apache.kafka.common.network.Selector.select(Selector.java:440)
at org.apache.kafka.common.network.Selector.poll(Selector.java:263)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:221)
at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:274)
at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:182)
at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:172)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:779)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:730)
at kafka.consumer.NewShinyConsumer.receive(BaseConsumer.scala:43)
at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:87)
at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:54)
at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:39)
at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (KAFKA-2072) Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module


 [ 
https://issues.apache.org/jira/browse/KAFKA-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on KAFKA-2072 started by David Jacot.
--
 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module
 --

 Key: KAFKA-2072
 URL: https://issues.apache.org/jira/browse/KAFKA-2072
 Project: Kafka
  Issue Type: Sub-task
Reporter: Gwen Shapira
Assignee: David Jacot
 Fix For: 0.8.3


 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2072) Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module


 [ 
https://issues.apache.org/jira/browse/KAFKA-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Jacot updated KAFKA-2072:
---
Status: Open  (was: Patch Available)

 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module
 --

 Key: KAFKA-2072
 URL: https://issues.apache.org/jira/browse/KAFKA-2072
 Project: Kafka
  Issue Type: Sub-task
Reporter: Gwen Shapira
Assignee: David Jacot
 Fix For: 0.8.3


 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2072) Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module


[ 
https://issues.apache.org/jira/browse/KAFKA-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710629#comment-14710629
 ] 

David Jacot commented on KAFKA-2072:


Moved it back to in progress.

 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module
 --

 Key: KAFKA-2072
 URL: https://issues.apache.org/jira/browse/KAFKA-2072
 Project: Kafka
  Issue Type: Sub-task
Reporter: Gwen Shapira
Assignee: David Jacot
 Fix For: 0.8.3


 Add StopReplica request/response to o.a.k.common.requests and replace the 
 usage in core module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-2467) ConsoleConsumer regressions