Re: [DISCUSS] KIPs

2015-02-05 Thread Harsha
Joel, Having only 2 or 3 KIPS under active discussion is concerning. This will slow down development process as well. Having a turn-around time for a KIP is a good idea but what will happen if it didn't received required votes within that time frame. Its probably more helpful for

[jira] [Commented] (KAFKA-1925) Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

2015-02-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308725#comment-14308725 ] Guozhang Wang commented on KAFKA-1925: -- Proposed solution: Change the INT_MAX to

[jira] [Created] (KAFKA-1925) Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

2015-02-05 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-1925: Summary: Coordinator Node Id set to INT_MAX breaks coordinator metadata updates Key: KAFKA-1925 URL: https://issues.apache.org/jira/browse/KAFKA-1925 Project: Kafka

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
I'm just thinking aloud - I don't know what a good number would be, and it is just one possibility to streamline how KIPs are processed. It largely depends on how complex the proposals are. What would be concerning is if there are 10 different threads all dealing with large KIPs and no one has the

[jira] [Comment Edited] (KAFKA-1925) Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

2015-02-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308725#comment-14308725 ] Guozhang Wang edited comment on KAFKA-1925 at 2/6/15 7:17 AM: --

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
Just wanted to add a few more comments on this: KIPs were suggested as a process to help reach early consensus on a major change or not so major (but tricky or backward incompatible) change in order to reduce the likelihood of multiple iterations and complete rewrites during code reviews (which is

[jira] [Commented] (KAFKA-1884) New Producer blocks forever for Invalid topic names

2015-02-05 Thread Pradeep Gollakota (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308670#comment-14308670 ] Pradeep Gollakota commented on KAFKA-1884: -- What makes the behavior in #2 earlier

[jira] [Work started] (KAFKA-1925) Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

2015-02-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-1925 started by Guozhang Wang. Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

[jira] [Updated] (KAFKA-1925) Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

2015-02-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1925: - Status: Patch Available (was: In Progress) Coordinator Node Id set to INT_MAX breaks

[jira] [Updated] (KAFKA-1925) Coordinator Node Id set to INT_MAX breaks coordinator metadata updates

2015-02-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1925: - Attachment: KAFKA-1925.v1.patch Reviewboard seems broken for now, uploading manually.

[jira] [Created] (KAFKA-1924) CI for Windows build

2015-02-05 Thread Jakob Homan (JIRA)
Jakob Homan created KAFKA-1924: -- Summary: CI for Windows build Key: KAFKA-1924 URL: https://issues.apache.org/jira/browse/KAFKA-1924 Project: Kafka Issue Type: Improvement Components:

[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

2015-02-05 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14307979#comment-14307979 ] Jakob Homan commented on KAFKA-1646: To make it easier to test future patches that

Re: [DISCUSS] KIPs

2015-02-05 Thread Jay Kreps
None on my part. -Jay On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy jjkosh...@gmail.com wrote: One amendment I would like to bring up for consideration wrt the KIP process (before we formally include it in our by-laws) is to not restrict the votes to be a lazy majority of the PMC, and to

Re: [ANNOUNCEMENT] Apache Kafka 0.8.2.0 Released

2015-02-05 Thread Otis Gospodnetic
Big thanks to Jun and everyone else involved! We're on 0.8.2 as of today. :) Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr Elasticsearch Support * http://sematext.com/ On Tue, Feb 3, 2015 at 8:37 PM, Jun Rao j...@confluent.io wrote: The Apache Kafka

Re: [DISCUSS] KIPs

2015-02-05 Thread Gwen Shapira
Isn't requiring 3 binding votes a bit overly strict here? We are talking about patches which in can be fixed, reverted, etc. Not releases, which have legal implications. Why not go with usual definition: lazy = No strong objections for few days? This means contributors will not be blocked on

Re: [DISCUSS] KIPs

2015-02-05 Thread Joe Stein
+1 On Thu, Feb 5, 2015 at 6:16 PM, Neha Narkhede n...@confluent.io wrote: Sounds good. On Thu, Feb 5, 2015 at 2:35 PM, Jay Kreps jay.kr...@gmail.com wrote: None on my part. -Jay On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy jjkosh...@gmail.com wrote: One amendment I would like

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
Sorry about this - I actually meant to suggest lazy consensus (which is a stronger requirement): 3 binding +1 votes and no binding vetoes. I have updated the wiki to lazy consensus - but can change it back if there is a reasonable objection. On Thu, Feb 05, 2015 at 06:17:44PM -0500, Joe Stein

Re: [DISCUSS] KIPs

2015-02-05 Thread Neha Narkhede
Sounds good. On Thu, Feb 5, 2015 at 2:35 PM, Jay Kreps jay.kr...@gmail.com wrote: None on my part. -Jay On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy jjkosh...@gmail.com wrote: One amendment I would like to bring up for consideration wrt the KIP process (before we formally include it in

Re: [DISCUSS] KIPs

2015-02-05 Thread Jay Kreps
Hey Joel, The problem with lazy consensus is that some people are too lazy. :-) I think the whole point of this is that you need to actively ensure people have read and understand the change before you proceed. I basically think all of us should be reading these proposals and giving feedback.

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
Yes - I realized that afterward. It is back to plain lazy majority. On Thu, Feb 05, 2015 at 04:33:48PM -0800, Jay Kreps wrote: Hey Joel, The problem with lazy consensus is that some people are too lazy. :-) I think the whole point of this is that you need to actively ensure people have read

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
The original requirement was lazy majority of PMC which definitely seems overly restrictive. Why not go with usual definition: lazy = No strong objections for few days? This means contributors will not be blocked on issues where no one cares to comment (and we had few of those). I think one

[jira] [Created] (KAFKA-1923) Negative offsets in replication-offset-checkpoint file

2015-02-05 Thread Oleg Golovin (JIRA)
Oleg Golovin created KAFKA-1923: --- Summary: Negative offsets in replication-offset-checkpoint file Key: KAFKA-1923 URL: https://issues.apache.org/jira/browse/KAFKA-1923 Project: Kafka Issue

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
This is exactly my concern. Even now important patches and features have very long development and review cycles due to Kafka's small and very busy committer community. I feel that this change takes things in the wrong direction I agree that we should improve on this. I think the only

Re: [DISCUSS] KIPs

2015-02-05 Thread Jiangjie Qin
I¹m having an impression that KIP is mostly for new features but not for bug fixes. But I agree with Joel that it might make sense to have some big patches, even if they are bug fixes, to follow the KIP like process which is more strict. Jiangjie (Becket) Qin On 2/5/15, 4:57 PM, Gwen Shapira

Re: [DISCUSS] KIPs

2015-02-05 Thread Gwen Shapira
Yes there are KIPs that are currently blocked on feedback/votes, but I don't think it is an issue of not caring to comment vs having so many KIPs and other code reviews in flight that people are just swamped. This is exactly my concern. Even now important patches and features have very long

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-02-05 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14307057#comment-14307057 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard

Re: Review Request 29831: Patch for KAFKA-1476

2015-02-05 Thread Onur Karaman
On Feb. 2, 2015, 5:42 p.m., Neha Narkhede wrote: Onur, we should be able to check in after these review comments are addressed. Also, how would deleting offsets for a group work when the offset storage is Kafka? It's fine to not address it in this patch. Can you please create a JIRA

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-02-05 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: sample-kafka-consumer-groups-sh-output-2-5-2015.txt Get a list of consumer groups

Re: [ANNOUNCEMENT] Apache Kafka 0.8.2.0 Released

2015-02-05 Thread Steve Morin
Congratz team it's a big accomplishment On Feb 5, 2015, at 14:22, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Big thanks to Jun and everyone else involved! We're on 0.8.2 as of today. :) Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr

[jira] [Commented] (KAFKA-1884) New Producer blocks forever for Invalid topic names

2015-02-05 Thread Pradeep Gollakota (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308539#comment-14308539 ] Pradeep Gollakota commented on KAFKA-1884: -- I'd like to work on this. Please

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-02-05 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-02-05_03:01:09.patch Get a list of consumer groups

Re: Review Request 29831: Patch for KAFKA-1476

2015-02-05 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Feb. 5, 2015, 11:01 a.m.) Review request for kafka. Bugs:

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
One amendment I would like to bring up for consideration wrt the KIP process (before we formally include it in our by-laws) is to not restrict the votes to be a lazy majority of the PMC, and to instead make it a lazy majority of committers. Our current requirement for code changes per our by-laws