[jira] [Commented] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738406#comment-14738406 ] Hari Sekhon commented on KAFKA-1397: I've got this same issue with 0.8.2.2 and there are no consumers

[jira] [Commented] (KAFKA-2181) online doc issues

[ https://issues.apache.org/jira/browse/KAFKA-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738565#comment-14738565 ] Bobby Tang commented on KAFKA-2181: --- I'm interested in this one, but I can't assign JIRA ticket to

Re: Maybe 0.8.3 should really be 0.9.0?

+1 for 0.9 On Wed, Sep 9, 2015 at 2:20 AM, Stevo Slavić wrote: > +1 (non-binding) for 0.9 > > On Wed, Sep 9, 2015 at 6:41 AM, Jun Rao wrote: > > > +1 for 0.9. > > > > Thanks, > > > > Jun > > > > On Tue, Sep 8, 2015 at 3:04 PM, Ismael Juma

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738890#comment-14738890 ] Mayuresh Gharat commented on KAFKA-2120: Hi [~junrao], Thanks for the comments. I have uploaded a

Re: Maybe 0.8.3 should really be 0.9.0?

are we going to deem the new consumer in 0.9.0 as beta? Do we want to-do a 0.9.0-beta and this way when the consumer is g2g we 0.9.0.0 0.9.0-beta also allows us to release a lot of new things a bit sooner and have some good cycles of fixes (because you know they will come) There is enough new

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739003#comment-14739003 ] Aditya Auradkar commented on KAFKA-2500: [~hachikuji] - Are you planning on supporting this in the

[jira] [Commented] (KAFKA-2447) Add capability to KafkaLog4jAppender to be able to use SSL

[ https://issues.apache.org/jira/browse/KAFKA-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739128#comment-14739128 ] Ashish K Singh commented on KAFKA-2447: --- [~gwenshap] perfectly makes sense! Created KAFKA-2531. >

[jira] [Updated] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2528: Description: In this document we present the results of experiments we did at LinkedIn, to validate the

Re: Maybe 0.8.3 should really be 0.9.0?

Jun, Makes sense, thanks! ~ Joestein On Sep 10, 2015 1:05 PM, "Jun Rao" wrote: > Hi, Joe, > > One of the reasons that we have been doing beta releases before is to > stabilize the public apis. However, in trunk, we have introduced the api > stability annotation. The new java

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 9, 2015, 4:43 a.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/Selector.java, lines > > 185-190 > > > > > > Do we still need disconnect(id)? It seems that we can just

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739059#comment-14739059 ] Will Funnell commented on KAFKA-2500: - If it helps understand our use case, this is the API we have

[jira] [Comment Edited] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739059#comment-14739059 ] Will Funnell edited comment on KAFKA-2500 at 9/10/15 4:26 PM: -- If it helps

[jira] [Created] (KAFKA-2530) metrics for old replica fetcher thread need to be deregistered

Jun Rao created KAFKA-2530: -- Summary: metrics for old replica fetcher thread need to be deregistered Key: KAFKA-2530 URL: https://issues.apache.org/jira/browse/KAFKA-2530 Project: Kafka Issue

[jira] [Resolved] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-2528. - Resolution: Fixed > Quota Performance Evaluation > > > Key:

[jira] [Work started] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2528 started by Dong Lin. --- > Quota Performance Evaluation > > > Key:

[jira] [Updated] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2528: Attachment: QuotaPerformanceEvaluation.pdf > Quota Performance Evaluation > >

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739246#comment-14739246 ] Ewen Cheslack-Postava commented on KAFKA-2500: -- [~hachikuji] The only drawback to pushing it

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739257#comment-14739257 ] Jason Gustafson commented on KAFKA-2500: [~ewencp] Would exposing the high watermarks from the

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739378#comment-14739378 ] Ewen Cheslack-Postava commented on KAFKA-2500: -- [~hachikuji] Yeah, sorry, that is what I

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739223#comment-14739223 ] Jason Gustafson commented on KAFKA-2500: [~willf] If I understand correctly, the basic requirement

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739235#comment-14739235 ] Will Funnell commented on KAFKA-2500: - [~hachikuji] yes, beyond the current cleaner point would be

Re: Review Request 36858: Patch for KAFKA-2120

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/#review98438 ---

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739324#comment-14739324 ] Aditya Auradkar commented on KAFKA-2419: Thanks [~ijuma] I'll take a look today. If I indeed can

[jira] [Comment Edited] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739660#comment-14739660 ] Jay Kreps edited comment on KAFKA-2528 at 9/10/15 9:44 PM: --- Yeah, sorry separate

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

Folks, I would like to revive this thread on KIP-28: I have just updated the patch rebased on latest trunk incorporating the feedbacks collected so far: https://github.com/apache/kafka/pull/130 And the wiki page for this KIP has also been updated with the API and architectural designs:

[jira] [Commented] (KAFKA-2532) Remove Consumer from rebalance callback arguments

[ https://issues.apache.org/jira/browse/KAFKA-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739911#comment-14739911 ] ASF GitHub Bot commented on KAFKA-2532: --- GitHub user hachikuji opened a pull request:

[GitHub] kafka pull request: KAFKA-2532; Remove Consumer reference from reb...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/203 KAFKA-2532; Remove Consumer reference from rebalance callback You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-2532

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 10, 2015, 6:19 p.m., Ismael Juma wrote: > > clients/src/main/java/org/apache/kafka/common/network/Selector.java, line > > 488 > > > > > > Sorry if this has already been covered before, but why are we

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739886#comment-14739886 ] Dong Lin commented on KAFKA-2528: - Yeah I think that is a reasonable expectation. We have done something

[jira] [Reopened] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reopened KAFKA-2528: - Explain the difference between broker throughput and quota when quota = 1 MBps > Quota Performance

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

Hi Jay, I just copy/pastes here your feedback on the timestamp proposal that was in the discussion thread of KIP-31. Please see the replies inline. The main change I made compared with previous proposal is to add both CreateTime and LogAppendTime to the message. On Tue, Sep 8, 2015 at 10:57 AM,

[jira] [Commented] (KAFKA-2529) Brokers should write current version to log when they first start

[ https://issues.apache.org/jira/browse/KAFKA-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739774#comment-14739774 ] Joel Koshy commented on KAFKA-2529: --- Yes this was done in KAFKA-1901, but one issue is that the above is

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739766#comment-14739766 ] Joel Koshy commented on KAFKA-2120: --- Thanks for the updated patch - I will review again later today >

Re: [DISCUSS] KIP-31 - Message format change proposal

Hi Jay, I just changed the KIP title and updated the KIP page. And yes, we are working on a general version control proposal to make the protocol migration like this more smooth. I will also create a KIP for that soon. Thanks, Jiangjie (Becket) Qin On Thu, Sep 10, 2015 at 2:21 PM, Jay Kreps

[jira] [Updated] (KAFKA-2532) Remove Consumer from rebalance callback arguments

[ https://issues.apache.org/jira/browse/KAFKA-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2532: --- Status: Patch Available (was: Open) > Remove Consumer from rebalance callback arguments >

Re: [DISCUSS] KIP-31 - Message format change proposal

Ah, I see, I think I misunderstood about MM, it was called out in the proposal and I thought you were saying you'd retain the timestamp but I think you're calling out that you're not. In that case you do have the opposite problem, right? When you add mirroring for a topic all that data will have a

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739660#comment-14739660 ] Jay Kreps commented on KAFKA-2528: -- Yeah, sorry separate thought. What I'm trying to figure out is how

[jira] [Updated] (KAFKA-2512) Add version and magic byte verification on broker and clients.

[ https://issues.apache.org/jira/browse/KAFKA-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2512: Status: Patch Available (was: Open) > Add version and magic byte verification on broker and

Re: [DISCUSS] KIP-31 - Message format change proposal

I just wanted to comment on a few points made earlier in this thread: Concerns on clock skew: at least for the original proposal's scope (which was more for honoring retention broker-side) this would only be an issue when spanning leader movements right? i.e., leader migration latency has to be

[jira] [Created] (KAFKA-2533) Create a member Metadata.Listener inside KafkaConsumer

Ashish K Singh created KAFKA-2533: - Summary: Create a member Metadata.Listener inside KafkaConsumer Key: KAFKA-2533 URL: https://issues.apache.org/jira/browse/KAFKA-2533 Project: Kafka Issue

Re: Review Request 36858: Patch for KAFKA-2120

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/#review98526 ---

Re: [DISCUSS] KIP-31 - Message format change proposal

Hey Joel, Yeah the clock issues would arrive when leadership changed or the time on the machine changed. Don't you see all the same issues you see with client-defined timestamp's if you let mm control the timestamp as you were proposing? That means time is no longer monotonic. A mixture of

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739917#comment-14739917 ] Jay Kreps commented on KAFKA-2528: -- Yeah that makes sense. So if that were the case the artifact would go

Re: Review Request 36858: Patch for KAFKA-2120

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/#review98451 --- Ship it! Thanks for the patch. Just a minor comment below.

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739899#comment-14739899 ] Aditya Auradkar commented on KAFKA-2528: One possible explanation for the difference is that we

Re: [DISCUSS] KIP-31 - Message format change proposal

> Don't you see all the same issues you see with client-defined timestamp's > if you let mm control the timestamp as you were proposing? That means time Actually I don't think that was in the proposal (or was it?). i.e., I think it was always supposed to be controlled by the broker (and not MM).

Re: [DISCUSS] KIP-31 - Message format change proposal

Re: MM preserving timestamps: Yes, this was how I interpreted the point in the KIP and I only raised the issue because it restricts the usefulness of timestamps anytime MM is involved. I agree it's not a deal breaker, but I wanted to understand exact impact of the change. Some users seem to want

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 10, 2015, 6:19 p.m., Ismael Juma wrote: > > clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java, line > > 137 > > > > > > Out of curiosity, why are we using `LinkedList` here? Generally

[GitHub] kafka pull request: Kafka 2477

GitHub user becketqin opened a pull request: https://github.com/apache/kafka/pull/204 Kafka 2477 Tried two fixes. I prefer the second approach because it saves an additional offset search. You can merge this pull request into a Git repository by running: $ git pull

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 11, 2015, 1:44 a.m., Jason Gustafson wrote: > > clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java, line > > 140 > > > > > > This line is puzzling me a little bit. Wouldn't we want to

Re: Review Request 36858: Patch for KAFKA-2120

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/ --- (Updated Sept. 11, 2015, 4:39 a.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-2120) Add a request timeout to NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-2120: --- Attachment: KAFKA-2120_2015-09-10_21:38:55.patch > Add a request timeout to NetworkClient >

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740166#comment-14740166 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739420#comment-14739420 ] Jay Kreps commented on KAFKA-2528: -- This is a great evaluation. It looks from the results like there must

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739493#comment-14739493 ] Jay Kreps commented on KAFKA-2528: -- I mention it because I have had to implement that kind of throttling

[jira] [Updated] (KAFKA-1893) Allow regex subscriptions in the new consumer

[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1893: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-1893) Allow regex subscriptions in the new consumer

[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739513#comment-14739513 ] ASF GitHub Bot commented on KAFKA-1893: --- Github user asfgit closed the pull request at:

[jira] [Comment Edited] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739424#comment-14739424 ] Will Funnell edited comment on KAFKA-2500 at 9/10/15 7:27 PM: -- [~hachikuji]

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739424#comment-14739424 ] Will Funnell commented on KAFKA-2500: - [~jasonxh] What is the complexity of exposing the current

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739438#comment-14739438 ] Aditya Auradkar commented on KAFKA-2528: I'm not quite sure why the actual rate is higher in this

Re: [DISCUSS] KIP-31 - Message format change proposal

Neha and Jay, Thanks a lot for the feedback. Good point about splitting the discussion. I have split the proposal to three KIPs and it does make each discussion more clear: KIP-31 - Message format change (Use relative offset) KIP-32 - Add CreateTime and LogAppendTime to Kafka message KIP-33 -

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739488#comment-14739488 ] Dong Lin commented on KAFKA-2528: - I think this difference may be explained by the fact that

[jira] [Comment Edited] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739505#comment-14739505 ] Dong Lin edited comment on KAFKA-2528 at 9/10/15 8:07 PM: -- Yeah I think it is

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739505#comment-14739505 ] Dong Lin commented on KAFKA-2528: - Yeah I think it is certainly doable, i.e. we can get very close to the

[GitHub] kafka pull request: KAFKA-1893: Allow regex subscriptions in the n...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/128 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739585#comment-14739585 ] Jay Kreps commented on KAFKA-2528: -- I guess my expectation is that if I run a producer perf test or

[jira] [Created] (KAFKA-2532) Remove Consumer from rebalance callback arguments

Jason Gustafson created KAFKA-2532: -- Summary: Remove Consumer from rebalance callback arguments Key: KAFKA-2532 URL: https://issues.apache.org/jira/browse/KAFKA-2532 Project: Kafka Issue

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739612#comment-14739612 ] Dong Lin commented on KAFKA-2528: - I am a bit confused.. I thought you were referring to the first

Re: [DISCUSS] KIP-31 - Message format change proposal

Great, can we change the name to something related to the change--"KIP-31: Move to relative offsets in compressed message sets". Also you had mentioned before you were going to expand on the mechanics of handling these log format changes, right? -Jay On Thu, Sep 10, 2015 at 12:42 PM, Jiangjie

Re: Review Request 36858: Patch for KAFKA-2120

> On July 27, 2015, 10:55 p.m., Jason Gustafson wrote: > > clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java, line > > 48 > > > > > > It seems like it might be more natural to set this in > >