Re: [KIP-DISCUSSION] KIP-13 Quotas

make. > > if broker just drop the msg and return an error/status code indicates the > drop and why. then producer can just move on and accept the drop. shared > buffer won't be saturated and other 9 topics won't be penalized. > > Thanks, > Steven > > >

[jira] [Commented] (KAFKA-1912) Create a simple request re-routing facility

[ https://issues.apache.org/jira/browse/KAFKA-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365766#comment-14365766 ] Jay Kreps commented on KAFKA-1912: -- I think this is easier than I thought. We hav

Re: [KIP-DISCUSSION] KIP-13 Quotas

; >>> else if(error.isFatal()) > > > >>> // non-retriable errors, etc: notify / terminate / other > > handling > > > >>> > > > >>> - > > > >>> > > > >>> And for implementation

Re: [KIP-DISCUSSION] KIP-13 Quotas

; e...@confluent.io> > >>> wrote: > >>> > >>> > Agreed that trying to shoehorn non-error codes into the error field > is > >>> a > >>> > bad idea. It makes it *way* too easy to write code that looks (and > >>> should

Re: [KIP-DISCUSSION] KIP-13 Quotas

for "non-error" codes. > > It won't be backward compatible (i.e. clients that currently do "else > throw" will throw on non-errors), but perhaps its worthwhile. > > On Mon, Mar 16, 2015 at 9:42 PM, Jay Kreps wrote: > > Hey Jun, > > > > I'd r

Re: [KIP-DISCUSSION] KIP-13 Quotas

Hey Jun, I'd really really really like to avoid that. Having just spent a bunch of time on the clients, using the error codes to encode other information about the response is super dangerous. The error handling is one of the hardest parts of the client (Guozhang chime in here). Generally the err

Re: Question about concurrency during Log config change

You are correct. Each read will be a valid value but there is no guarantee that subsequent reads will read from the same config. I don't think that is a problem, do you? If we want to strengthen the guarantee we can grab the config once in the method val config = log.config and then do however m

Re: [DISCUSSION] KIP-15 close(timeout) for producer

der thread so user will notice something went > wrong. > > Thanks. > > Jiangjie (Becket) Qin > > On 3/14/15, 11:37 AM, "Jay Kreps" wrote: > > >Hey Jiangjie, > > > >I think this is going to be very confusing that > > close(0) waits indefinitel

Re: [DISCUSSION] KIP-15 close(timeout) for producer

vide implementation details. > > Thanks again for all the comments and suggestions! > > Jiangjie (Becket) Qin > > On 3/10/15, 8:57 PM, "Jiangjie Qin" wrote: > > >The KIP page has been updated per Jay¹s comments. > >I¹d like to initiate the voting proces

[jira] [Commented] (KAFKA-2020) I expect ReplicaNotAvailableException to have proper Javadocs

[ https://issues.apache.org/jira/browse/KAFKA-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361952#comment-14361952 ] Jay Kreps commented on KAFKA-2020: -- Yeah we need to just fix that. I actually stro

Re: [VOTE] KIP-16: Replica Lag Tuning

+1 -Jay On Fri, Mar 13, 2015 at 9:54 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Details in the KIP, Jira and RB. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning > https://issues.apache.org/jira/browse/KAFKA-1546 > https://reviews.

[jira] [Commented] (KAFKA-2020) I expect ReplicaNotAvailableException to have proper Javadocs

[ https://issues.apache.org/jira/browse/KAFKA-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361868#comment-14361868 ] Jay Kreps commented on KAFKA-2020: -- Is this the case where we are sending back an e

Re: [DISCUSSION] KIP-15 close(timeout) for producer

tes to complete all the requests. But when we close a > >>> producer, we might not want to wait for that long as it might cause > >>>some > >>> other problem like deployment tool timeout. > >>> > >>> There is also a subtle difference betwee

[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359829#comment-14359829 ] Jay Kreps commented on KAFKA-1646: -- Yeah I was proposing doing this across the b

Re: [KIP-DISCUSSION] KIP-13 Quotas

bols are allowed? > Why didn't they just tell you all that up front so you could get it right > the first time? > > -Todd > > > On Wed, Mar 11, 2015 at 10:22 PM, Jay Kreps wrote: > > > Hey Todd, > > > > Yeah it is kind of weird to do the quota check after t

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

> > > request, etc. With admin requests we could add a flag to enable / > > > disable > > > > > synchronous requests; when it is turned on, the response will not > > > return > > > > > until the request has been completed

[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358125#comment-14358125 ] Jay Kreps commented on KAFKA-1646: -- Hey [~waldenchen] this patch is adding a TO

[jira] [Updated] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1646: - Reviewer: (was: Jay Kreps) > Improve consumer read performance for Wind

Re: [KIP-DISCUSSION] KIP-13 Quotas

a minimum timeout > allowed would make the delay more palatable. > > I'm still not sure it's the right solution, and that we're not just going > with what's fast and cheap as opposed to what is good (or right). But given > the details of where to hold the

Re: [KIP-DISCUSSION] KIP-13 Quotas

will only be delayed for another 3 seconds. Is there a reason > this isn't good enough? I'm also going to dig into the existing quota > package and see what we can leverage. > > 7. Exposing quota usage: We do plan to expose the quota metrics via JMX. > Adding an API to

Re: 0.8.3 release plan

With regard to mm, I was kind of assuming just based on the amount of work that that would go in for sure, but yeah I agree it is important. -Jay On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps wrote: > What I was trying to say was let's do a real release whenever either > consumer or au

Re: 0.8.3 release plan

wed sample). > > Gwen > > > > On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps wrote: > > Yeah the real question is always what will we block on? > > > > I don't think we should try to hold back smaller changes. In this bucket > I > > would include

Re: 0.8.3 release plan

a nice incremental > improvement. > > Gwen > > > On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps wrote: > > Yeah I'd be in favor of a quicker, smaller release but I think as long as > > we have these big things in flight we should probably keep the release > >

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358063#comment-14358063 ] Jay Kreps commented on KAFKA-1546: -- Well iiuc the wonderfulness of this feature is

Re: [VOTE] KIP-15 add a close method with timeout to KafkaProducer

Yeah guys, I'd like to second that. I'd really really love to get the quality of these to the point where we could broadly solicit user input and use them as a permanent document of the alternatives and rationale. I know it is a little painful to have process, but I think we all saw what happened

Re: 0.8.3 release plan

Yeah I'd be in favor of a quicker, smaller release but I think as long as we have these big things in flight we should probably keep the release criteria feature-based rather than time-based, though (e.g. "when X works" not "every other month). Ideally the next release would have at least a "beta"

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358039#comment-14358039 ] Jay Kreps commented on KAFKA-1546: -- Personally I don't think it really needs

[jira] [Commented] (KAFKA-1930) Move server over to new metrics library

[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354376#comment-14354376 ] Jay Kreps commented on KAFKA-1930: -- Hey [~gwenshap] this is wheel reinvention

Re: [KIP-DISCUSSION] KIP-13 Quotas

tas, and it will allow for emitting metrics on the client side for quota > status (rather than relying on the server-side metrics, which tends to put > the responsibility in the wrong place). > > > -Todd > > > On Sun, Mar 8, 2015 at 10:52 AM, Jay Kreps wrote: > > &

[jira] [Updated] (KAFKA-2009) Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker

[ https://issues.apache.org/jira/browse/KAFKA-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-2009: - Resolution: Fixed Reviewer: Jay Kreps Status: Resolved (was: Patch Available) Committed

[jira] [Commented] (KAFKA-1313) Support adding replicas to existing topic partitions

[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352246#comment-14352246 ] Jay Kreps commented on KAFKA-1313: -- This is actually in the alter topic command t

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-527: Assignee: Yasuhiro Matsuda (was: Jay Kreps) > Compression support does numerous byte cop

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352227#comment-14352227 ] Jay Kreps commented on KAFKA-527: - The clients already use MemoryRecords, so 0.8.2

[jira] [Assigned] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-527: --- Assignee: Jay Kreps (was: Yasuhiro Matsuda) > Compression support does numerous byte cop

Re: TL;DR; of the first KIP hangout

I took a pass on these and tried to update our summary table based on Gwen's summary. There are a couple that are just waiting on the owner to hold their vote. There are a number that are still very sparse on detail. I really want to encourage everyone to take a look at the python PEPs, I think th

Re: [KIP-DISCUSSION] KIP-13 Quotas

Hey Adi, Great write-up. Here are some comments: 1. I don't think you need a way to disable quotas on a per-client basis, that is just the equivalent of setting the quota to be infinite, right? 2. I agree that the configuration problem is a general part of doing dynamic configuration, and it is

Re: [DISCUSS] KIP-6 - New reassignment partition logic for re-balancing

Hey Joe, This still seems pretty incomplete. It still has most the sections just containing the default text you are supposed to replace. It is really hard to understand what is being proposed and why and how much of the problem we are addressing. For example the motivation section just says "oper

Re: [DISCUSSION] KIP-15 close(timeout) for producer

Hey Jiangjie, Can you capture the full motivation and use cases for the feature? This mentions your interest in having a way of aborting from inside the Callback. But it doesn't really explain that usage or why other people would want to do that. It also doesn't list the primary use case for havin

Re: Review Request 31650: Drag Coordinator and FetchManager out of KafkaConsumer, fix a bunch of consumer test issues

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31650/#review75419 --- Ship it! Ship It! - Jay Kreps On March 5, 2015, 10:57 p.m

Re: Review Request 31650: Drag Coordinator and FetchManager out of KafkaConsumer, fix a bunch of consumer test issues

you should consider just checking in now and we can do any additional items as follow-ups. - Jay Kreps On March 5, 2015, 10:57 p.m., Guozhang Wang wrote: > > --- > This is an automatically generated e-mail. To reply,

[jira] [Commented] (KAFKA-1998) Partitions Missing From MetadataResponse

[ https://issues.apache.org/jira/browse/KAFKA-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347740#comment-14347740 ] Jay Kreps commented on KAFKA-1998: -- This seems like a bug, shouldn't we alwa

Re: TL;DR; of the first KIP hangout

Thanks Gwen! -Jay On Tue, Mar 3, 2015 at 12:12 PM, Gwen Shapira wrote: > Hi, > > I put together a (very) short summary of the discussion and decisions: > > KIPs: > We reviewed the list of KIPs posted here: > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > * K

[jira] [Resolved] (KAFKA-789) Producer-side persistence for delivery guarantee

[ https://issues.apache.org/jira/browse/KAFKA-789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-789. - Resolution: Duplicate > Producer-side persistence for delivery guaran

Re: Review Request 29467: Patch for KAFKA-1660

/KafkaProducer.java <https://reviews.apache.org/r/29467/#comment121725> Now there is a bit of duplicate code between the two close methods. Maybe this would be cleaner if we just made public void close() { close(Long.MAX_VALUE, TimeUnit.MILLISECONDS); } - Jay Kreps On M

Re: Review Request 29467: Patch for KAFKA-1660

> On March 2, 2015, 11:04 p.m., Jiangjie Qin wrote: > > clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java, > > line 219 > > > > > > We probably need to release the caller threads that are w

Re: Review Request 29467: Patch for KAFKA-1660

On March 3, 2015, 4:10 a.m., Parth Brahmbhatt wrote: > > Two minor changes I noted, but otherwise looks good to me. Needs some unit > > tests, as you mentioned. Actually one probably I didn't think of is that forceClose() leaves the in-flight requests forever incomplete. A better approach woul

Re: Review Request 31566: Patch for KAFKA-1988

scala code to all use this single abs function. - Jay Kreps On Feb. 27, 2015, 11:16 p.m., Tong Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

[jira] [Commented] (KAFKA-902) Randomize backoff on the clients for metadata requests

[ https://issues.apache.org/jira/browse/KAFKA-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344500#comment-14344500 ] Jay Kreps commented on KAFKA-902: - This looks good to me. I'd second Grant'

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344481#comment-14344481 ] Jay Kreps commented on KAFKA-1660: -- [~parth.brahmbhatt] Yeah this was exactly what I

Re: Review Request 29467: Patch for KAFKA-1660

changes I noted, but otherwise looks good to me. Needs some unit tests, as you mentioned. - Jay Kreps On March 2, 2015, 6:41 p.m., Parth Brahmbhatt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > ht

Re: Unit tests in java7 vs java8

uozhang Wang > > wrote: > > > I am working on the test handing / NPE / failure issues of ConsumerTest > > only. > > > > I currently run Java 8 and the unit test takes about 10 minutes, I can do > > "time ./gradlew test" with both versions and see if

[jira] [Updated] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Resolution: Fixed Status: Resolved (was: Patch Available) > Add a flush() call to the

Re: Review Request 30763: KAFKA-1865: add flush to producer

> On Feb. 26, 2015, 7:15 p.m., Jun Rao wrote: > > clients/src/test/java/org/apache/kafka/clients/producer/MetadataTest.java, > > line 87 > > <https://reviews.apache.org/r/30763/diff/6/?file=878560#file878560line87> > > > > Do we really want to prin

[jira] [Created] (KAFKA-1990) Add unlimited time-based log retention

Jay Kreps created KAFKA-1990: Summary: Add unlimited time-based log retention Key: KAFKA-1990 URL: https://issues.apache.org/jira/browse/KAFKA-1990 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-1877) Expose version via JMX for 'new' producer

[ https://issues.apache.org/jira/browse/KAFKA-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341744#comment-14341744 ] Jay Kreps commented on KAFKA-1877: -- The metrics are just for metrics which are by na

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

(Becket) Qin > > On 2/26/15, 8:40 PM, "Jay Kreps" wrote: > > >Can we change the command line arguments for mm to match the command line > >arguments elsewhere. This proposal seems to have two formats: > >*--consumer.rebalance.listener* > >and > >*--

[jira] [Commented] (KAFKA-330) Add delete topic support

[ https://issues.apache.org/jira/browse/KAFKA-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14340767#comment-14340767 ] Jay Kreps commented on KAFKA-330: - +1 on enabling by default. > Add delete topic

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

t; > >> >1) Ability to publish message from src topic to different destination > >> >topic > >> >via --overidenTopics=srcTopic:newDestinationTopic > >> > > >> >In order to adopt, new MM enhancement customer will compare > >>perform

Re: Review Request 30763: KAFKA-1865: add flush to producer

reviews.apache.org/r/30763/#review74332 ------- On Feb. 26, 2015, 6:40 p.m., Jay Kreps wrote: > > --- > This is an automatically generated e-mail. To re

Re: Review Request 30763: KAFKA-1865: add flush to producer

Diff: https://reviews.apache.org/r/30763/diff/ Testing (updated) --- The latest patch uses Jiangjie's suggestion to remove the synchronization on flush. Fixed the boolean logic in the unit test as per Jun's suggestion. Thanks, Jay Kreps

[jira] [Updated] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Attachment: KAFKA-1865_2015-02-26_10:37:16.patch > Add a flush() call to the new producer

[jira] [Commented] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338880#comment-14338880 ] Jay Kreps commented on KAFKA-1865: -- Updated reviewboard https://reviews.apache.o

Re: Review Request 30763: KAFKA-1865: add flush to producer

it is kind of waste to create duplicate RecordMetadata > > here and in done().. > > Jay Kreps wrote: > I take your point, but I tried the refactor to remove it and it wasn't > very simple plus I think it would be quite rare that you would ever actually > invoke both

Re: Review Request 30763: KAFKA-1865: add flush to producer

/scala/unit/kafka/utils/TestUtils.scala 21d0ed2cb7c9459261d3cdc7c21dece5e2079698 Diff: https://reviews.apache.org/r/30763/diff/ Testing --- The latest patch uses Jiangjie's suggestion to remove the synchronization on flush. Thanks, Jay Kreps

Re: Review Request 30763: KAFKA-1865: Producer Flush: Remove synchronization.

-- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30763/#review74229 ------- On Feb. 26, 2015, 1:16 a.m., Jay Kreps wrote: > > --

Re: Review Request 30763: KAFKA-1865: add flush to producer

t/scala/integration/kafka/api/ProducerSendTest.scala > b15237b76def3b234924280fa3fdb25dbb0cc0dc > core/src/test/scala/unit/kafka/utils/TestUtils.scala > 21d0ed2cb7c9459261d3cdc7c21dece5e2079698 > > Diff: https://reviews.apache.org/r/30763/diff/ > > > Testing > --- > > The latest patch uses Jiangjie's suggestion to remove the synchronization on > flush. > > > Thanks, > > Jay Kreps > >

Re: Review Request 30763: KAFKA-1865: Producer Flush: Remove synchronization.

automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30763/#review74196 --- On Feb. 26, 2015, 1:16 a.m., Jay Kreps wrote: > > ---

Re: Review Request 30763: KAFKA-1865: Producer Flush: Remove synchronization.

Diff: https://reviews.apache.org/r/30763/diff/ Testing (updated) --- The latest patch uses Jiangjie's suggestion to remove the synchronization on flush. Thanks, Jay Kreps

[jira] [Commented] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337636#comment-14337636 ] Jay Kreps commented on KAFKA-1865: -- Updated reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Attachment: KAFKA-1865_2015-02-25_17:15:26.patch > Add a flush() call to the new producer

Re: Review Request 30763: KAFKA-1865: Producer Flush: Remove synchronization.

unit tests I found that were in the wrong packages. Thanks, Jay Kreps

Re: Review Request 30763: KAFKA-1865: Producer Flush: if at first your patch doesn't succeed, try, try, try again.

optimized out anyway... - Jay --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30763/#review73829 --- On Feb. 24, 2015, 2:31 a.m., Jay

[jira] [Commented] (KAFKA-1910) Refactor KafkaConsumer

[ https://issues.apache.org/jira/browse/KAFKA-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337621#comment-14337621 ] Jay Kreps commented on KAFKA-1910: -- Where is the RB? > Refactor KafkaC

Re: Unit tests in java7 vs java8

Yeah, hey Guozhang, is that fix part of the larger consumer patch you just posted or is that a separate issue? -Jay On Wed, Feb 25, 2015 at 4:25 PM, Gwen Shapira wrote: > The Consumer tests are currently hanging :( > > I think Guozhang is working on a solution. I'm commenting them out until > t

[jira] [Commented] (KAFKA-1985) Document on possible error codes for each response type

[ https://issues.apache.org/jira/browse/KAFKA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337214#comment-14337214 ] Jay Kreps commented on KAFKA-1985: -- Totally agree. The documentation for this sh

Tips for working with Kafka and data streams

Hey guys, One thing we tried to do along with the product release was start to put together a practical guide for using Kafka. I wrote this up here: http://blog.confluent.io/2015/02/25/stream-data-platform-1/ I'd like to keep expanding on this as good practices emerge and we learn more stuff. So

Re: [VOTE] KIP-8: Add a flush() method to the new producer

015 at 6:29 AM, Joel Koshy wrote: > > > >> +1 (binding) > >> > >> On Wed, Feb 18, 2015 at 07:03:26PM -0500, Joe Stein wrote: > >> > +1 binding > >> > > >> > ~ Joestein > >> > On Feb 18, 2015 6:50 PM, "Jay Kreps&qu

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

periment and I'm a +1 on everything else, ncluding the > >rebalance callback and record handler. > > > >-Neha > > > >On Wed, Feb 11, 2015 at 1:13 PM, Jay Kreps wrote: > > > >> Cool, I agree with all that. > >> > >> I agree about the need f

Re: Review Request 31367: Patch for kafka-1984

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31367/#review73910 --- Ship it! Ship It! - Jay Kreps On Feb. 24, 2015, 6:16 p.m., Jun

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

t and publish them out. > > Thanks > > Jeff > > On Tue, Feb 24, 2015 at 11:28 AM, Jay Kreps wrote: > > > Let's try to get the technical hang-ups sorted out, though. I really > think > > there is some benefit to live discussion vs writing. I am hopeful that i

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

it > goes? We can pull transcripts too and associate to tickets if need be makes > it helpful for things. > > ~ Joestein > > On Tue, Feb 24, 2015 at 11:10 AM, Jay Kreps wrote: > > > We'd talked about doing a Google Hangout to chat about this. What about > > generalizi

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

, I agree my proposal is not consistent with other cases. > > Maybe we can find better solution or something in-between. > > > > Re Hangout chat: I think it is a great idea. This way we can move on > > faster. > > Let's agree somehow on date/time so people can j

Re: Review Request 30763: KAFKA-1865: Producer Flush: if at first your patch doesn't succeed, try, try, try again.

, 2015, 4:22 a.m., Jiangjie Qin wrote: > > > LGTM. Thanks, Jay. > > > I actually tried just putting a synchronized block around the line > where we copy the imcomplete set and it seems worked. Maybe we can do that > if you prefer less code. > > > > Jay Kreps w

Re: Review Request 30763: KAFKA-1865: Producer Flush: if at first your patch doesn't succeed, try, try, try again.

-- On Feb. 24, 2015, 2:31 a.m., Jay Kreps wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30763/ > ---

Re: Review Request 30763: KAFKA-1865: Producer Flush: if at first your patch doesn't succeed, try, try, try again.

Diff: https://reviews.apache.org/r/30763/diff/ Testing (updated) --- New patch addresses feedback. Also (1) comments out the consumer tests so I could verify everything else passes and (2) moves some unit tests I found that were in the wrong packages. Thanks, Jay Kreps

[jira] [Updated] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Attachment: KAFKA-1865_2015-02-23_18:29:16.patch > Add a flush() call to the new producer

[jira] [Commented] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14334281#comment-14334281 ] Jay Kreps commented on KAFKA-1865: -- Updated reviewboard https://reviews.apache.o

Re: Review Request 30763: KAFKA-1865: Producer Flush: if at first your patch doesn't succeed, try, try, try again.

scala b15237b76def3b234924280fa3fdb25dbb0cc0dc core/src/test/scala/unit/kafka/utils/TestUtils.scala 21d0ed2cb7c9459261d3cdc7c21dece5e2079698 Diff: https://reviews.apache.org/r/30763/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 30763: Second attempt at flush()

omatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30763/#review73505 ------- On Feb. 23, 2015, 12:26 a.m., Jay Kreps wrote: > > --- > This is an automatically gener

Re: Review Request 30763: Second attempt at flush()

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30763/#review73665 --- On Feb. 23, 2015, 12:26 a.m., Jay Kreps

Re: Review Request 30763: Second attempt at flush()

-- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30763/#review73475 --- On Feb. 23, 2015, 12:26 a.m., Jay Kreps wrote: > > ---

[jira] [Updated] (KAFKA-1981) Make log compaction point configurable

[ https://issues.apache.org/jira/browse/KAFKA-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1981: - Labels: newbie++ (was: ) > Make log compaction point configura

[jira] [Created] (KAFKA-1981) Make log compaction point configurable

Jay Kreps created KAFKA-1981: Summary: Make log compaction point configurable Key: KAFKA-1981 URL: https://issues.apache.org/jira/browse/KAFKA-1981 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-1979) LogCleaner delete file asynchronously and can cause inconsistency between log cleaner checkpoint and first dirty log segment file.

[ https://issues.apache.org/jira/browse/KAFKA-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14333703#comment-14333703 ] Jay Kreps commented on KAFKA-1979: -- Can you expand on this a bit. The delete protoco

Re: How to get a JIRA assigned

Anyone know if there a way to turn this off? Is it possible to configure JIRA to let anyone assign them? Unlike the other lockdown stuff which prevents spam this doesn't seem like it could be a spam vector and it would be awesome to make it easier for people. -Jay On Sun, Feb 22, 2015 at 3:43 PM,

[jira] [Commented] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14332452#comment-14332452 ] Jay Kreps commented on KAFKA-1865: -- Uploaded a new patch that tracks all incomp

[jira] [Commented] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14332450#comment-14332450 ] Jay Kreps commented on KAFKA-1865: -- Updated reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1865) Add a flush() call to the new producer API

[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Attachment: KAFKA-1865_2015-02-22_16:26:46.patch > Add a flush() call to the new producer

Re: Review Request 30763: Second attempt at flush()

b15237b76def3b234924280fa3fdb25dbb0cc0dc core/src/test/scala/unit/kafka/utils/TestUtils.scala 21d0ed2cb7c9459261d3cdc7c21dece5e2079698 Diff: https://reviews.apache.org/r/30763/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 29794: Patch for KAFKA-1757

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29794/#review73439 --- Ship it! Ship It! - Jay Kreps On Jan. 10, 2015, 8:16 p.m

<    2   3   4   5   6   7   8   9   10   11   >