Review Request 35832: Patch for KAFKA-2199

2015-06-24 Thread Ewen Cheslack-Postava
://reviews.apache.org/r/35832/diff/ Testing --- Thanks, Ewen Cheslack-Postava

[jira] [Commented] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-06-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599743#comment-14599743 ] Ewen Cheslack-Postava commented on KAFKA-2199: -- [~omkreddy] Sorry about

Re: [GitHub] kafka pull request: Kafka 2276

2015-06-24 Thread Ewen Cheslack-Postava
Github either lets you do inline comments on the diff (Files Changed tab) or on the entire commit (Conversation). One of the things review board does right that's a bit frustrating in Github his that you can't group a bunch of comments into a single review and post it all at once (and get only one

[jira] [Commented] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-06-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599738#comment-14599738 ] Ewen Cheslack-Postava commented on KAFKA-2199: -- Created reviewboard https

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-06-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2199: - Attachment: KAFKA-2199.patch Make signing artifacts optional, setting maven

[jira] [Commented] (KAFKA-2203) Get gradle build to work with Java 8

2015-06-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599771#comment-14599771 ] Ewen Cheslack-Postava commented on KAFKA-2203: -- [~gwenshap] Agreed

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-23 Thread Ewen Cheslack-Postava
On Mon, Jun 22, 2015 at 8:32 PM, Roshan Naik ros...@hortonworks.com wrote: Thanks Jay and Ewen for the response. @Jay 3. This has a built in notion of parallelism throughout. It was not obvious how it will look like or differ from existing systemsÅ  since all of existing ones do

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-23 Thread Ewen Cheslack-Postava
can find a draft of these docs here: https://s3-us-west-2.amazonaws.com/confluent-files/copycat-docs-wip/intro.html -Ewen On Tue, Jun 23, 2015 at 6:11 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Mon, Jun 22, 2015 at 8:32 PM, Roshan Naik ros...@hortonworks.com wrote: Thanks Jay

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-16 Thread Ewen Cheslack-Postava
Oops, linked the wrong thing. Here's the correct one: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851767 -Ewen On Tue, Jun 16, 2015 at 4:32 PM, Ewen Cheslack-Postava e...@confluent.io wrote: Hi all, I just posted KIP-26 - Add Copycat, a connector framework for data

[DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-16 Thread Ewen Cheslack-Postava
Hi all, I just posted KIP-26 - Add Copycat, a connector framework for data import/export here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals This is a large KIP compared to what we've had so far, and is a bit different from most. We're proposing the addition of a

Re: Review Request 35421: Patch for KAFKA-2026

2015-06-13 Thread Ewen Cheslack-Postava
initially filed this) will be a simpler solution... - Ewen Cheslack-Postava On June 13, 2015, 10:07 a.m., Manikumar Reddy O wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35421

Re: Review Request 35076: Patch for KAFKA-2248

2015-06-13 Thread Ewen Cheslack-Postava
/r/35076/diff/ Testing --- Thanks, Ewen Cheslack-Postava

[jira] [Updated] (KAFKA-2248) Use Apache Rat to enforce copyright headers

2015-06-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2248: - Attachment: KAFKA-2248_2015-06-13_14:10:23.patch Use Apache Rat to enforce

[jira] [Commented] (KAFKA-2248) Use Apache Rat to enforce copyright headers

2015-06-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584817#comment-14584817 ] Ewen Cheslack-Postava commented on KAFKA-2248: -- Updated reviewboard https

[jira] [Commented] (KAFKA-2248) Use Apache Rat to enforce copyright headers

2015-06-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584818#comment-14584818 ] Ewen Cheslack-Postava commented on KAFKA-2248: -- [~junrao] Try this new

[jira] [Resolved] (KAFKA-2242) Fix license headers (remove copyright notices) in a few test sources

2015-06-02 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2242. -- Resolution: Duplicate [~jjkoshy] Yup, already caught this and it's filed

Re: Review Request 34789: Patch for KAFKA-2168

2015-05-28 Thread Ewen Cheslack-Postava
/consumer/internals/Coordinator.java https://reviews.apache.org/r/34789/#comment137314 Why do these all have ensureNotClosed(), but the KafkaConsumer methods don't all have it? - Ewen Cheslack-Postava On May 28, 2015, 10:58 p.m., Jason Gustafson wrote

Re: Review Request 33196: Patch for KAFKA-2123

2015-05-29 Thread Ewen Cheslack-Postava
-CREATION core/src/test/scala/integration/kafka/api/ConsumerTest.scala a1eed965a148eb19d9a6cefbfce131f58aaffc24 Diff: https://reviews.apache.org/r/33196/diff/ Testing --- Thanks, Ewen Cheslack-Postava

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-05-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2199: - Attachment: KAFKA-2199_2015-05-29_11:00:44.patch Make signing artifacts optional

[jira] [Commented] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-05-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565155#comment-14565155 ] Ewen Cheslack-Postava commented on KAFKA-2199: -- Updated reviewboard https

Re: Review Request 34369: Patch for KAFKA-2199

2015-05-29 Thread Ewen Cheslack-Postava
946ec62cc71df93c905c5f35caf5cdb9c78e5c10 build.gradle 3dca28eee55e04d4349fbada2079c64b0f1ef6a2 gradle.properties 90b1945372e767b9c2d0a50df9eb7063e0629952 Diff: https://reviews.apache.org/r/34369/diff/ Testing --- Thanks, Ewen Cheslack-Postava

[jira] [Commented] (KAFKA-2123) Make new consumer offset commit API use callback + future

2015-05-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565164#comment-14565164 ] Ewen Cheslack-Postava commented on KAFKA-2123: -- Updated reviewboard https

[jira] [Updated] (KAFKA-2123) Make new consumer offset commit API use callback + future

2015-05-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2123: - Attachment: KAFKA-2123_2015-05-29_11:11:05.patch Make new consumer offset commit

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-06-02 Thread Ewen Cheslack-Postava
sense to have it maintain the old buggy behavior. So we can set it to a reasonable value instead of infinite. Jiangjie (Becket) Qin On 5/12/15, 12:03 PM, Ewen Cheslack-Postava e...@confluent.io wrote: I think my

Re: Review Request 35850: Patch for KAFKA-2301

2015-07-02 Thread Ewen Cheslack-Postava
110) https://reviews.apache.org/r/35850/#comment143205 Since this is a warning, can we just use warn() instead of info()? Otherwise, this trivial patch LGTM! - Ewen Cheslack-Postava On July 2, 2015, 12:46 a.m., Ashish Singh wrote

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-29 Thread Ewen Cheslack-Postava
Flume and NiFi, Sentry and Argus, Flink and Storm, Parquet and ORC, I'm sure we can also have CopyCat in the zoo :) Gwen On Tue, Jun 23, 2015 at 6:11 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Mon, Jun 22, 2015 at 8:32 PM, Roshan Naik ros...@hortonworks.com wrote

[jira] [Reopened] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-07-06 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava reopened KAFKA-2199: -- Needs followup patch applied. Make signing artifacts optional, setting maven

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-07-06 Thread Ewen Cheslack-Postava
these processes (and help maintain the cluster by restarting them if they crash), but there's no other deep integration or custom code required given the current description in the KIP. -Ewen Guozhang On Mon, Jun 29, 2015 at 9:50 AM, Ewen Cheslack-Postava e...@confluent.io wrote: Seems

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-07-06 Thread Ewen Cheslack-Postava
On Mon, Jul 6, 2015 at 6:24 PM, Guozhang Wang wangg...@gmail.com wrote: On Mon, Jul 6, 2015 at 4:33 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Mon, Jul 6, 2015 at 11:40 AM, Guozhang Wang wangg...@gmail.com wrote: Hi Ewen, I read through the KIP page and here are some

Re: Review Request 36242: Patch for KAFKA-2311

2015-07-06 Thread Ewen Cheslack-Postava
) https://reviews.apache.org/r/36242/#comment143767 Only related to this patch because I'm looking at ensureNotClosed -- it's only used in one place and is a trivial check. We could just inline it. - Ewen Cheslack-Postava On July 7, 2015, 4:29 a.m., Tim Brooks wrote

Re: Review Request 36244: Patch for KAFKA-2312

2015-07-06 Thread Ewen Cheslack-Postava
doesn't have any real impact in practical code, but may be a bit more intuitive. - Ewen Cheslack-Postava On July 7, 2015, 5 a.m., Tim Brooks wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-24 Thread Ewen Cheslack-Postava
. clients/src/main/java/org/apache/kafka/common/network/Selector.java https://reviews.apache.org/r/34608/#comment136579 checkstyle is complaining because this isn't indented correctly. - Ewen Cheslack-Postava On May 22, 2015, 7 p.m., Jason Gustafson wrote

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-25 Thread Ewen Cheslack-Postava
On May 24, 2015, 7:16 p.m., Ewen Cheslack-Postava wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, line 226 https://reviews.apache.org/r/34608/diff/1/?file=969830#file969830line226 One drawback to this is that we're now constantly reallocating

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-05-21 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554799#comment-14554799 ] Ewen Cheslack-Postava commented on KAFKA-2168: -- But isn't that going to make

Re: [DISCUSS] KIP-25 System test improvements

2015-05-21 Thread Ewen Cheslack-Postava
Ashish, 1. That was the plan. We put some effort into cleanly separating the framework so it would be reusable across many projects. 2. I think you're seeing a test in progress where the final report hasn't been created yet. If you visit one of the older ones you'll see it has a landing page with

Re: Review Request 34608: Patch for KAFKA-2217

2015-05-26 Thread Ewen Cheslack-Postava
as long as it doesn't seem to affect performance. clients/src/main/java/org/apache/kafka/common/network/Selector.java https://reviews.apache.org/r/34608/#comment136767 This shortcut seems like a good idea, but skips the call to this.sensors.ioTime.record(). - Ewen Cheslack-Postava

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-05-26 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14559650#comment-14559650 ] Ewen Cheslack-Postava commented on KAFKA-2168: -- Some reasons you might want

[jira] [Commented] (KAFKA-2304) Support enabling JMX in Kafka Vagrantfile

2015-07-07 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616955#comment-14616955 ] Ewen Cheslack-Postava commented on KAFKA-2304: -- [~joestein] assigned to you

[jira] [Updated] (KAFKA-2304) Support enabling JMX in Kafka Vagrantfile

2015-07-07 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2304: - Assignee: Joe Stein Support enabling JMX in Kafka Vagrantfile

Re: Build failed in Jenkins: KafkaPreCommit #138

2015-07-07 Thread Ewen Cheslack-Postava
; Broker topic metadata not kept in sync with ZooKeeper; patched by Ashish Singh; reviewed by Jun Rao [joe.stein] KAFKA-2304 Supported enabling JMX in Kafka Vagrantfile patch by Stevo Slavic reviewed by Ewen Cheslack-Postava -- Started by an SCM change

Copycat data API serializers

2015-07-31 Thread Ewen Cheslack-Postava
Hi all, With the initial patch of of the Copycat APIs under review, I want to get started on the data API. I wrote up some notes with a rough proposal for what it should cover and a few requirements here: https://cwiki.apache.org/confluence/display/KAFKA/Copycat+Data+API Since they are very

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

2015-08-11 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692853#comment-14692853 ] Ewen Cheslack-Postava commented on KAFKA-2367: -- The runtime API should

[jira] [Work started] (KAFKA-2429) Add annotations to mark classes as stable/unstable

2015-08-12 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2429 started by Ewen Cheslack-Postava. Add annotations to mark classes as stable/unstable

[jira] [Updated] (KAFKA-2429) Add annotations to mark classes as stable/unstable

2015-08-12 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2429: - Status: Patch Available (was: In Progress) Add annotations to mark classes

[jira] [Created] (KAFKA-2429) Add annotations to mark classes as stable/unstable

2015-08-12 Thread Ewen Cheslack-Postava (JIRA)
Ewen Cheslack-Postava created KAFKA-2429: Summary: Add annotations to mark classes as stable/unstable Key: KAFKA-2429 URL: https://issues.apache.org/jira/browse/KAFKA-2429 Project: Kafka

Re: NEW: API Stability annotations!

2015-08-13 Thread Ewen Cheslack-Postava
We could potentially add more annotations to cover things like public/private APIs, deprecation, etc. I lifted this initial implementation from Hadoop (thanks for the pointer Gwen!). Hadoop also has a public/private annotation that we could lift, and they use this to filter javadocs. These are

Re: [Copycat] How will copycat serialize its metadata

2015-08-17 Thread Ewen Cheslack-Postava
REST APIs... most of them are :) On Fri, Aug 14, 2015 at 8:14 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Fri, Aug 14, 2015 at 6:35 PM, Gwen Shapira g...@confluent.io wrote: Yeah, I missed the option to match serialization of offsets to data, which solves the configuration

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-17 Thread Ewen Cheslack-Postava
metadata change, which is infrequent. Any thoughts? Thanks, Jiangjie (Becket) Qin On Fri, Aug 14, 2015 at 12:57 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Fri, Aug 14, 2015 at 10:59 AM

[jira] [Updated] (KAFKA-2330) Vagrantfile sets global configs instead of per-provider override configs

2015-08-18 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2330: - Reviewer: Gwen Shapira (was: Joe Stein) Vagrantfile sets global configs instead

[jira] [Commented] (KAFKA-2330) Vagrantfile sets global configs instead of per-provider override configs

2015-08-18 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702443#comment-14702443 ] Ewen Cheslack-Postava commented on KAFKA-2330: -- [~gwenshap] Reassigned to you

[jira] [Resolved] (KAFKA-2444) Fail test: kafka.api.QuotasTest testThrottledProducerConsumer FAILED

2015-08-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2444. -- Resolution: Duplicate Other one is already assigned, not sure if anyone

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

2015-08-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703901#comment-14703901 ] Ewen Cheslack-Postava commented on KAFKA-2367: -- Yes. Add Copycat runtime

Re: Review Request 36427: Patch for KAFKA-2330

2015-08-19 Thread Ewen Cheslack-Postava
73cf0390bc4c76be310c09ddedf91de5ddf1b473 Diff: https://reviews.apache.org/r/36427/diff/ Testing --- Thanks, Ewen Cheslack-Postava

[jira] [Commented] (KAFKA-2330) Vagrantfile sets global configs instead of per-provider override configs

2015-08-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704066#comment-14704066 ] Ewen Cheslack-Postava commented on KAFKA-2330: -- Updated reviewboard https

[jira] [Updated] (KAFKA-2330) Vagrantfile sets global configs instead of per-provider override configs

2015-08-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2330: - Attachment: KAFKA-2330_2015-08-19_17:50:17.patch Vagrantfile sets global configs

[jira] [Created] (KAFKA-2459) Connection backoff/blackout period should start when a connection is disconnected, not when the connection attempt was initiated

2015-08-22 Thread Ewen Cheslack-Postava (JIRA)
Ewen Cheslack-Postava created KAFKA-2459: Summary: Connection backoff/blackout period should start when a connection is disconnected, not when the connection attempt was initiated Key: KAFKA-2459 URL

[jira] [Commented] (KAFKA-1566) Kafka environment configuration (kafka-env.sh)

2015-08-23 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708306#comment-14708306 ] Ewen Cheslack-Postava commented on KAFKA-1566: -- Basic idea seems fine, but I

[jira] [Updated] (KAFKA-2377) Add copycat system tests

2015-08-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2377: - Reviewer: Gwen Shapira Status: Patch Available (was: Open) [~gwenshap

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

2015-08-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708866#comment-14708866 ] Ewen Cheslack-Postava commented on KAFKA-2389: -- If I'm understanding people's

[jira] [Commented] (KAFKA-2304) Support enabling JMX in Kafka Vagrantfile

2015-06-30 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609046#comment-14609046 ] Ewen Cheslack-Postava commented on KAFKA-2304: -- Patch works and looks fine

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-06-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14607548#comment-14607548 ] Ewen Cheslack-Postava commented on KAFKA-2168: -- This version was committed

[jira] [Resolved] (KAFKA-2382) KafkaConsumer seek methods should throw an exception when called for partitions not assigned to this consumer instance

2015-07-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2382. -- Resolution: Invalid Ah, good call. Somehow I missed that, guess I didn't dig

[jira] [Commented] (KAFKA-1913) App hungs when calls producer.send to wrong IP of Kafka broker

2015-07-29 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646373#comment-14646373 ] Ewen Cheslack-Postava commented on KAFKA-1913: -- [~igor.quickblox] I think

Re: New (0.8.2.1) Sync Producer with Batch ?

2015-08-01 Thread Ewen Cheslack-Postava
Sunil, Batching is performed for you automatically. There are a few settings you can use to tweak it's behavior (e.g. linger.ms and batch.size), but the defaults are good for common workloads and can achieve very high throughput. If you want to send some messages and then wait for them to be

Re: Review Request 35421: Patch for KAFKA-2026

2015-07-30 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35421/#review93636 --- Ship it! Ship It! - Ewen Cheslack-Postava On June 13, 2015, 10

[jira] [Commented] (KAFKA-2026) Logging of unused options always shows null for the value and is misleading if the option is used by serializers

2015-07-30 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648279#comment-14648279 ] Ewen Cheslack-Postava commented on KAFKA-2026: -- [~guozhang] This looks fine

[jira] [Commented] (KAFKA-2385) zookeeper-shell does not work

2015-07-31 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648890#comment-14648890 ] Ewen Cheslack-Postava commented on KAFKA-2385: -- What exactly doesn't work

[jira] [Updated] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-28 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2350: - Component/s: consumer Add KafkaConsumer pause capability

[jira] [Updated] (KAFKA-2343) Clarify KafkaConsumer.poll rebalance behavior

2015-07-28 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2343: - Component/s: consumer Clarify KafkaConsumer.poll rebalance behavior

Re: Kafka Consumer thoughts

2015-07-28 Thread Ewen Cheslack-Postava
On Tue, Jul 28, 2015 at 5:18 PM, Guozhang Wang wangg...@gmail.com wrote: I think Ewen has proposed these APIs for using callbacks along with returning future in the commit calls, i.e. something similar to: public Futurevoid commit(ConsumerCommitCallback callback); public Futurevoid

[jira] [Created] (KAFKA-2382) KafkaConsumer seek methods should throw an exception when called for partitions not assigned to this consumer instance

2015-07-28 Thread Ewen Cheslack-Postava (JIRA)
Ewen Cheslack-Postava created KAFKA-2382: Summary: KafkaConsumer seek methods should throw an exception when called for partitions not assigned to this consumer instance Key: KAFKA-2382 URL: https

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

2015-08-06 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660629#comment-14660629 ] Ewen Cheslack-Postava commented on KAFKA-2143: -- [~wushujames] I think one

[jira] [Commented] (KAFKA-2398) Transient test failure for SocketServerTest - Socket closed.

2015-08-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695775#comment-14695775 ] Ewen Cheslack-Postava commented on KAFKA-2398: -- Sadly this does not seem

[jira] [Reopened] (KAFKA-2398) Transient test failure for SocketServerTest - Socket closed.

2015-08-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava reopened KAFKA-2398: -- Transient test failure for SocketServerTest - Socket closed

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-13 Thread Ewen Cheslack-Postava
On Tue, Aug 11, 2015 at 11:43 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Tue, Aug 11, 2015 at 11:29 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Ewen, Thanks for the explanation. For (1), I am more concerned about the failure case

[jira] [Commented] (KAFKA-2397) leave group request

2015-08-13 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696327#comment-14696327 ] Ewen Cheslack-Postava commented on KAFKA-2397: -- Can we be more concrete about

[jira] [Commented] (KAFKA-2397) leave group request

2015-08-14 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696602#comment-14696602 ] Ewen Cheslack-Postava commented on KAFKA-2397: -- [~becket_qin] I was more

Re: [copycat] How to upgrade connectors

2015-08-14 Thread Ewen Cheslack-Postava
Yes, I think that makes sense. As I see it, the tradeoffs are: 1. Complexity - adding these APIs increases the number of things connector developers need to implement just to get started. In a lot of cases, the first version of a connector might literally only have some connection string as a

Re: Copycat (KIP-26. PR-99) - plan on moving forward

2015-08-14 Thread Ewen Cheslack-Postava
For obvious reasons, I definitely won't object, but I will probably quickly have a follow-up patch to add a very basic system test for Copycat. I was trying to get it in with this patch since I think it is a good idea to encourage people to write system tests with major additions like this, but I

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-14 Thread Ewen Cheslack-Postava
to fix it. Thanks, Jiangjie (Becket) Qin On Tue, Aug 11, 2015 at 11:43 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Tue, Aug 11, 2015 at 11:29 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Ewen

Re: [copycat] support for batch jobs

2015-08-14 Thread Ewen Cheslack-Postava
The JDBC connector I started implementing just handles this manually, and isn't much code (and could be made into a simple utility): https://github.com/confluentinc/copycat-jdbc/blob/master/src/main/java/io/confluent/copycat/jdbc/JdbcSourceTask.java#L152 Given the current APIs, sources can just

Re: [Copycat] How will copycat serialize its metadata

2015-08-14 Thread Ewen Cheslack-Postava
I'm not sure the existing discussion is clear about how the format of offset data is decided. One possibility is that we choose one fixed format and that is what we use internally to store offsets no matter what serializer you choose. This would be similar to how the __offsets topic is currently

[jira] [Updated] (KAFKA-2281) org.apache.kafka.clients.producer.internals.ErrorLoggingCallback holds unnecessary byte[] value

2015-08-10 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2281: - Fix Version/s: 0.8.3

Re: Review Request 35437: Patch for KAFKA-2202

2015-08-10 Thread Ewen Cheslack-Postava
Cheslack-Postava On June 14, 2015, 11:27 a.m., Manikumar Reddy O wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35437

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-14 Thread Ewen Cheslack-Postava
, this is probably orthogonal to this thread given the current new consumer also has this problem and I believe we need to fix it. Thanks, Jiangjie (Becket) Qin On Tue, Aug 11, 2015 at 11:43 PM, Ewen Cheslack-Postava e

Re: [copycat] support for batch jobs

2015-08-14 Thread Ewen Cheslack-Postava
On Fri, Aug 14, 2015 at 10:57 AM, Jay Kreps j...@confluent.io wrote: I thought batch was dead? :-) Yeah I think this would be really useful. Kafka kind of allows you to unify batch and streams since you produce or consume your stream on your own schedule so you would want the ingress/egress

[jira] [Updated] (KAFKA-1782) Junit3 Misusage

2015-08-14 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-1782: - Status: Patch Available (was: Reopened) Junit3 Misusage

Re: [Copycat] How will copycat serialize its metadata

2015-08-14 Thread Ewen Cheslack-Postava
and parse using their favorite JSON library) vs not being accessible (they won't pull in whatever serializer we use internally, or don't want to write a custom parser for our custom serialization format). -Ewen Gwen On Fri, Aug 14, 2015 at 1:29 AM, Ewen Cheslack-Postava e...@confluent.io wrote

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-15 Thread Ewen Cheslack-Postava
) Qin On Fri, Aug 14, 2015 at 12:57 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Fri, Aug 14, 2015 at 10:59 AM, Jiangjie Qin j...@linkedin.com.invalid wrote: Neha and Ewen, About the metadata change frequency. I guess it really depends on how frequent

Re: Copycat data API serializers

2015-08-11 Thread Ewen Cheslack-Postava
, 2015 at 6:21 PM, Ewen Cheslack-Postava e...@confluent.io wrote: Hi all, With the initial patch of of the Copycat APIs under review, I want to get started on the data API. I wrote up some notes with a rough proposal for what it should cover and a few requirements here: https://cwiki.apache.org

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-12 Thread Ewen Cheslack-Postava
not address. -Ewen Thanks, Jiangjie (Becket) Qin On Tue, Aug 11, 2015 at 11:06 PM, Ewen Cheslack-Postava e...@confluent.io wrote: On Tue, Aug 11, 2015 at 10:15 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi Jason, Thanks for writing this up. It would be useful

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-11 Thread Ewen Cheslack-Postava
On Tue, Aug 11, 2015 at 10:03 PM, Onur Karaman onurkaraman.apa...@gmail.com wrote: Just to make the conversation a bit easier (I don't think we have really established names for these modes yet), basically with the new KafkaConsumer today there's: - external management, where the application

Re: [DISCUSS] Client-side Assignment for New Consumer

2015-08-12 Thread Ewen Cheslack-Postava
On Tue, Aug 11, 2015 at 10:15 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi Jason, Thanks for writing this up. It would be useful to generalize the group concept. I have a few questions below. 1. In old consumer actually the partition assignment are done by consumers themselves. We

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

2015-08-11 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692893#comment-14692893 ] Ewen Cheslack-Postava commented on KAFKA-2367: -- [~wushujames] see the Schema

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

2015-08-11 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692903#comment-14692903 ] Ewen Cheslack-Postava commented on KAFKA-2367: -- I don't think it's useful

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

2015-08-12 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694057#comment-14694057 ] Ewen Cheslack-Postava commented on KAFKA-2367: -- [~ijuma] No huge benefits

[jira] [Updated] (KAFKA-2367) Add Copycat runtime data API

2015-07-27 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2367: - Summary: Add Copycat runtime data API (was: Add Copycat runtime API) Add

[jira] [Updated] (KAFKA-2368) Add Copycat standalone CLI

2015-07-27 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2368: - Component/s: copycat Add Copycat standalone CLI

[jira] [Updated] (KAFKA-2366) Initial patch for Copycat

2015-07-27 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2366: - Component/s: copycat Initial patch for Copycat

[jira] [Updated] (KAFKA-2376) Add Copycat metrics

2015-07-27 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2376: - Component/s: copycat Add Copycat metrics

<    1   2   3   4   5   6   7   8   9   10   >