[jira] [Commented] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596298#comment-14596298 ] Aditya Auradkar commented on KAFKA-2293: [~junrao] Can you take a look at this

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/#review88796 --- Ship it! I did not review it thoroughly but the design looks clean

[jira] [Updated] (KAFKA-2288) Follow-up to KAFKA-2249 - reduce logging and testing

2015-06-22 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2288: Attachment: KAFKA-2288_2015-06-22_10:02:27.patch Follow-up to KAFKA-2249 - reduce logging and

Re: Review Request 35677: Patch for KAFKA-2288

2015-06-22 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35677/ --- (Updated June 22, 2015, 5:02 p.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596296#comment-14596296 ] Aditya A Auradkar commented on KAFKA-2293: -- Created reviewboard

[jira] [Resolved] (KAFKA-2235) LogCleaner offset map overflow

2015-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2235. Resolution: Fixed Fix Version/s: 0.8.3 Assignee: Ivan Simoneko (was: Jay Kreps) Thanks for

[jira] [Updated] (KAFKA-2293) IllegalFormatConversionException in Partition.scala

2015-06-22 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2293: - Attachment: KAFKA-2293.patch IllegalFormatConversionException in Partition.scala

[jira] [Resolved] (KAFKA-2290) OffsetIndex should open RandomAccessFile consistently

2015-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2290. Resolution: Fixed Fix Version/s: 0.8.3 Thanks for the patch. +1 and committed to trunk.

[jira] [Commented] (KAFKA-2276) Initial patch for KIP-25

2015-06-22 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596624#comment-14596624 ] Geoffrey Anderson commented on KAFKA-2276: -- Pull request is here:

Re: [GitHub] kafka pull request: Kafka 2276

2015-06-22 Thread Geoffrey Anderson
Hi, I'm pinging the dev list regarding KAFKA-2276 (KIP-25 initial patch) again since it sounds like at least one person I spoke with did not see the initial pull request. Pull request: https://github.com/apache/kafka/pull/70/ JIRA: https://issues.apache.org/jira/browse/KAFKA-2276 Thanks! Geoff

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35655/#review88877 --- Ship it! LGTM. There were 9 question marks when 10 characters were

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Ewen Cheslack-Postava
I'll respond to specific comments, but at the bottom of this email I've included some comparisons with other connector frameworks and Kafka import/export tools. This definitely isn't an exhaustive list, but hopefully will clarify how I'm thinking about Copycat should live wrt these other systems.

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Jay Kreps
Hey Gwen, That makes a lot of sense. Here was the thinking on our side. I guess there are two questions, where does Copycat go and where do the connectors go? I'm in favor of Copycat being in Kafka and the connectors being federated. Arguments for federating connectors: - There will be like

[jira] [Commented] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

2015-06-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596965#comment-14596965 ] Aditya Auradkar commented on KAFKA-2205: [~junrao] - Can you review this patch?

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Jay Kreps
Hey Roshan, That is definitely the key question in this space--what can we do that other systems don't? It's true that there are a number of systems that copy data between things. At a high enough level of abstraction I suppose they are somewhat the same. But I think this area is the source of

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 22, 2015, 11:35 p.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-06-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596850#comment-14596850 ] Jason Gustafson commented on KAFKA-2168: Updated reviewboard

[jira] [Updated] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

2015-06-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2168: --- Attachment: KAFKA-2168_2015-06-22_16:34:37.patch New consumer poll() can block other calls

Re: Review Request 35734: Patch for KAFKA-2293

2015-06-22 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35734/#review88866 --- Ship it! Minor comment: would use `%s` with `TopicAndPartition`

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Jason Gustafson
On June 21, 2015, 5:49 a.m., Guozhang Wang wrote: I think the main reason is that util.Random.nextString() may include other non-ascii chars and hence its toString may not be well-defined: http://alvinalexander.com/scala/creating-random-strings-in-scala So I think bottom-line is

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Jason Gustafson
On June 22, 2015, 11:43 p.m., Ewen Cheslack-Postava wrote: LGTM. There were 9 question marks when 10 characters were requested, so the problem was probably just that a whitespace character at the start or end would get trimmed during AbstractConfig's parsing. That's what I thought as

[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

2015-06-22 Thread Honghai Chen (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595450#comment-14595450 ] Honghai Chen commented on KAFKA-1646: - See the commit

[jira] [Updated] (KAFKA-2235) LogCleaner offset map overflow

2015-06-22 Thread Ivan Simoneko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Simoneko updated KAFKA-2235: - Attachment: KAFKA-2235_v2.patch LogCleaner offset map overflow --

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Jiangjie Qin
Very useful KIP. I have no clear opinion over where to put the framework will be better yet. I agree with Gwen on the benefits we can get from have a separate project for Copycat. But still have a few questions: 1. As far as code is concerned, Copycat would be some datasource adapters + Kafka

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

2015-06-22 Thread Ivan Simoneko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595798#comment-14595798 ] Ivan Simoneko commented on KAFKA-2235: -- [~junrao] thank you for review. Please check

[jira] [Assigned] (KAFKA-2290) OffsetIndex should open RandomAccessFile consistently

2015-06-22 Thread Chris Black (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Black reassigned KAFKA-2290: -- Assignee: Chris Black OffsetIndex should open RandomAccessFile consistently

[jira] [Commented] (KAFKA-2092) New partitioning for better load balancing

2015-06-22 Thread Gianmarco De Francisci Morales (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595510#comment-14595510 ] Gianmarco De Francisci Morales commented on KAFKA-2092: --- Any more

[jira] [Comment Edited] (KAFKA-2092) New partitioning for better load balancing

2015-06-22 Thread Gianmarco De Francisci Morales (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589634#comment-14589634 ] Gianmarco De Francisci Morales edited comment on KAFKA-2092 at 6/22/15 8:42 AM:

[jira] [Updated] (KAFKA-2290) OffsetIndex should open RandomAccessFile consistently

2015-06-22 Thread Chris Black (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Black updated KAFKA-2290: --- Attachment: KAFKA-2290.patch OffsetIndex should open RandomAccessFile consistently

[jira] [Comment Edited] (KAFKA-2276) Initial patch for KIP-25

2015-06-22 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596624#comment-14596624 ] Geoffrey Anderson edited comment on KAFKA-2276 at 6/22/15 9:00 PM:

[jira] [Created] (KAFKA-2294) javadoc compile error due to illegal p/ , build failing (jdk 8)

2015-06-22 Thread Jeremy Fields (JIRA)
Jeremy Fields created KAFKA-2294: Summary: javadoc compile error due to illegal p/ , build failing (jdk 8) Key: KAFKA-2294 URL: https://issues.apache.org/jira/browse/KAFKA-2294 Project: Kafka

Re: [DISCUSS] KIP-26 - Add Copycat, a connector framework for data import/export

2015-06-22 Thread Roshan Naik
Thanks Jay and Ewen for the response. @Jay 3. This has a built in notion of parallelism throughout. It was not obvious how it will look like or differ from existing systemsÅ  since all of existing ones do parallelize data movement. @Ewen, Import: Flume is just one of many similar systems

Re: Review Request 35655: Patch for KAFKA-2271

2015-06-22 Thread Ewen Cheslack-Postava
On June 22, 2015, 11:43 p.m., Ewen Cheslack-Postava wrote: LGTM. There were 9 question marks when 10 characters were requested, so the problem was probably just that a whitespace character at the start or end would get trimmed during AbstractConfig's parsing. Jason Gustafson wrote:

Re: Review Request 34789: Patch for KAFKA-2168

2015-06-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/#review88914 --- Ship it! Thanks for the latest patch. Looks good overall. To avoid

Re: [GitHub] kafka pull request: Kafka 2276

2015-06-22 Thread Gwen Shapira
Thanks, I indeed missed the original :) Is the plan to squash the commits and merge a pull request with single commit that matches the JIRA #? This will be more in line with how commits were organized until now and will make life much easier when cherry-picking. Gwen On Mon, Jun 22, 2015 at