[jira] [Comment Edited] (KAFKA-5040) Increase number of Streams producer retries from the default of 0

2017-05-05 Thread Peter Davis (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999309#comment-15999309 ] Peter Davis edited comment on KAFKA-5040 at 5/6/17 5:58 AM: Just confirming

[jira] [Commented] (KAFKA-5040) Increase number of Streams producer retries from the default of 0

2017-05-05 Thread Peter Davis (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999309#comment-15999309 ] Peter Davis commented on KAFKA-5040: Just confirming that per [this comment on

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

2017-05-05 Thread James Cheng
Hi Ismael, Thanks for the KIP. I see that in the ListConfigs Response protocol, that configs have an is_default field. Does that mean that it will include *all* config values, instead of just overridden ones? As an example, kafka-config.sh --describe on a topic will, right now, only show

[jira] [Created] (KAFKA-5184) Transient failure: MultipleListenersWithAdditionalJaasContextTest.testProduceConsume

2017-05-05 Thread JIRA
Xavier Léauté created KAFKA-5184: Summary: Transient failure: MultipleListenersWithAdditionalJaasContextTest.testProduceConsume Key: KAFKA-5184 URL: https://issues.apache.org/jira/browse/KAFKA-5184

[jira] [Reopened] (KAFKA-2910) Failure in kafka.api.SslEndToEndAuthorizationTest.testNoGroupAcl

2017-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté reopened KAFKA-2910: -- > Failure in kafka.api.SslEndToEndAuthorizationTest.testNoGroupAcl >

[GitHub] kafka pull request #2986: KAFKA-5099: Replica Deletion Regression from KIP-1...

2017-05-05 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2986 KAFKA-5099: Replica Deletion Regression from KIP-101 Replica deletion regressed from KIP-101. Replica deletion happens when a broker receives a StopReplicaRequest with delete=true. Ever since

Re: [VOTE] KIP-144: Exponential backoff for broker reconnect attempts

2017-05-05 Thread Jay Kreps
+1 On Fri, May 5, 2017 at 7:29 PM Sriram Subramanian wrote: > +1 > > On Fri, May 5, 2017 at 6:04 PM, Gwen Shapira wrote: > > > +1 > > > > On Fri, May 5, 2017 at 3:32 PM, Ismael Juma wrote: > > > > > Hi all, > > > > > > Given the simple

Re: [VOTE] KIP-144: Exponential backoff for broker reconnect attempts

2017-05-05 Thread Sriram Subramanian
+1 On Fri, May 5, 2017 at 6:04 PM, Gwen Shapira wrote: > +1 > > On Fri, May 5, 2017 at 3:32 PM, Ismael Juma wrote: > > > Hi all, > > > > Given the simple and non controversial nature of the KIP, I would like to > > start the voting process for KIP-144:

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

2017-05-05 Thread Ismael Juma
Hi Colin, Thanks for the review. Comments inline. On Fri, May 5, 2017 at 9:45 PM, Colin McCabe wrote: > As Jun commented, it's probably more consistent to use > Collection, unless this really needs to be an ordered > list. That way people can use any container they want. >

Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-05 Thread Randall Hauch
Thanks, Gwen. Switching to low-priority is a great idea. The default value for the replication factor configuration is 3, since that makes sense and is safe for production. Using the default values in the example would mean it could only be run against a Kafka cluster with a minimum of 3

Build failed in Jenkins: kafka-trunk-jdk8 #1488

2017-05-05 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-5143: add kafka-broker-api-version.bat for Windows platform -- [...truncated 1.65 MB...] kafka.utils.CommandLineUtilsTest > testParseArgs

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

2017-05-05 Thread Ismael Juma
Hi Jun, Thanks for the review. 1. Yes, a Set would work too. I updated to take Collection for consistency as Colin said. 2. That was a typo, it should have been a boolean indicating whether we should only perform validation. Updated the KIP. Thanks, Ismael On Fri, May 5, 2017 at 6:35 PM, Jun

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-05-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999218#comment-15999218 ] Jun Rao commented on KAFKA-5099: [~onurkaraman], that's probably expected. The issue is that java doesn't

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-05-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999211#comment-15999211 ] Jun Rao commented on KAFKA-5099: [~onurkaraman], that's probably expected. The issue is that java doesn't

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Xavier Léauté
Any reason we are not keeping the per-topic breakdown for inter-broker traffic? On Fri, May 5, 2017 at 4:52 PM Onur Karaman wrote: > Looks good. Thanks! > > On Fri, May 5, 2017 at 4:44 PM, Roger Hoover > wrote: > > > Very helpful. Thank

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Onur Karaman
I noticed that both the ControllerState metric and the *RateAndTimeMs metrics only cover a subset of the controller event types. Was this intentional? On Fri, May 5, 2017 at 6:03 PM, Gwen Shapira wrote: > +1 > > On Thu, May 4, 2017 at 7:34 PM, Ismael Juma

Re: [VOTE] KIP-136: Add Listener name and Security Protocol name to SelectorMetrics tags

2017-05-05 Thread Gwen Shapira
+1 On Fri, May 5, 2017 at 1:26 PM, Edoardo Comar wrote: > Hi, > I'd like to start the vote on this simple KIP : > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 136%3A+Add+Listener+name+and+Security+Protocol+name+to+ > SelectorMetrics+tags > > It improves the

Re: [VOTE] KIP-144: Exponential backoff for broker reconnect attempts

2017-05-05 Thread Gwen Shapira
+1 On Fri, May 5, 2017 at 3:32 PM, Ismael Juma wrote: > Hi all, > > Given the simple and non controversial nature of the KIP, I would like to > start the voting process for KIP-144: Exponential backoff for broker > reconnect attempts: > >

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Gwen Shapira
+1 On Thu, May 4, 2017 at 7:34 PM, Ismael Juma wrote: > Hi everyone, > > It seems like the discussion has converged, so I would like to initiate the > voting process for KIP-143: Controller Health Metrics: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-143 >

Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-05 Thread Gwen Shapira
Looks great to me :) Just one note - configurations have levels (which reflect in the docs) - I suggest putting the whole thing as LOW. Most users will never need to worry about these. For same reason I recommend leaving them out of the example config files - we already have issues with users

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-05-05 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999200#comment-15999200 ] Onur Karaman commented on KAFKA-5099: - Hi [~junrao]. Thanks for figuring it out. I gave your solution

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread James Cheng
+1 (non-binding) > On May 4, 2017, at 7:34 PM, Ismael Juma wrote: > > Hi everyone, > > It seems like the discussion has converged, so I would like to initiate the > voting process for KIP-143: Controller Health Metrics: > >

[jira] [Assigned] (KAFKA-5172) CachingSessionStore doesn't fetchPrevious correctly.

2017-05-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-5172: Assignee: Kyle Winkelman > CachingSessionStore doesn't fetchPrevious correctly. >

Re: [DISCUSS] KIP-123: Allow per stream/table timestamp extractor

2017-05-05 Thread Guozhang Wang
Hi Ewen, Personally I'm thinking to remove deprecated APIs after one minor release, so yes, 0.12.0.0 would be the preferred timeline. Guozhang On Thu, Apr 6, 2017 at 8:40 PM, Ewen Cheslack-Postava wrote: > I agree there's a potential issue that Eno raises, but probably

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

2017-05-05 Thread Guozhang Wang
THank you, Jeyhun! Will made a final pass on the PR soon. Guozhang On Tue, Apr 25, 2017 at 11:13 PM, Michael Noll wrote: > Thanks for your work and for driving this, Jeyhun! :-) > > -Michael > > > On Tue, Apr 25, 2017 at 11:11 PM, Jeyhun Karimov >

Re: Please give me permission to create/edit Kafka Improvement Proposals

2017-05-05 Thread Guozhang Wang
Hello Randall, I saw your name on the permissions list already. Cheers, Guozhang On Fri, May 5, 2017 at 1:18 PM, Randall Hauch wrote: > See subject. I'm a contributor to Kafka Connect. Thanks! > > Randall > -- -- Guozhang

[jira] [Commented] (KAFKA-5143) Windows platform does not offer kafka-broker-api-versions.bat

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999156#comment-15999156 ] ASF GitHub Bot commented on KAFKA-5143: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2943: KAFKA-5143: add kafka-broker-api-version.bat for W...

2017-05-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2943 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5143) Windows platform does not offer kafka-broker-api-versions.bat

2017-05-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-5143. -- Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2943

Re: [DISCUSS] KIP-148: Add a connect timeout for client

2017-05-05 Thread Guozhang Wang
Hello David, Thanks for the KIP. For the described issue, I'm wondering if it can be resolved by tuning the CONNECTIONS_MAX_IDLE_MS_CONFIG ( connections.max.idle.ms) on the client side? Default is 9 minutes. Guozhang On Tue, May 2, 2017 at 8:22 AM, 东方甲乙 <254479...@qq.com> wrote: > Hi all, > >

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Onur Karaman
Looks good. Thanks! On Fri, May 5, 2017 at 4:44 PM, Roger Hoover wrote: > Very helpful. Thank you, Jun. > > On Fri, May 5, 2017 at 4:42 PM, Guozhang Wang wrote: > > > Jun, > > > > Thanks for the KIP, LGTM. > > > > Guozhang > > > > On Fri, May 5,

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Roger Hoover
Very helpful. Thank you, Jun. On Fri, May 5, 2017 at 4:42 PM, Guozhang Wang wrote: > Jun, > > Thanks for the KIP, LGTM. > > Guozhang > > On Fri, May 5, 2017 at 3:38 PM, Ismael Juma wrote: > > > Thanks Jun, looks good to me. > > > > Ismael > > > > On Fri,

Re: [VOTE] KIP-144: Exponential backoff for broker reconnect attempts

2017-05-05 Thread Guozhang Wang
+1. On Fri, May 5, 2017 at 3:32 PM, Ismael Juma wrote: > Hi all, > > Given the simple and non controversial nature of the KIP, I would like to > start the voting process for KIP-144: Exponential backoff for broker > reconnect attempts: > >

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Guozhang Wang
Jun, Thanks for the KIP, LGTM. Guozhang On Fri, May 5, 2017 at 3:38 PM, Ismael Juma wrote: > Thanks Jun, looks good to me. > > Ismael > > On Fri, May 5, 2017 at 11:35 PM, Jun Rao wrote: > > > Hi, Ismael, > > > > Good point. Updated the KIP with

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-05-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999104#comment-15999104 ] Jun Rao commented on KAFKA-5099: [~onurkaraman], thanks for reporting this. It seem the issue is that when

[jira] [Updated] (KAFKA-5183) Transient failure: ReassignPartitionsClusterTest.shouldChangeThrottleOnRerunAndRemoveOnVerify

2017-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté updated KAFKA-5183: - Description:

[jira] [Created] (KAFKA-5183) Transient failure: ReassignPartitionsClusterTest.shouldChangeThrottleOnRerunAndRemoveOnVerify

2017-05-05 Thread JIRA
Xavier Léauté created KAFKA-5183: Summary: Transient failure: ReassignPartitionsClusterTest.shouldChangeThrottleOnRerunAndRemoveOnVerify Key: KAFKA-5183 URL: https://issues.apache.org/jira/browse/KAFKA-5183

[jira] [Comment Edited] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999061#comment-15999061 ] Randall Hauch edited comment on KAFKA-4667 at 5/5/17 10:45 PM: --- Per

[DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-05 Thread Randall Hauch
Hi, all. I've been working on KAFKA-4667 to change the distributed worker of Kafka Connect to look for the topics used to store connector and task configurations, offsets, and status, and if those tasks do not exist to create them using the new AdminClient. To make this as useful as possible and

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Ismael Juma
Thanks Jun, looks good to me. Ismael On Fri, May 5, 2017 at 11:35 PM, Jun Rao wrote: > Hi, Ismael, > > Good point. Updated the KIP with ReplicationBytesInPerSec. > > Thanks, > > Jun > > On Fri, May 5, 2017 at 3:16 PM, Ismael Juma wrote: > > > Thanks for

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Jun Rao
Hi, Ismael, Good point. Updated the KIP with ReplicationBytesInPerSec. Thanks, Jun On Fri, May 5, 2017 at 3:16 PM, Ismael Juma wrote: > Thanks for the KIP, Jun. Good to fix this inconsistency. Do I understand > correctly that we are introducing ReplicationBytesOutPerSec,

[VOTE] KIP-144: Exponential backoff for broker reconnect attempts

2017-05-05 Thread Ismael Juma
Hi all, Given the simple and non controversial nature of the KIP, I would like to start the voting process for KIP-144: Exponential backoff for broker reconnect attempts: https://cwiki.apache.org/confluence/display/KAFKA/KIP-144%3A+Exponential+ backoff+for+broker+reconnect+attempts The vote

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999061#comment-15999061 ] Randall Hauch commented on KAFKA-4667: -- Per [~ewencp], I created

[jira] [Commented] (KAFKA-5150) LZ4 decompression is 4-5x slower than Snappy on small batches / messages

2017-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999059#comment-15999059 ] Xavier Léauté commented on KAFKA-5150: -- I also have a versions of the same patch for 0.10.2.x in case

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Ismael Juma
Thanks for the KIP, Jun. Good to fix this inconsistency. Do I understand correctly that we are introducing ReplicationBytesOutPerSec, but not ReplicationBytesInPerSec? If so, what's the reason? Ismael On Fri, May 5, 2017 at 11:11 PM, Jun Rao wrote: > Hi, Everyone, > > We

Re: [VOTE] KIP-136: Add Listener name and Security Protocol name to SelectorMetrics tags

2017-05-05 Thread Jun Rao
Hi, Edo, Thanks for the KIP. +1 from me. Jun On Fri, May 5, 2017 at 1:26 PM, Edoardo Comar wrote: > Hi, > I'd like to start the vote on this simple KIP : > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 136%3A+Add+Listener+name+and+Security+Protocol+name+to+ >

[DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

2017-05-05 Thread Jun Rao
Hi, Everyone, We created "KIP-153 : Include only client traffic in BytesOutPerSec metric". https://cwiki.apache.org/confluence/display/KAFKA/KIP-153+%3A+Include+only+client+traffic+in+BytesOutPerSec+metric Please take a look and provide your feedback. Thanks, Jun

[jira] [Commented] (KAFKA-5182) Transient failure: RequestQuotaTest.testResponseThrottleTime

2017-05-05 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998994#comment-15998994 ] Ismael Juma commented on KAFKA-5182: cc [~rsivaram] > Transient failure:

[jira] [Updated] (KAFKA-5182) Transient failure: RequestQuotaTest.testResponseThrottleTime

2017-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté updated KAFKA-5182: - Description: Stacktrace {code} java.util.concurrent.ExecutionException:

[jira] [Created] (KAFKA-5182) Transient failure: RequestQuotaTest.testResponseThrottleTime

2017-05-05 Thread JIRA
Xavier Léauté created KAFKA-5182: Summary: Transient failure: RequestQuotaTest.testResponseThrottleTime Key: KAFKA-5182 URL: https://issues.apache.org/jira/browse/KAFKA-5182 Project: Kafka

[jira] [Commented] (KAFKA-5173) SASL tests failing with Could not find a 'KafkaServer' or 'sasl_plaintext.KafkaServer' entry in the JAAS configuration

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998932#comment-15998932 ] ASF GitHub Bot commented on KAFKA-5173: --- GitHub user rajinisivaram opened a pull request:

[GitHub] kafka pull request #2985: KAFKA-5173: Log jaas.config if broker fails to sta...

2017-05-05 Thread rajinisivaram
GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/2985 KAFKA-5173: Log jaas.config if broker fails to start You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka KAFKA-5173-debug

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

2017-05-05 Thread Colin McCabe
Hi Ismael, Thanks for the KIP. This will be a great improvement. > public class AdminClient { >public ListConfigsResult listConfigs(List entities, > ListConfigsOptions options); >public AlterConfigsResult alterConfigs(Map >

Re: [VOTE] KIP-136: Add Listener name and Security Protocol name to SelectorMetrics tags

2017-05-05 Thread Ismael Juma
Thanks for the KIP, +1 (binding) Ismael On Fri, May 5, 2017 at 9:26 PM, Edoardo Comar wrote: > Hi, > I'd like to start the vote on this simple KIP : > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 136%3A+Add+Listener+name+and+Security+Protocol+name+to+ >

[VOTE] KIP-136: Add Listener name and Security Protocol name to SelectorMetrics tags

2017-05-05 Thread Edoardo Comar
Hi, I'd like to start the vote on this simple KIP : https://cwiki.apache.org/confluence/display/KAFKA/KIP-136%3A+Add+Listener+name+and+Security+Protocol+name+to+SelectorMetrics+tags It improves the consumability of the socket-server-metrics emitted by the brokers when they are configured with

Please give me permission to create/edit Kafka Improvement Proposals

2017-05-05 Thread Randall Hauch
See subject. I'm a contributor to Kafka Connect. Thanks! Randall

[jira] [Assigned] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

2017-05-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-1194: Assignee: (was: Jay Kreps) > The kafka broker cannot delete the old log files after

[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

2017-05-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998866#comment-15998866 ] Guozhang Wang commented on KAFKA-1194: -- [~miguno] I am not sure if this issue ever gets resolved

[jira] [Commented] (KAFKA-4743) Add a tool to Reset Consumer Group Offsets

2017-05-05 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998851#comment-15998851 ] Guozhang Wang commented on KAFKA-4743: -- Created 5181 without being aware of this ticket. I think if

[jira] [Created] (KAFKA-5181) Add a new request into admin client to manually setting the committed offsets

2017-05-05 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-5181: Summary: Add a new request into admin client to manually setting the committed offsets Key: KAFKA-5181 URL: https://issues.apache.org/jira/browse/KAFKA-5181 Project:

[jira] [Commented] (KAFKA-5166) Add option "dry run" to Streams application reset tool

2017-05-05 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998798#comment-15998798 ] Bharat Viswanadham commented on KAFKA-5166: --- Hi Matthias, Thank you for info.. Could you please

Re: [VOTE] KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs

2017-05-05 Thread Colin McCabe
Hi all, Thanks for all the discussion. I made some changes to the KIP. * After some discussion about permissions, CreateAcls/DeleteAcls now require Cluster:Alter rather than Cluster:All. * The resource type, operation type, etc. enums now start at 0 rather than at -2, for simplicity's sake *

Need Permissions for creating KIP in Kafka

2017-05-05 Thread BigData dev
Hi, Could you please provide permission for creating KIP in Kafka. username: bharatv email: bigdatadev...@gmail.com

[jira] [Work started] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4667 started by Randall Hauch. > Connect should create internal topics > -

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-05 Thread Mathieu Fenniak
Hey Matthias, My opinion would be that documenting the immutability of the key is the best approach available. Better than requiring the key to be serializable (as with Jeyhun's last pass at the PR), no performance risk. It'd be different if Java had immutable type constraints of some kind. :-)

[jira] [Assigned] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch reassigned KAFKA-4667: Assignee: Randall Hauch > Connect should create internal topics >

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998675#comment-15998675 ] ASF GitHub Bot commented on KAFKA-4667: --- GitHub user rhauch opened a pull request:

[GitHub] kafka pull request #2984: KAFKA-4667 Connect uses AdminClient to create inte...

2017-05-05 Thread rhauch
GitHub user rhauch opened a pull request: https://github.com/apache/kafka/pull/2984 KAFKA-4667 Connect uses AdminClient to create internal topics when needed The backing store for offsets, status, and configs now attempts to use the new AdminClient to look up the internal topics

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Mickael Maison
+1 (non binding) Thanks On Fri, May 5, 2017 at 5:34 PM, Neha Narkhede wrote: > +1 > > On Fri, May 5, 2017 at 9:23 AM Sriram Subramanian wrote: > >> +1 >> >> On Fri, May 5, 2017 at 6:46 AM, Tom Crayford wrote: >> >> > +1 (non-binding)

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

2017-05-05 Thread Jun Rao
Hi, Ismael, Thanks for the KIP. Looks good overall. Just a couple of minor comments. 1. Should AdminClient.listConfigs() take a set of ConfigEntity instead of a list? 2. validateOnly(boolean timeout): Should the time be Integer? Jun On Thu, May 4, 2017 at 7:32 PM, Ismael Juma

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-05 Thread Matthias J. Sax
Agreed about RichFunction. If we follow this path, it should cover all(?) DSL interfaces. About guarding the key -- I am still not sure what to do about it... Maybe it might be enough to document it (and name the key parameter like `readOnlyKey` to make is very clear). Ultimately, I would prefer

[jira] [Commented] (KAFKA-5150) LZ4 decompression is 4-5x slower than Snappy on small batches / messages

2017-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998617#comment-15998617 ] Xavier Léauté commented on KAFKA-5150: -- I expanded the patch for this issue to also address the

[jira] [Commented] (KAFKA-4222) Transient failure in QueryableStateIntegrationTest.queryOnRebalance

2017-05-05 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998598#comment-15998598 ] Matthias J. Sax commented on KAFKA-4222: You can reproduce! Awesome! Great job [~enothereska]! >

[jira] [Commented] (KAFKA-4222) Transient failure in QueryableStateIntegrationTest.queryOnRebalance

2017-05-05 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998573#comment-15998573 ] Eno Thereska commented on KAFKA-4222: - Reopened. Continues to fail. I can repro on Linux with 1 core,

[jira] [Updated] (KAFKA-4222) Transient failure in QueryableStateIntegrationTest.queryOnRebalance

2017-05-05 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-4222: Fix Version/s: (was: 0.10.1.0) 0.11.0.0 > Transient failure in

[jira] [Assigned] (KAFKA-4222) Transient failure in QueryableStateIntegrationTest.queryOnRebalance

2017-05-05 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-4222: --- Assignee: Eno Thereska > Transient failure in QueryableStateIntegrationTest.queryOnRebalance

[jira] [Updated] (KAFKA-5169) KafkaConsumer.close should be idempotent

2017-05-05 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5169: --- Resolution: Fixed Status: Resolved (was: Patch Available) > KafkaConsumer.close

[jira] [Updated] (KAFKA-5168) Cleanup delayed produce purgatory during partition emmigration

2017-05-05 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-5168: -- Resolution: Not A Problem Status: Resolved (was: Patch Available) This is already handled >

[jira] [Commented] (KAFKA-5168) Cleanup delayed produce purgatory during partition emmigration

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998552#comment-15998552 ] ASF GitHub Bot commented on KAFKA-5168: --- Github user dguy closed the pull request at:

[GitHub] kafka pull request #2965: KAFKA-5168: Cleanup delayed produce purgatory duri...

2017-05-05 Thread dguy
Github user dguy closed the pull request at: https://github.com/apache/kafka/pull/2965 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-4762) Consumer throwing RecordTooLargeException even when messages are not that large

2017-05-05 Thread Vipul Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vipul Singh resolved KAFKA-4762. Resolution: Not A Problem > Consumer throwing RecordTooLargeException even when messages are not

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Neha Narkhede
+1 On Fri, May 5, 2017 at 9:23 AM Sriram Subramanian wrote: > +1 > > On Fri, May 5, 2017 at 6:46 AM, Tom Crayford wrote: > > > +1 (non-binding) > > > > On Fri, 5 May 2017 at 00:28, Michael André Pearce < > > michael.andre.pea...@me.com> wrote: > > > > >

[jira] [Updated] (KAFKA-5135) Controller Health Metrics (KIP-143)

2017-05-05 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5135: --- Status: Patch Available (was: Open) > Controller Health Metrics (KIP-143) >

[jira] [Commented] (KAFKA-5135) Controller Health Metrics (KIP-143)

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998530#comment-15998530 ] ASF GitHub Bot commented on KAFKA-5135: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request #2983: KAFKA-5135: Controller Health Metrics (KIP-143)

2017-05-05 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2983 KAFKA-5135: Controller Health Metrics (KIP-143) You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka

Re: [DISCUSS] KIP 151 - Expose Connector type in REST API

2017-05-05 Thread Konstantine Karantasis
Thank you for the KIP. It's a nice improvement. Two small suggestions: 1) Let's not use all caps to describe the type of the connector. "Source" and "Sink" seem more appropriate (but even all lower case would be better). 2) It's been discussed in other contexts recently, but I believe finally

[jira] [Commented] (KAFKA-4764) Improve diagnostics for SASL authentication failures

2017-05-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998527#comment-15998527 ] Jun Rao commented on KAFKA-4764: [~rsivaram], thanks for the explanation. That makes sense. > Improve

[jira] [Assigned] (KAFKA-5180) Transient failure: ControllerIntegrationTest.testControllerMoveIncrementsControllerEpoch

2017-05-05 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman reassigned KAFKA-5180: --- Assignee: Onur Karaman > Transient failure: >

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Sriram Subramanian
+1 On Fri, May 5, 2017 at 6:46 AM, Tom Crayford wrote: > +1 (non-binding) > > On Fri, 5 May 2017 at 00:28, Michael André Pearce < > michael.andre.pea...@me.com> wrote: > > > +1 , really good to see better operational visibility being added to the > > broker. > > > > Sent

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-05 Thread Jeyhun Karimov
Hi Matthias, I think extending KIP to include RichFunctions totally makes sense. So, we don't want to guard the keys because it is costly. If we introduce RichFunctions I think it should not be limited only the 3 interfaces proposed in KIP but to wide range of interfaces. Please correct me if I

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-05-05 Thread Jay Kreps
I haven't digested the proposal but the use case is pretty common. An example would be the "customer 360" or "unified customer profile" use case we often use. In that use case you have a dozen systems each of which has some information about your customer (account details, settings, billing info,

[jira] [Work started] (KAFKA-4996) Fix findbugs multithreaded correctness warnings for streams

2017-05-05 Thread Amit Daga (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4996 started by Amit Daga. > Fix findbugs multithreaded correctness warnings for streams >

Re: [DISCUSS] KIP-138: Change punctuate semantics

2017-05-05 Thread Damian Guy
Thanks Michael - LGTM On Fri, 5 May 2017 at 12:04 Michal Borowiecki wrote: > I shall move all alternatives other than the main proposal into the > Rejected Alternatives section and if I hear any objections, I'll move those > back up and we'll discuss further. > >

[jira] [Commented] (KAFKA-5174) RocksDb might stall in environments with 1 core only

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998373#comment-15998373 ] ASF GitHub Bot commented on KAFKA-5174: --- GitHub user enothereska opened a pull request:

[GitHub] kafka pull request #2982: KAFKA-5174: Have at least 2 threads

2017-05-05 Thread enothereska
GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/2982 KAFKA-5174: Have at least 2 threads You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka KAFKA-5174-1-core Alternatively you

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Tom Crayford
+1 (non-binding) On Fri, 5 May 2017 at 00:28, Michael André Pearce < michael.andre.pea...@me.com> wrote: > +1 , really good to see better operational visibility being added to the > broker. > > Sent from my iPhone > > > On 5 May 2017, at 03:34, Ismael Juma wrote: > > > > Hi

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

2017-05-05 Thread Mathieu Fenniak
Hi Konstantine, As KIP-151 is currently adding the connector class information to the Kafka Connect REST API, it might be a nice small detail to augment this KIP to add the module name to the same API.

[GitHub] kafka pull request #2981: KAFKA-5173: Add logging to dump JaasContext before...

2017-05-05 Thread baluchicken
GitHub user baluchicken opened a pull request: https://github.com/apache/kafka/pull/2981 KAFKA-5173: Add logging to dump JaasContext before writing it to file @rajinisivaram plz review. You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (KAFKA-5173) SASL tests failing with Could not find a 'KafkaServer' or 'sasl_plaintext.KafkaServer' entry in the JAAS configuration

2017-05-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998322#comment-15998322 ] ASF GitHub Bot commented on KAFKA-5173: --- GitHub user baluchicken opened a pull request:

[jira] [Commented] (KAFKA-5178) Potential Performance Degradation in Kafka Producer when using Multiple Threads

2017-05-05 Thread huxi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998316#comment-15998316 ] huxi commented on KAFKA-5178: - I suggest splitting the test into multiple sub-testings in order to find out

  1   2   >