Re: [DISCUSS] KIP-44 - Allow Kafka to have a customized security protocol

2016-01-26 Thread Rajini Sivaram
Hi Tao, I have a couple of questions: 1. Is there a reason why you wouldn't want to implement a custom SASL mechanism to use your authentication mechanism? SASL itself aims to provide pluggable authentication mechanisms. 2. The KIP suggests that you are interested in plugging in a

Re: Kafka KIP meeting Jan 26 at 11:00am PST

2016-01-26 Thread Rajini Sivaram
Jun, Can you send me an invite, please? Thank you... Regards, Rajini On Mon, Jan 25, 2016 at 10:56 PM, Jun Rao wrote: > Hi, Everyone, > > We will have a Kafka KIP meeting tomorrow at 11:00am PST. If you plan to > attend but haven't received an invite, please let me know.

Re: [DISCUSS] KIP-44 - Allow Kafka to have a customized security protocol

2016-01-26 Thread Ismael Juma
Tao, Thanks for the KIP. As others are saying, it would be helpful to have more details on why a new SASL mechanism was rejected in this KIP. An example of how using a new SASL mechanism would be more complex when compared to using a customised security protocol (for example) would help. Ismael

Re: [DISCUSS] KIP-44 - Allow Kafka to have a customized security protocol

2016-01-26 Thread Harsha
SASL itself can provide pluggable authentication , why not extend there. There is also proposal for SASL/PLAIN which does extend the current authentication options. I think thats what Rajini is also talking about. -Harsha On Tue, Jan 26, 2016, at 01:56 AM, tao xiao wrote: > Hi Rajini, > > I

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Ismael Juma
Hi Anna and Neha, I think it makes a lot of sense to try and keep the interface lean and to add more methods later when/if there is a need. What is the current thinking with regards to compatibility when/if we add new methods? A few options come to mind: 1. Change the interface to an abstract

[jira] [Commented] (KAFKA-3149) Extend SASL implementation to support more mechanisms

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117091#comment-15117091 ] ASF GitHub Bot commented on KAFKA-3149: --- GitHub user rajinisivaram opened a pull request:

[jira] [Created] (KAFKA-3150) kafka.tools.UpdateOffsetsInZK not work (sasl enabled)

2016-01-26 Thread linbao111 (JIRA)
linbao111 created KAFKA-3150: Summary: kafka.tools.UpdateOffsetsInZK not work (sasl enabled) Key: KAFKA-3150 URL: https://issues.apache.org/jira/browse/KAFKA-3150 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-26 Thread Ismael Juma
Hi Rajini, Thanks for the KIP. As stated in the KIP, it does not address "Support for multiple SASL mechanisms within a broker". Maybe we should also mention this in the "Rejected Alternatives" section with the reasoning. I think it's particularly relevant to understand if it's not being proposed

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

2016-01-26 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117071#comment-15117071 ] Eno Thereska commented on KAFKA-3068: - Ok, I'll revert to using the 0.8.2 solution (by updating the

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-26 Thread Rajini Sivaram
Ismael, Thank you for your review. The main reason I didn't address the support for multiple mechanisms within a broker is because it requires changes to the wire protocol to propagate mechanisms. But I do agree that we need to understand whether it would be even harder to support this in the

Re: [DISCUSS] KIP-44 - Allow Kafka to have a customized security protocol

2016-01-26 Thread tao xiao
Hi Rajini, I think I need to rephrase some of the wordings in the KIP. I meant to provide a customized security protocol which may/may not include SSL underneath. With CUSTOMIZED security protocol users have the ability to plugin both authentication and security communication components. On

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-26 Thread Rajini Sivaram
Hi Tao, Thank you for the review. The changes I had in mind are in the PR https://github.com/apache/kafka/pull/812. Login for non-Kerberos protocols contains very little logic. I was expecting that combined with a custom login module specified in JAAS configuration, this would give sufficient

[jira] [Commented] (KAFKA-3006) Make collection default container type for sequences in the consumer API

2016-01-26 Thread Pierre-Yves Ritschard (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116960#comment-15116960 ] Pierre-Yves Ritschard commented on KAFKA-3006: -- [~gwenshap] - thanks! the KIP is up:

[jira] [Created] (KAFKA-3151) kafka-consumer-groups.sh fail with sasl enabled

2016-01-26 Thread linbao111 (JIRA)
linbao111 created KAFKA-3151: Summary: kafka-consumer-groups.sh fail with sasl enabled Key: KAFKA-3151 URL: https://issues.apache.org/jira/browse/KAFKA-3151 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-26 Thread tao xiao
Hi Rajini, I think it makes sense to change LoginManager or Login to an interface which users can extend to provide their own logic of login otherwise it is hard for users to implement a custom SASL mechanism but have no control over login On Tue, 26 Jan 2016 at 18:45 Ismael Juma

[GitHub] kafka pull request: KAFKA-3149: Extend SASL implementation to supp...

2016-01-26 Thread rajinisivaram
GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/812 KAFKA-3149: Extend SASL implementation to support more mechanisms Code changes corresponding to KIP-43 to enable review of the KIP. You can merge this pull request into a Git repository by

Build failed in Jenkins: kafka-trunk-jdk7 #993

2016-01-26 Thread Apache Jenkins Server
See Changes: [cshapi] MINOR: remove FilteredIterator -- [...truncated 3302 lines...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[0] PASSED kafka.log.BrokerCompressionTest >

[jira] [Assigned] (KAFKA-2673) Log JmxTool output to logger

2016-01-26 Thread chen zhu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen zhu reassigned KAFKA-2673: --- Assignee: chen zhu > Log JmxTool output to logger > > >

[jira] [Assigned] (KAFKA-2676) AclCommandTest has wrong package name

2016-01-26 Thread chen zhu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen zhu reassigned KAFKA-2676: --- Assignee: chen zhu > AclCommandTest has wrong package name > --

[jira] [Assigned] (KAFKA-3142) Improve error message in kstreams

2016-01-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-3142: Assignee: Guozhang Wang > Improve error message in kstreams >

[jira] [Assigned] (KAFKA-3125) Exception Hierarchy for Streams

2016-01-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-3125: Assignee: Guozhang Wang > Exception Hierarchy for Streams >

[jira] [Updated] (KAFKA-3132) URI scheme in "listeners" property should not be case-sensitive

2016-01-26 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-3132: Resolution: Fixed Fix Version/s: 0.9.1.0 Status: Resolved (was: Patch Available)

Build failed in Jenkins: kafka-trunk-jdk8 #320

2016-01-26 Thread Apache Jenkins Server
See Changes: [cshapi] MINOR: remove FilteredIterator -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker Ubuntu ubuntu yahoo-not-h2) in

Build failed in Jenkins: kafka-trunk-jdk7 #994

2016-01-26 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: join test for windowed keys -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-4 (docker Ubuntu ubuntu4

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

2016-01-26 Thread Guozhang Wang
One motivation of my proposal is actually to avoid any clients trying to read the timestamp type from the topic metadata response and behave differently since: 1) topic metadata response is not always in-sync with the source-of-truth (ZK), hence when the clients realized that the config has

[jira] [Created] (KAFKA-3155) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-26 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-3155: Summary: Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack Key: KAFKA-3155 URL:

[GitHub] kafka pull request: MINOR: join test for windowed keys

2016-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/814 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: MINOR: remove FilteredIterator

2016-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/816 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: valueFactory's key is consumerId

2016-01-26 Thread zqhxuyuan
GitHub user zqhxuyuan opened a pull request: https://github.com/apache/kafka/pull/817 valueFactory's key is consumerId the key of partitionAssignment's valueFactory is consumerId, not topic. partitionAssignment.getAndMaybePut(threadId.consumer) You can merge this pull request

[jira] [Resolved] (KAFKA-3142) Improve error message in kstreams

2016-01-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3142. -- Resolution: Fixed This is fixed as part of KAFKA-3125. > Improve error message in kstreams >

[jira] [Updated] (KAFKA-3142) Improve error message in kstreams

2016-01-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3142: - Fix Version/s: 0.9.1.0 > Improve error message in kstreams > - >

[jira] [Commented] (KAFKA-2985) Consumer group stuck in rebalancing state

2016-01-26 Thread Federico Fissore (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116905#comment-15116905 ] Federico Fissore commented on KAFKA-2985: - [~hachikuji] you're right. Sorry for having bothered

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

2016-01-26 Thread Ismael Juma
Thanks Pierre. Including the dev mailing list. A few comments: 1. It's worth mentioning that the KafkaConsumer has the @InterfaceStability.Unstable annotation. 2. It would be good to show the existing signatures of the methods being changed before we show the changed signatures. 3. The proposed

[jira] [Commented] (KAFKA-3110) can't set cluster acl for a user to CREATE topics without first creating a topic

2016-01-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117246#comment-15117246 ] Thomas Graves commented on KAFKA-3110: -- I was using 0.9 release so I'll give it a try with trunk. >

[jira] [Commented] (KAFKA-1189) kafka-server-stop.sh doesn't stop broker

2016-01-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117254#comment-15117254 ] Ismael Juma commented on KAFKA-1189: [~linwukang], please file a new issue. > kafka-server-stop.sh

[jira] [Commented] (KAFKA-3150) kafka.tools.UpdateOffsetsInZK not work (sasl enabled)

2016-01-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117261#comment-15117261 ] Ismael Juma commented on KAFKA-3150: UpdateOffsetsInZK requires a PLAINTEXT port to be open by the

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Todd Palino
Finally got a chance to take a look at this. I won’t be able to make the KIP meeting due to a conflict. I’m somewhat disappointed in this proposal. I think that the explicit exclusion of modification of the messages is short-sighted, and not accounting for it now is going to bite us later. Jay,

[jira] [Commented] (KAFKA-3151) kafka-consumer-groups.sh fail with sasl enabled

2016-01-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117285#comment-15117285 ] Ismael Juma commented on KAFKA-3151: You need to configure the tool to use SASL (similarly to how you

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

2016-01-26 Thread Pierre-Yves Ritschard
Hi Ismael, Thanks for the review, I will act on these a bit later today. - pyr Ismael Juma writes: > Thanks Pierre. Including the dev mailing list. > > A few comments: > > 1. It's worth mentioning that the KafkaConsumer has the > @InterfaceStability.Unstable annotation. > 2. It would be

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-26 Thread Rajini Sivaram
Ismael, I have written up a section on supporting multiple mechanisms within a Kafka broker. At the moment, it is under "Rejected Alternatives", even though having thought about it, we should possibly include it in this KIP, unless we are sure it is not going to come up as a requirement. We don't

[jira] [Created] (KAFKA-3152) kafka-acl doesn't allow space in principal name

2016-01-26 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-3152: -- Summary: kafka-acl doesn't allow space in principal name Key: KAFKA-3152 URL: https://issues.apache.org/jira/browse/KAFKA-3152 Project: Kafka Issue Type: Bug

[jira] [Assigned] (KAFKA-3152) kafka-acl doesn't allow space in principal name

2016-01-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reassigned KAFKA-3152: -- Assignee: Ismael Juma > kafka-acl doesn't allow space in principal name >

[GitHub] kafka pull request: KAFKA-3125: Add Kafka Streams Exceptions

2016-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/809 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: KAFKA-3132: URI scheme in "listeners" property...

2016-01-26 Thread granthenke
GitHub user granthenke reopened a pull request: https://github.com/apache/kafka/pull/811 KAFKA-3132: URI scheme in "listeners" property should not be case-sen… …sitive You can merge this pull request into a Git repository by running: $ git pull

[GitHub] kafka pull request: KAFKA-3132: URI scheme in "listeners" property...

2016-01-26 Thread granthenke
Github user granthenke closed the pull request at: https://github.com/apache/kafka/pull/811 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3132) URI scheme in "listeners" property should not be case-sensitive

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117582#comment-15117582 ] ASF GitHub Bot commented on KAFKA-3132: --- GitHub user granthenke reopened a pull request:

[jira] [Commented] (KAFKA-3132) URI scheme in "listeners" property should not be case-sensitive

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117581#comment-15117581 ] ASF GitHub Bot commented on KAFKA-3132: --- Github user granthenke closed the pull request at:

[jira] [Commented] (KAFKA-3125) Exception Hierarchy for Streams

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117580#comment-15117580 ] ASF GitHub Bot commented on KAFKA-3125: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-3125) Exception Hierarchy for Streams

2016-01-26 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3125. -- Resolution: Fixed Fix Version/s: 0.9.1.0 Issue resolved by pull request

Build failed in Jenkins: kafka-trunk-jdk7 #989

2016-01-26 Thread Apache Jenkins Server
See Changes: [me] KAFKA-3125: Add Kafka Streams Exceptions -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker Ubuntu ubuntu

[jira] [Commented] (KAFKA-2985) Consumer group stuck in rebalancing state

2016-01-26 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117616#comment-15117616 ] Jason Gustafson commented on KAFKA-2985: [~fridrik] No worries, I'm just glad to see people

[jira] [Commented] (KAFKA-1791) Corrupt index after safe shutdown and restart

2016-01-26 Thread sandeep reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117667#comment-15117667 ] sandeep reddy commented on KAFKA-1791: -- I deleted corrupted index's and restarted the Kafka broker. I

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Anna Povzner
Thanks Ismael and Todd for your feedback! I agree about coming up with lean, but useful interfaces that will be easy to extend later. When we discuss the minimal set of producer and consumer interceptor API in today’s KIP meeting (discussion item #2 in my previous email), lets compare two

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Mayuresh Gharat
Hi, I won't be able to make it to KIP hangout due to conflict. Anna, here is the use case where knowing if there are messages in the RecordAccumulator left to be sent to the kafka cluster for a topic is useful. 1) Consider a pipeline : A ---> Mirror-maker -> B 2) We have a topic T in

[GitHub] kafka pull request: KAFKA-2478: Fix manual committing example in j...

2016-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/210 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2478) KafkaConsumer javadoc example seems wrong

2016-01-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2478: - Resolution: Fixed Fix Version/s: 0.9.0.1 Status: Resolved (was: Patch

[jira] [Commented] (KAFKA-2478) KafkaConsumer javadoc example seems wrong

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117762#comment-15117762 ] ASF GitHub Bot commented on KAFKA-2478: --- Github user asfgit closed the pull request at:

Jenkins build is back to normal : kafka-trunk-jdk8 #316

2016-01-26 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-trunk-jdk8 #317

2016-01-26 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2478: Fix manual committing example in javadoc -- [...truncated 82 lines...]

[jira] [Assigned] (KAFKA-3116) Failure to build

2016-01-26 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-3116: -- Assignee: Vahid Hashemian > Failure to build > - > >

[GitHub] kafka pull request: KAFKA-3116: Specify minimum Gradle version req...

2016-01-26 Thread vahidhashemian
GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/813 KAFKA-3116: Specify minimum Gradle version required in Readme You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka

[jira] [Commented] (KAFKA-3116) Failure to build

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117864#comment-15117864 ] ASF GitHub Bot commented on KAFKA-3116: --- GitHub user vahidhashemian opened a pull request:

Build failed in Jenkins: kafka_0.9.0_jdk7 #95

2016-01-26 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2478: Fix manual committing example in javadoc -- [...truncated 1507 lines...] kafka.integration.PrimitiveApiTest >

Re: Kafka KIP meeting Jan 26 at 11:00am PST

2016-01-26 Thread Jun Rao
The following are the notes from today's KIP discussion. - KIP-42: We agreed to leave the broker side interceptor for another KIP. On the client side, people favor the 2nd option in Anna's proposal. Anna will update the wiki accordingly. - KIP-43: We discussed whether there is a need

Re: Kafka KIP meeting Jan 26 at 11:00am PST

2016-01-26 Thread Anna Povzner
I will send more detailed meeting notes regarding KIP-42 in the KIP-42 discussion thread. Thanks, Anna On Tue, Jan 26, 2016 at 1:21 PM, Jun Rao wrote: > The following are the notes from today's KIP discussion. > > >- KIP-42: We agreed to leave the broker side interceptor

Build failed in Jenkins: kafka-trunk-jdk7 #990

2016-01-26 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2478: Fix manual committing example in javadoc -- [...truncated 6887 lines...] org.apache.kafka.connect.runtime.rest.resources.ConnectorsResourceTest >

[jira] [Commented] (KAFKA-1791) Corrupt index after safe shutdown and restart

2016-01-26 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118078#comment-15118078 ] Jun Rao commented on KAFKA-1791: Could you share the error and the stack trace in the log? Thanks, >

[jira] [Updated] (KAFKA-3088) 0.9.0.0 broker crash on receipt of produce request with empty client ID

2016-01-26 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3088: --- Assignee: (was: Jun Rao) > 0.9.0.0 broker crash on receipt of produce request with empty client ID >

[jira] [Assigned] (KAFKA-3088) 0.9.0.0 broker crash on receipt of produce request with empty client ID

2016-01-26 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke reassigned KAFKA-3088: -- Assignee: Grant Henke > 0.9.0.0 broker crash on receipt of produce request with empty client

[jira] [Commented] (KAFKA-3088) 0.9.0.0 broker crash on receipt of produce request with empty client ID

2016-01-26 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118088#comment-15118088 ] Grant Henke commented on KAFKA-3088: I will pick this one up quick if thats alright. > 0.9.0.0

[GitHub] kafka pull request: MINOR: jointest for windowed keys

2016-01-26 Thread ymatsuda
GitHub user ymatsuda opened a pull request: https://github.com/apache/kafka/pull/814 MINOR: jointest for windowed keys @guozhangwang You can merge this pull request into a Git repository by running: $ git pull https://github.com/ymatsuda/kafka windowed_key_join_test

[GitHub] kafka pull request: KAFKA-3116: Specify minimum Gradle version req...

2016-01-26 Thread vahidhashemian
Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/813 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request: KAFKA-3116: Specify minimum Gradle version req...

2016-01-26 Thread vahidhashemian
GitHub user vahidhashemian reopened a pull request: https://github.com/apache/kafka/pull/813 KAFKA-3116: Specify minimum Gradle version required in Readme You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka

[jira] [Commented] (KAFKA-3116) Failure to build

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118142#comment-15118142 ] ASF GitHub Bot commented on KAFKA-3116: --- Github user vahidhashemian closed the pull request at:

[jira] [Commented] (KAFKA-3116) Failure to build

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118143#comment-15118143 ] ASF GitHub Bot commented on KAFKA-3116: --- GitHub user vahidhashemian reopened a pull request:

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

2016-01-26 Thread Becket Qin
Bump up this thread per discussion on the KIP hangout. During the implementation of the KIP, Guozhang raised another proposal on how to indicate the message timestamp type used by messages. So we want to see people's opinion on this proposal. The difference between current and the new proposal

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Anna Povzner
Hi All, Here is meeting notes from today’s KIP meeting: 1. We agreed to keep the scope of this KIP to be producer and consumer interceptors only. Broker-side interceptor will be added later as a separate KIP. The reasons were already mentioned in this thread, but the summary is: * Broker

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Todd Palino
This looks good. As noted, having one mutable interceptor on each side allows for the use cases we can envision right now, and I think that’s going to provide a great deal of opportunity for implementing things like audit, especially within a multi-tenant environment. Looking forward to getting

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

2016-01-26 Thread Jay Kreps
I'm in favor of Guozhang's proposal. I think that logic is a bit hacky, but I agree that this is better than the alternative, and the hackiness only effects people using log append time which I think will be pretty uncommon. I think setting that bit will have the additional added value that

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

2016-01-26 Thread Becket Qin
My hesitation for the changed protocol is that I think If we will have topic configuration returned in the topic metadata, the current protocol makes more sense. Because the timestamp type is a topic level setting so we don't need to put it into each message. That is assuming the timestamp type

[jira] [Created] (KAFKA-3153) Serializer/Deserializer Registration and Type inference

2016-01-26 Thread Yasuhiro Matsuda (JIRA)
Yasuhiro Matsuda created KAFKA-3153: --- Summary: Serializer/Deserializer Registration and Type inference Key: KAFKA-3153 URL: https://issues.apache.org/jira/browse/KAFKA-3153 Project: Kafka

[GitHub] kafka pull request: KAFKA-3116: Specify minimum Gradle version req...

2016-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/813 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3116) Failure to build

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118280#comment-15118280 ] ASF GitHub Bot commented on KAFKA-3116: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-3116) Failure to build

2016-01-26 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-3116. - Resolution: Fixed Fix Version/s: 0.9.1.0 Issue resolved by pull request 813

Build failed in Jenkins: kafka-trunk-jdk8 #318

2016-01-26 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-3116; Specify minimum Gradle version required in Readme -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker

[jira] [Updated] (KAFKA-3141) kafka-acls.sh throws ArrayIndexOutOfBoundsException for an invalid authorizer-property

2016-01-26 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-3141: -- Status: Patch Available (was: Open) > kafka-acls.sh throws ArrayIndexOutOfBoundsException for

[jira] [Commented] (KAFKA-3092) Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118330#comment-15118330 ] ASF GitHub Bot commented on KAFKA-3092: --- GitHub user hachikuji opened a pull request:

[GitHub] kafka pull request: KAFKA-3092: Replace SinkTask onPartitionsAssig...

2016-01-26 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/815 KAFKA-3092: Replace SinkTask onPartitionsAssigned/onPartitionsRevoked with open/close You can merge this pull request into a Git repository by running: $ git pull

[GitHub] kafka pull request: MINOR: remove FilteredIterator

2016-01-26 Thread ymatsuda
GitHub user ymatsuda opened a pull request: https://github.com/apache/kafka/pull/816 MINOR: remove FilteredIterator @guozhangwang removing an unused class, FilteredIterator, and its test. You can merge this pull request into a Git repository by running: $ git pull

Jenkins build is back to normal : kafka-trunk-jdk7 #991

2016-01-26 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-3147) Memory records is not writable in MirrorMaker

2016-01-26 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118362#comment-15118362 ] Jun Rao commented on KAFKA-3147: I think the issue could be related to "abort.on.send.failure". There is a

[GitHub] kafka pull request: KAFKA-3132: URI scheme in "listeners" property...

2016-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/811 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3132) URI scheme in "listeners" property should not be case-sensitive

2016-01-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118364#comment-15118364 ] ASF GitHub Bot commented on KAFKA-3132: --- Github user asfgit closed the pull request at:

[jira] [Work started] (KAFKA-3152) kafka-acl doesn't allow space in principal name

2016-01-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3152 started by Ismael Juma. -- > kafka-acl doesn't allow space in principal name >

[jira] [Commented] (KAFKA-3152) kafka-acl doesn't allow space in principal name

2016-01-26 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118401#comment-15118401 ] Ismael Juma commented on KAFKA-3152: This is a trivial bug in the kafka-acls shell script. Many other

Re: Request to be added to contributor list

2016-01-26 Thread Gwen Shapira
Will be happy to add you. What's your username in Apache's Jira? Gwen On Mon, Jan 25, 2016 at 8:43 PM, Chen Zhu wrote: > Hi, > > I am a CS major student. My jira username is chenzhu, can someone please > add me to the contributor list? > > Thank you, > Chen Zhu >

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-26 Thread tao xiao
Hi Rajini, One requirement I have is to refresh the login token every X hours. Like what the Kerberos login does I need to have a background thread that refreshes the token periodically. I understand most of the login logic would be simple but it is good that we can expose the logic login to

[jira] [Commented] (KAFKA-3151) kafka-consumer-groups.sh fail with sasl enabled

2016-01-26 Thread linbao111 (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118474#comment-15118474 ] linbao111 commented on KAFKA-3151: -- sasl is enabled on my test cluster,following tools work fine on the

[jira] [Commented] (KAFKA-3150) kafka.tools.UpdateOffsetsInZK not work (sasl enabled)

2016-01-26 Thread linbao111 (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118477#comment-15118477 ] linbao111 commented on KAFKA-3150: -- so how can i manage topic offset on sasl-enabled cluster? i also find

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-26 Thread Mayuresh Gharat
Hi Anna, Thanks a lot for summarizing the discussion on this kip. It LGTM. This is really nice : We decided not to add any callbacks to producer and consumer interceptors that will depend on internal implementation as part of this KIP. *However, it is possible to add them later as part of

  1   2   >