Re: [DISCUSS] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

Hi Ashish, Thanks for the updates. I have a few questions below: > Move following interfaces to new package, org.apche.kafka.authorizer. > >1. Authorizer >2. Acl >3. Operation >4. PermissionType >5. Resource >6. ResourceType >7. KafkaPrincipal >8. Session > > This

[GitHub] kafka pull request: KAFKA-3586: Revert to default quota when clien...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/1238 KAFKA-3586: Revert to default quota when client quota override is deleted Update client quota when override is deleted from config You can merge this pull request into a Git repository by

[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens

[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248064#comment-15248064 ] Sriharsha Chintalapani commented on KAFKA-1696: --- [~singhashish] Yes we are actively working

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

Thanks for review Jitendra. I don't like the idea of infinite lifetime but I see the Streaming use case. Even for Streaming use case I was hoping there will be some notion of master/driver that can get new delegation tokens at fixed interval and distribute to workers. If that is not the case for

[jira] [Assigned] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-3579: -- Assignee: Vahid Hashemian > TopicCommand references outdated consumer property

[jira] [Comment Edited] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248198#comment-15248198 ] Jay Kreps edited comment on KAFKA-3565 at 4/19/16 5:11 PM: --- I think

[jira] [Commented] (KAFKA-3117) Fail test at: PlaintextConsumerTest. testAutoCommitOnRebalance

[ https://issues.apache.org/jira/browse/KAFKA-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248119#comment-15248119 ] Jason Gustafson commented on KAFKA-3117: I ran this test case for about an hour and couldn't

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248198#comment-15248198 ] Jay Kreps commented on KAFKA-3565: -- I think [~becket_qin] noted a good possible explanation. The dominate

[jira] [Commented] (KAFKA-3147) Memory records is not writable in MirrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247835#comment-15247835 ] Cedrik Neumann commented on KAFKA-3147: --- It would be very much appreciated to get fixes for those

[jira] [Created] (KAFKA-3586) Quotas do not revert to default values when override config is deleted

Rajini Sivaram created KAFKA-3586: - Summary: Quotas do not revert to default values when override config is deleted Key: KAFKA-3586 URL: https://issues.apache.org/jira/browse/KAFKA-3586 Project:

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247802#comment-15247802 ] Flavio Junqueira commented on KAFKA-3042: - [~junrao] It makes sense, thanks for the analysis.

Re: [DISCUSS] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

I have updated KIP-50 and PR as per recent discussions. Please take a look. @Harsha / Don, it would be nice if you guys can review the KIP and PR as well.

[jira] [Updated] (KAFKA-3421) Update docs with new connector features

[ https://issues.apache.org/jira/browse/KAFKA-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3421: - Resolution: Fixed Fix Version/s: (was: 0.10.0.0)

[GitHub] kafka pull request: KAFKA-3421: Update docs with new connector fea...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1227 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3421) Update docs with new connector features

[ https://issues.apache.org/jira/browse/KAFKA-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248321#comment-15248321 ] ASF GitHub Bot commented on KAFKA-3421: --- Github user asfgit closed the pull request at:

[jira] [Commented] (KAFKA-3564) Count metric always increments by 1.0

[ https://issues.apache.org/jira/browse/KAFKA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248370#comment-15248370 ] Guozhang Wang commented on KAFKA-3564: -- Your case fits with Rate, and your can do {{Rate(Count)}} or

[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens

[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248268#comment-15248268 ] Ashish K Singh commented on KAFKA-1696: --- Good to know [~harsha_ch]! Thanks for the quick response.

[jira] [Created] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

Kiran created KAFKA-3587: Summary: LogCleaner fails due to incorrect offset map computation on a replica Key: KAFKA-3587 URL: https://issues.apache.org/jira/browse/KAFKA-3587 Project: Kafka Issue

[jira] [Commented] (KAFKA-3117) Fail test at: PlaintextConsumerTest. testAutoCommitOnRebalance

[ https://issues.apache.org/jira/browse/KAFKA-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248331#comment-15248331 ] Jason Gustafson commented on KAFKA-3117: On second thought, the case I mentioned seems to already

Build failed in Jenkins: kafka-trunk-jdk8 #536

See Changes: [me] KAFKA-3421: Update docs with new connector features -- [...truncated 912 lines...] kafka.coordinator.GroupMetadataTest > testCanRebalanceWhenAwaitingSync PASSED

[jira] [Commented] (KAFKA-3573) JSON (de)serialization for kafka-client

[ https://issues.apache.org/jira/browse/KAFKA-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248377#comment-15248377 ] Guozhang Wang commented on KAFKA-3573: -- The goal is to move the already existing JSON serde that was

[jira] [Commented] (KAFKA-3564) Count metric always increments by 1.0

[ https://issues.apache.org/jira/browse/KAFKA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248385#comment-15248385 ] Michael Coon commented on KAFKA-3564: - If you look at the impl for Count, it only increments by 1.0

[GitHub] kafka pull request: KAFKA-3579 - Update reference to the outdated ...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1239 KAFKA-3579 - Update reference to the outdated consumer property Replace references to the old consumer property 'fetch.message.max.bytes' with the corresponding property in the new consumer

[jira] [Commented] (KAFKA-3564) Count metric always increments by 1.0

[ https://issues.apache.org/jira/browse/KAFKA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248399#comment-15248399 ] Guozhang Wang commented on KAFKA-3564: -- I see your point now, in that case you can just call

[jira] [Commented] (KAFKA-3564) Count metric always increments by 1.0

[ https://issues.apache.org/jira/browse/KAFKA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248352#comment-15248352 ] Guozhang Wang commented on KAFKA-3564: -- Sorry I took the name wrong, it was {{Total}}:

[jira] [Updated] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-3579: --- Status: Patch Available (was: Open) > TopicCommand references outdated consumer property

[jira] [Commented] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248395#comment-15248395 ] ASF GitHub Bot commented on KAFKA-3579: --- GitHub user vahidhashemian opened a pull request:

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

Did not realize that you folks were not able to access the discuss thread. Harsha has explicitly added Jitendra and posted his comments, so we should be good to carry on the discussion there. Thanks Harsha! On Mon, Apr 18, 2016 at 11:23 AM, Jitendra Pandey wrote: >

[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens

[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247978#comment-15247978 ] Ashish K Singh commented on KAFKA-1696: --- [~parth.brahmbhatt] are you still working on this? > Kafka

[jira] [Commented] (KAFKA-3586) Quotas do not revert to default values when override config is deleted

[ https://issues.apache.org/jira/browse/KAFKA-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247982#comment-15247982 ] ASF GitHub Bot commented on KAFKA-3586: --- GitHub user rajinisivaram opened a pull request:

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

On Mon, Apr 18, 2016 at 11:26 AM, Harsha wrote: > Unifying the two discussion threads on this KIP. > > Here is the response from Jitendra > > "The need for a large number of clients that are running all over the > cluster that authenticate with Kafka brokers, is very similar to

[jira] [Updated] (KAFKA-3586) Quotas do not revert to default values when override config is deleted

[ https://issues.apache.org/jira/browse/KAFKA-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram updated KAFKA-3586: -- Status: Patch Available (was: Open) > Quotas do not revert to default values when override

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248498#comment-15248498 ] Onur Karaman commented on KAFKA-3042: - It looks like LeaderAndIsrRequest structure is identical to

[jira] [Resolved] (KAFKA-3573) JSON (de)serialization for kafka-client

[ https://issues.apache.org/jira/browse/KAFKA-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Stepanov resolved KAFKA-3573. -- Resolution: Invalid > JSON (de)serialization for kafka-client >

Jenkins build is back to normal : kafka-trunk-jdk7 #1205

See

[jira] [Commented] (KAFKA-3568) KafkaProducer fails with timeout on message send()

[ https://issues.apache.org/jira/browse/KAFKA-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248554#comment-15248554 ] Greg Zoller commented on KAFKA-3568: Ok... I don't believe my test results, but I have repeated them.

[GitHub] kafka pull request: MINOR: Fix typos in code comments

GitHub user imandhan opened a pull request: https://github.com/apache/kafka/pull/1240 MINOR: Fix typos in code comments This patch fixes all occurances of two consecutive 'the's in the code comments. Author: Ishita Mandhan (iman...@us.ibm.com) You can merge this pull

[jira] [Work started] (KAFKA-3101) Optimize Aggregation Outputs

[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3101 started by Bill Bejeck. -- > Optimize Aggregation Outputs > > >

Re: [VOTE] KIP-33 - Add a time based log index

Thanks Joel and Ismael. I just updated the KIP based on your feedback. KIP-33 has passed with +4 (binding) and +2 (non-binding) Thanks everyone for the reading, feedback and voting! Jiangjie (Becket) Qin On Tue, Apr 19, 2016 at 5:25 PM, Ismael Juma wrote: > Thanks Becket.

Build failed in Jenkins: kafka-trunk-jdk8 #537

See Changes: [wangguoz] MINOR: Fix typos in code comments -- [...truncated 1628 lines...] kafka.log.OffsetIndexTest > appendTooMany PASSED kafka.log.OffsetIndexTest > randomLookupTest PASSED

kafka support at producer side needed

Hi I have a one scenario ass below, I want to know whether its supported currently. If not is there any work around by using existing kafka features. I have kafka producer, currently he doesn’t have connection to the broker. I want to send the messages to kafka broker when the connection is

[jira] [Commented] (KAFKA-3532) add principal.builder.class that can extract user from a field

[ https://issues.apache.org/jira/browse/KAFKA-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248831#comment-15248831 ] Ismael Juma commented on KAFKA-3532: I think we should allow one to to pass a list of fields instead

[jira] [Comment Edited] (KAFKA-3532) add principal.builder.class that can extract user from a field

[ https://issues.apache.org/jira/browse/KAFKA-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248831#comment-15248831 ] Ismael Juma edited comment on KAFKA-3532 at 4/19/16 10:45 PM: -- I think we

[jira] [Updated] (KAFKA-3158) ConsumerGroupCommand should tell whether group is actually dead

[ https://issues.apache.org/jira/browse/KAFKA-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3158: --- Assignee: (was: Neha Narkhede) > ConsumerGroupCommand should tell whether group is actually dead

[jira] [Commented] (KAFKA-3158) ConsumerGroupCommand should tell whether group is actually dead

[ https://issues.apache.org/jira/browse/KAFKA-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248867#comment-15248867 ] Ishita Mandhan commented on KAFKA-3158: --- Yes, I'd like to give it a shot. Assigning it to myself.

[jira] [Assigned] (KAFKA-3158) ConsumerGroupCommand should tell whether group is actually dead

[ https://issues.apache.org/jira/browse/KAFKA-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishita Mandhan reassigned KAFKA-3158: - Assignee: Ishita Mandhan > ConsumerGroupCommand should tell whether group is actually

Re: [VOTE] KIP-33 - Add a time based log index

Thanks Becket. I think it would be nice to update the KIP with regards to point 3 and 4. In any case, +1 (non-binding) Ismael On Tue, Apr 19, 2016 at 2:03 AM, Becket Qin wrote: > Thanks for the comments Ismael. Please see the replies inline. > > On Mon, Apr 18, 2016 at

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

Just to close the loop on this. Discussed with Magnus offline on how KIP-43 and KIP-35 can play together. We agreed upon the following proposal. On a SASL port, client sends: ApiVersionRequest (optional), SaslHandshakeRequest, SASL tokens (size delimited as being done now), regular api

[jira] [Commented] (KAFKA-3552) New Consumer: java.lang.OutOfMemoryError: Direct buffer memory

[ https://issues.apache.org/jira/browse/KAFKA-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15249065#comment-15249065 ] Kanak Biscuitwala commented on KAFKA-3552: -- In fact, I'm reasonably certain there is some kind of

[jira] [Commented] (KAFKA-3158) ConsumerGroupCommand should tell whether group is actually dead

[ https://issues.apache.org/jira/browse/KAFKA-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248856#comment-15248856 ] Ismael Juma commented on KAFKA-3158: [~imandhan], no-one is working on this JIRA. Would you like to

Re: KTable.count(...)

Hi Damain, The semantics of Count() is still depending on the KTable, where the records are of changelogs based on key. So for your example the KTable is really conceptually from an updating "source table" with only one record but with updating values from 1 to 5. And KTable.count(...) is

[jira] [Resolved] (KAFKA-3569) commitAsync() sometimes fails with errors

[ https://issues.apache.org/jira/browse/KAFKA-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Zoller resolved KAFKA-3569. Resolution: Fixed Fix Version/s: 0.10.1.0 I was able to re-test in 0.10.1.0-SNAPSHOT. The

[jira] [Commented] (KAFKA-3158) ConsumerGroupCommand should tell whether group is actually dead

[ https://issues.apache.org/jira/browse/KAFKA-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248852#comment-15248852 ] Ishita Mandhan commented on KAFKA-3158: --- Neha, are you working on this bug? > ConsumerGroupCommand

[jira] [Comment Edited] (KAFKA-3158) ConsumerGroupCommand should tell whether group is actually dead

[ https://issues.apache.org/jira/browse/KAFKA-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248852#comment-15248852 ] Ishita Mandhan edited comment on KAFKA-3158 at 4/19/16 10:57 PM: - Neha,

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248954#comment-15248954 ] Jun Rao commented on KAFKA-3042: Currently, the main differences between LeaderAndIsrRequest and

[GitHub] kafka pull request: MINOR: Fix typos in code comments

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1240 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-3588) Document how Producer Settings affect In Sync Replicas

Alan Braithwaite created KAFKA-3588: --- Summary: Document how Producer Settings affect In Sync Replicas Key: KAFKA-3588 URL: https://issues.apache.org/jira/browse/KAFKA-3588 Project: Kafka

[jira] [Updated] (KAFKA-3588) Document how Producer Settings affect In Sync Replicas

[ https://issues.apache.org/jira/browse/KAFKA-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Braithwaite updated KAFKA-3588: Description: One of the big changes from 0.8.x to 0.9.x was of course the way that Kafka

[jira] [Resolved] (KAFKA-2912) Add error code 4 (InvalidFetchSize) to Errors.java

[ https://issues.apache.org/jira/browse/KAFKA-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy resolved KAFKA-2912. Resolution: Fixed Assignee: Grant Henke (was: jin xing) Fix Version/s:

[jira] [Commented] (KAFKA-3480) Autogenerate metrics documentation

[ https://issues.apache.org/jira/browse/KAFKA-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247295#comment-15247295 ] James Cheng commented on KAFKA-3480: Pinging [~hachikuji] and [~gwenshap]. What do you think of the

[jira] [Resolved] (KAFKA-2524) Examples in class javadoc use unimplemented subscribe method

[ https://issues.apache.org/jira/browse/KAFKA-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy resolved KAFKA-2524. Resolution: Fixed Fix Version/s: 0.10.0.0 > Examples in class javadoc use

[jira] [Commented] (KAFKA-3577) Partial cluster breakdown

[ https://issues.apache.org/jira/browse/KAFKA-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247287#comment-15247287 ] James Cheng commented on KAFKA-3577: The symptoms seem similar to

Re: [VOTE] KIP-33 - Add a time based log index

Hi Becket, +1 Just a few more minor comments that I don't think should result in any material change to the KIP: - Under "time index format": *A time index entry (timestamp, offset) means that any message whose timestamp is greater than timestamp come after offset. * - I think it

[jira] [Commented] (KAFKA-3573) JSON (de)serialization for kafka-client

[ https://issues.apache.org/jira/browse/KAFKA-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247641#comment-15247641 ] Igor Stepanov commented on KAFKA-3573: -- So the ticket is irrelevant for the project, right? If I got

[jira] [Created] (KAFKA-3584) NullPointer Exception found when Delete Topic and Log delete concurrent

Pengwei created KAFKA-3584: -- Summary: NullPointer Exception found when Delete Topic and Log delete concurrent Key: KAFKA-3584 URL: https://issues.apache.org/jira/browse/KAFKA-3584 Project: Kafka

[jira] [Commented] (KAFKA-3564) Count metric always increments by 1.0

[ https://issues.apache.org/jira/browse/KAFKA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247662#comment-15247662 ] Michael Coon commented on KAFKA-3564: - Hmm, Sum wasn't available when I did a clone on the codebase.

[jira] [Created] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

Pengwei created KAFKA-3585: -- Summary: Shutdown slow when there is only one broker which is controller Key: KAFKA-3585 URL: https://issues.apache.org/jira/browse/KAFKA-3585 Project: Kafka Issue