[jira] [Commented] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969669#comment-14969669 ] ASF GitHub Bot commented on KAFKA-2686: --- GitHub user guozhangwang opened a pull request:

Re: [VOTE] KIP-38: ZooKeeper authentication

+1 (non-binding) On Thu, Oct 22, 2015 at 3:46 PM, Neha Narkhede wrote: > +1 (binding) > > On Wed, Oct 21, 2015 at 7:24 PM, Todd Palino wrote: > > > +1 (non-binding) > > > > On Wed, Oct 21, 2015 at 6:53 PM, Jiangjie Qin > > >

Re: [DISCUSS] KIP-38: ZooKeeper Authentication

This might have been explained before. I had a question : In the KIP it says : "One ZooKeeper setting of interest on the server side is zookeeper.allowSaslFailedClients. If this is false, then clients trying to authenticate with an incorrect configuration will have their connections dropped.

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

Hey Kartik, Yes, I agree exactly with your characterization. The question is "what is the meaning of retention?" It could mean either: 1. "Retain data that is no more than 7 days old" 2. "Retain data for 7 days from when you get it" I don't know if either is actually a clear winner. Each is

Build failed in Jenkins: kafka-trunk-jdk7 #711

See Changes: [wangguoz] KAFKA-2678; partition level lag metrics can be negative -- [...truncated 2826 lines...] kafka.consumer.TopicFilterTest > testWhitelists PASSED

[jira] [Updated] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2686: - Reviewer: Jason Gustafson > unsubscribe() call leaves KafkaConsumer in invalid state for manual

[GitHub] kafka pull request: KAFKA-2686: Reset needsPartitionAssignment in ...

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/352 KAFKA-2686: Reset needsPartitionAssignment in SubscriptionState.assign() You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka

Question about compressed messages...

I’m working on a Twisted Python Kafka client and I was wondering what the ‘key’ on a gzip’d block of messages “means”. That is, if the client has a batch of messages to send, with a mix of keys, would it be a bug to batch them together and gzip into a single message? Or is the key on the

Re: [DISCUSS] KIP-38: ZooKeeper Authentication

Hi Mayuresh, See comments below, please: > On 22 Oct 2015, at 18:17, Mayuresh Gharat wrote: > > This might have been explained before. I had a question : > In the KIP it says : > "One ZooKeeper setting of interest on the server side is >

[jira] [Assigned] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-2686: Assignee: Guozhang Wang (was: Neha Narkhede) > unsubscribe() call leaves KafkaConsumer in

[GitHub] kafka pull request: KAFKA-2652: integrate new group protocol into ...

GitHub user ymatsuda opened a pull request: https://github.com/apache/kafka/pull/353 KAFKA-2652: integrate new group protocol into partition grouping @guozhangwang * added ```PartitionGrouper``` (abstract class) * This class is responsible for grouping partitions to

[jira] [Commented] (KAFKA-2652) Incorporate the new consumer protocol with partition-group interface

[ https://issues.apache.org/jira/browse/KAFKA-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969687#comment-14969687 ] ASF GitHub Bot commented on KAFKA-2652: --- GitHub user ymatsuda opened a pull request:

Build failed in Jenkins: kafka-trunk-jdk8 #50

See Changes: [wangguoz] KAFKA-2678; partition level lag metrics can be negative [wangguoz] MINOR: Restore `SslConsumerTest` which was accidentally deleted in -- [...truncated 5329 lines...]

Re: [VOTE] KIP-38: ZooKeeper authentication

+1 (binding) On Wed, Oct 21, 2015 at 7:24 PM, Todd Palino wrote: > +1 (non-binding) > > On Wed, Oct 21, 2015 at 6:53 PM, Jiangjie Qin > wrote: > > > +1 (non-binding) > > > > On Wed, Oct 21, 2015 at 3:40 PM, Joel Koshy wrote: >

Jenkins build is back to normal : kafka-trunk-jdk7 #712

See

[jira] [Assigned] (KAFKA-2685) "alter topic" on non-existent topic exits without error

[ https://issues.apache.org/jira/browse/KAFKA-2685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro reassigned KAFKA-2685: - Assignee: Edward Ribeiro > "alter topic" on non-existent topic exits without error >

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

Hey Joel, I think we're mostly in agreement. I don't know about you but I'm not head-over-heals in love with any of these proposals, including mine--time is just pretty complicated and icky. With respect to using log append time here was basically how I felt: 1. I think in a vacuum basing offset

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

Hey Becket, Let me see if I can address your concerns: 1. Let's say we have two source clusters that are mirrored to the same > target cluster. For some reason one of the mirror maker from a cluster dies > and after fix the issue we want to resume mirroring. In this case it is > possible that

[jira] [Work started] (KAFKA-2675) SASL/Kerberos follow-up

[ https://issues.apache.org/jira/browse/KAFKA-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2675 started by Ismael Juma. -- > SASL/Kerberos follow-up > --- > > Key:

Re: [VOTE] KIP-38: ZooKeeper authentication

+1 Thanks, Jun On Wed, Oct 21, 2015 at 8:17 AM, Flavio Junqueira wrote: > Thanks everyone for the feedback so far. At this point, I'd like to start > a vote for KIP-38. > > Summary: Add support for ZooKeeper authentication > KIP page: >

Re: Java high level consumer providing duplicate messages when auto commit is off

We did some more testing with logging turned on (I figured out why it wasn't working). We tried increasing the JVM memory capacity on our test server (it's lower than in production) and increasing the zookeeper timeouts. Neither changed the results. With trace logging enabled, we saw that we

Re: [VOTE] KIP-38: ZooKeeper authentication

+1 (non-binding) On Thu, Oct 22, 2015 at 12:10 PM, Edward Ribeiro wrote: > +1 (non-binding) > > On Thu, Oct 22, 2015 at 3:46 PM, Neha Narkhede wrote: > > > +1 (binding) > > > > On Wed, Oct 21, 2015 at 7:24 PM, Todd Palino wrote:

[jira] [Commented] (KAFKA-2675) SASL/Kerberos follow-up

[ https://issues.apache.org/jira/browse/KAFKA-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970115#comment-14970115 ] Jun Rao commented on KAFKA-2675: For #4, making the login thread a daemon thread is also fine. >

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

For 2 and 3, I have updated the KIP. Please take a look. One thing I have changed is removing the proposal to add rack to TopicMetadataResponse. The reason is that unlike UpdateMetadataRequest, TopicMetadataResponse does not understand version. I don't see a way to include rack without breaking

[GitHub] kafka pull request: Restore `SslConsumerTest` which was accidental...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/350 Restore `SslConsumerTest` which was accidentally deleted in client-side assignment commit Probably happened while resolving conflicts, commit: 86eb74d9236c586af5889fe79f4b9e066c9c2af3 You can merge

[jira] [Commented] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970459#comment-14970459 ] Jiangjie Qin commented on KAFKA-2017: - [~hachikuji] That makes sense. Persisting consumer metadata in

Build failed in Jenkins: kafka-trunk-jdk8 #51

See Changes: [wangguoz] KAFKA-2686: Reset needsPartitionAssignment in SubscriptionState.assign() -- [...truncated 5267 lines...] kafka.coordinator.MemberMetadataTest >

[GitHub] kafka pull request: MINOR: follow-up to KAFKA-2464 for renaming/cl...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/354 MINOR: follow-up to KAFKA-2464 for renaming/cleanup You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-2464 Alternatively

[jira] [Commented] (KAFKA-2464) Client-side assignment and group generalization

[ https://issues.apache.org/jira/browse/KAFKA-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970358#comment-14970358 ] ASF GitHub Bot commented on KAFKA-2464: --- GitHub user hachikuji opened a pull request:

[jira] [Resolved] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-2686. -- Resolution: Fixed Fix Version/s: 0.9.0.0 Issue resolved by pull request 352

[GitHub] kafka pull request: KAFKA-2686: Reset needsPartitionAssignment in ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/352 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970392#comment-14970392 ] ASF GitHub Bot commented on KAFKA-2686: --- Github user asfgit closed the pull request at:

Re: Java high level consumer providing duplicate messages when auto commit is off

Hi Cliff, If auto.offset.commit is set to true, the offset will be committed in following cases in addition to periodical offset commit: 1. During consumer rebalance before release the partition ownership. If consumer A owns partition P before rebalance, it will commit offset for partition P

[GitHub] kafka pull request: KAFKA-2338: add force option to topic / config...

GitHub user benstopford opened a pull request: https://github.com/apache/kafka/pull/351 KAFKA-2338: add force option to topic / config command so they can be called programatically Tiny change to add a force option to the topic and config commands so they can be called

[jira] [Commented] (KAFKA-2675) SASL/Kerberos follow-up

[ https://issues.apache.org/jira/browse/KAFKA-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969096#comment-14969096 ] Ismael Juma commented on KAFKA-2675: Some input would be appreciated on the following: 1. Decide on

[jira] [Commented] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings

[ https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969092#comment-14969092 ] ASF GitHub Bot commented on KAFKA-2338: --- GitHub user benstopford opened a pull request:

[GitHub] kafka pull request: KAFKA-2671: Enable starting Kafka server with ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/330 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2671) Enable starting Kafka server with a Properties object

[ https://issues.apache.org/jira/browse/KAFKA-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970474#comment-14970474 ] ASF GitHub Bot commented on KAFKA-2671: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-2671) Enable starting Kafka server with a Properties object

[ https://issues.apache.org/jira/browse/KAFKA-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-2671. - Resolution: Fixed Fix Version/s: 0.9.0.0 Issue resolved by pull request 330

[jira] [Work started] (KAFKA-2684) Add force option to TopicCommand & ConfigCommand to suppress console prompts

[ https://issues.apache.org/jira/browse/KAFKA-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2684 started by Ben Stopford. --- > Add force option to TopicCommand & ConfigCommand to suppress console prompts >

[jira] [Updated] (KAFKA-2684) Add force option to TopicCommand & ConfigCommand to suppress console prompts

[ https://issues.apache.org/jira/browse/KAFKA-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford updated KAFKA-2684: Description: Add force option to TopicCommand & Config Command to suppress console prompts This is

Re: [DISCUSS] KIP-38: ZooKeeper Authentication

Ok, thanks for spotting it. -Flavio > On 22 Oct 2015, at 05:54, Jun Rao wrote: > > It seems that in the secure -> unsecure plan, step 3 needs to be done > before step 2. > > Thanks, > > Jun > > On Wed, Oct 21, 2015 at 3:59 PM, Flavio Junqueira wrote: >

[jira] [Assigned] (KAFKA-2684) Add force option to TopicCommand & ConfigCommand to suppress console prompts

[ https://issues.apache.org/jira/browse/KAFKA-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford reassigned KAFKA-2684: --- Assignee: Ben Stopford > Add force option to TopicCommand & ConfigCommand to suppress

[jira] [Created] (KAFKA-2684) Add force option to TopicCommand & ConfigCommand to suppress console prompts

Ben Stopford created KAFKA-2684: --- Summary: Add force option to TopicCommand & ConfigCommand to suppress console prompts Key: KAFKA-2684 URL: https://issues.apache.org/jira/browse/KAFKA-2684 Project:

[jira] [Updated] (KAFKA-2684) Add force option to TopicCommand & ConfigCommand to suppress console prompts

[ https://issues.apache.org/jira/browse/KAFKA-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford updated KAFKA-2684: Status: Patch Available (was: In Progress) > Add force option to TopicCommand & ConfigCommand to

[jira] [Created] (KAFKA-2685) "alter topic" on non-existent topic exits without error

Gwen Shapira created KAFKA-2685: --- Summary: "alter topic" on non-existent topic exits without error Key: KAFKA-2685 URL: https://issues.apache.org/jira/browse/KAFKA-2685 Project: Kafka Issue

[jira] [Updated] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] The Data Lorax updated KAFKA-2686: -- Description: The bellow code snippet demonstrated the problem. Basically, the unsubscribe()

[GitHub] kafka pull request: MINOR: Restore `SslConsumerTest` which was acc...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: KAFKA-2678; partition level lag metrics can be...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/346 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-2686) unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment

The Data Lorax created KAFKA-2686: - Summary: unsubscribe() call leaves KafkaConsumer in invalid state for manual topic-partition assignment Key: KAFKA-2686 URL: https://issues.apache.org/jira/browse/KAFKA-2686

[jira] [Resolved] (KAFKA-2678) partition level lag metrics can be negative

[ https://issues.apache.org/jira/browse/KAFKA-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-2678. -- Resolution: Fixed Fix Version/s: 0.9.0.0 Issue resolved by pull request 346

[jira] [Commented] (KAFKA-2678) partition level lag metrics can be negative

[ https://issues.apache.org/jira/browse/KAFKA-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969386#comment-14969386 ] ASF GitHub Bot commented on KAFKA-2678: --- Github user asfgit closed the pull request at: