Re: Java high level consumer providing duplicate messages when auto commit is off

Thanks, Jiangjie. Understanding more about the auto-commit behavior and why it's resilient to these is a big help. We're going to do some deeper investigation and testing. I'll report back when I have more information. Thanks, Cliff On Thu, Oct 22, 2015 at 11:48 PM, Jiangjie Qin

[jira] [Commented] (KAFKA-2675) SASL/Kerberos follow-up

[ https://issues.apache.org/jira/browse/KAFKA-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971271#comment-14971271 ] Ismael Juma commented on KAFKA-2675: Thanks [~harsha_ch]. You mentioned that many projects use

Re: (User) Document/guide on new Kafka features on security in upcoming 0.9.0?

Jaikiran, Here is the SSL setup doc https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka . I am going to write one for SASL ( kerberos ) soon and Parth is working on pushing one for Authorizer as well. Thanks, Harsha On Fri, Oct 23, 2015, at 04:42

[jira] [Commented] (KAFKA-2644) Run relevant ducktape tests with SASL_PLAINTEXT and SASL_SSL

[ https://issues.apache.org/jira/browse/KAFKA-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972339#comment-14972339 ] Sriharsha Chintalapani commented on KAFKA-2644: --- [~junrao] If possible we should use

[jira] [Created] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

Jason Gustafson created KAFKA-2687: -- Summary: Allow GroupMetadataRequest to return member metadata when received by group coordinator Key: KAFKA-2687 URL: https://issues.apache.org/jira/browse/KAFKA-2687

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971728#comment-14971728 ] Joel Koshy commented on KAFKA-2235: --- While I think the change is well-motivated I'm not sure this is the

[jira] [Commented] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971732#comment-14971732 ] Gwen Shapira commented on KAFKA-2687: - If we allow regexp as the string in Groups, then it will handle

[jira] [Commented] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971770#comment-14971770 ] Jason Gustafson commented on KAFKA-2687: [~gwenshap] That's an interesting idea. I wonder if there

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971806#comment-14971806 ] Todd Palino commented on KAFKA-2235: I don't think we can. I have already increased it from 512MB to

[jira] [Commented] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971660#comment-14971660 ] Jason Gustafson commented on KAFKA-2687: [~junrao] [~guozhang] I can write up a KIP for this if

[jira] [Commented] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971677#comment-14971677 ] Gwen Shapira commented on KAFKA-2687: - This seems like a per-group request. Will there be a way to

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971740#comment-14971740 ] Todd Palino commented on KAFKA-2235: I'm sure [~jjkoshy] will follow along with more detail on this,

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971804#comment-14971804 ] Jun Rao commented on KAFKA-2235: [~toddpalino], thanks for reporting this issue. I agree that it's better

[jira] [Commented] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971799#comment-14971799 ] Gwen Shapira commented on KAFKA-2687: - Is it any different from consumer subscribe() taking regexp? >

Jenkins build is back to normal : kafka-trunk-jdk7 #716

See

Build failed in Jenkins: kafka-trunk-jdk8 #54

See Changes: [cshapi] KAFKA-2371: Add distributed support for Copycat. [cshapi] KAFKA-2626: Handle null keys and value validation properly in -- [...truncated 5007 lines...]

Build failed in Jenkins: kafka-trunk-jdk7 #717

See Changes: [cshapi] KAFKA-2626: Handle null keys and value validation properly in -- [...truncated 1797 lines...] kafka.log.CleanerTest > testCleaningWithDeletes PASSED kafka.log.CleanerTest

[jira] [Commented] (KAFKA-2675) SASL/Kerberos follow-up

[ https://issues.apache.org/jira/browse/KAFKA-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971107#comment-14971107 ] Sriharsha Chintalapani commented on KAFKA-2675: --- 1. Decide on `serviceName` configuration:

Build failed in Jenkins: kafka_system_tests #116

See -- Started by timer Building in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson3879296377443603922.sh Running command: git pull &&

[jira] [Commented] (KAFKA-2235) LogCleaner offset map overflow

[ https://issues.apache.org/jira/browse/KAFKA-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971848#comment-14971848 ] Ivan Simoneko commented on KAFKA-2235: -- The core problem is that offset map is limited by number of

[jira] [Resolved] (KAFKA-2641) Upgrade path for ZK authentication

[ https://issues.apache.org/jira/browse/KAFKA-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2641. Resolution: Fixed Fix Version/s: 0.9.0.0 Issue resolved by pull request 313

[jira] [Commented] (KAFKA-2641) Upgrade path for ZK authentication

[ https://issues.apache.org/jira/browse/KAFKA-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971981#comment-14971981 ] ASF GitHub Bot commented on KAFKA-2641: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-2626: Handle null keys and value validat...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/345 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2626) Null offsets in copycat causes exception in OffsetStorageWriter

[ https://issues.apache.org/jira/browse/KAFKA-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972153#comment-14972153 ] ASF GitHub Bot commented on KAFKA-2626: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-2688) Avoid forcing reload of `Configuration`

[ https://issues.apache.org/jira/browse/KAFKA-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2688: --- Summary: Avoid forcing reload of `Configuration` (was: Improve) > Avoid forcing reload of

[jira] [Commented] (KAFKA-2687) Allow GroupMetadataRequest to return member metadata when received by group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972063#comment-14972063 ] Jun Rao commented on KAFKA-2687: [~gwenshap], the regex in the consumer is only used in the java client.

Build failed in Jenkins: kafka-trunk-jdk7 #715

See Changes: [junrao] KAFKA-2641; Upgrade path for ZK authentication -- [...truncated 324 lines...] :kafka-trunk-jdk7:clients:createVersionFile :kafka-trunk-jdk7:clients:jar UP-TO-DATE

Build failed in Jenkins: kafka-trunk-jdk8 #53

See Changes: [junrao] KAFKA-2641; Upgrade path for ZK authentication -- [...truncated 4611 lines...] org.apache.kafka.copycat.file.FileStreamSinkConnectorTest > testSinkTasks PASSED

[GitHub] kafka pull request: KAFKA-2371: Add distributed support for Copyca...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/321 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2371) Add distributed coordinator implementation for Copycat

[ https://issues.apache.org/jira/browse/KAFKA-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972113#comment-14972113 ] ASF GitHub Bot commented on KAFKA-2371: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-2371) Add distributed coordinator implementation for Copycat

[ https://issues.apache.org/jira/browse/KAFKA-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-2371. - Resolution: Fixed Issue resolved by pull request 321 [https://github.com/apache/kafka/pull/321]

(User) Document/guide on new Kafka features on security in upcoming 0.9.0?

Given that 0.9.0 is being planned to released sometime soon in November, our team has started looking into what's coming in the newer version and what we need to do to start using the features. We are very much and mainly interested in the security features of it for the upcoming releases. We

Re: [DISCUSS] KIP-37 - Add namespaces in Kafka

What about topic-level metrics? Are we going to report metrics at all level now? Or maybe just at partition-level and use the monitoring app to aggregate them in different levels (i.e. remove topic metrics completely)? On Wed, Oct 21, 2015 at 3:47 PM, Ashish Singh wrote:

[GitHub] kafka pull request: KAFKA-2641: Upgrade path for ZK authentication

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/313 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2644) Run relevant ducktape tests with SASL_PLAINTEXT and SASL_SSL

[ https://issues.apache.org/jira/browse/KAFKA-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972026#comment-14972026 ] Jun Rao commented on KAFKA-2644: [~sriharsha], is MiniKDC realistic enough for testing Kerberos? Are there

[jira] [Created] (KAFKA-2688) Improve

Ismael Juma created KAFKA-2688: -- Summary: Improve Key: KAFKA-2688 URL: https://issues.apache.org/jira/browse/KAFKA-2688 Project: Kafka Issue Type: Bug Components: security

Build failed in Jenkins: kafka-trunk-jdk8 #52

See Changes: [cshapi] KAFKA-2671: Enable starting Kafka server with a Properties object -- [...truncated 3116 lines...] kafka.integration.PrimitiveApiTest > testFetchRequestCanProperlySerialize