[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223699#comment-15223699 ] Guozhang Wang commented on KAFKA-3491: -- So do you think that we probably should reconsider whether

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223693#comment-15223693 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- Automatic offset commit. Unless we can

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223685#comment-15223685 ] Guozhang Wang commented on KAFKA-3491: -- "if it's actually a good feature to provide": what feature

Re: KIP-4 Wiki Update

Hi Grant, Thanks for the write-ups. I had some comments besides the debate of metadata request principles: 1. One minor thing: "Note there is a special use case - automatic topic creation for TopicMetadataRequest, to trigger it user should set client_id=consumer and define only topic name:" I

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223682#comment-15223682 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- [~guozhang] Comparing to other clients

Re: [VOTE] KIP-4 Metadata Schema

Grant, 2. Hmm, I am not sure about the listener approach. It ignores configs like enable.topic.deletion and also opens the door for potential ordering issues since now there are two separate paths for propagating the metadata to the brokers. Could we just leave out markedForDeletion for now? In

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223656#comment-15223656 ] Guozhang Wang commented on KAFKA-3491: -- If we do not want to enforce users to keep track of a

Build failed in Jenkins: kafka-trunk-jdk7 #1172

See Changes: [me] MINOR: Clean up of SourceTaskOffsetCommiter -- [...truncated 2819 lines...] kafka.integration.SaslPlaintextTopicMetadataTest >

Kafka Mailing List Filters

The Kafka development mailing list can be a lot to keep up with. To cope, I have created a few filters to separate the most important communication from the noise. I have found using just a few filters help a lot. Below are my most reliable Gmail filters (they should be able to map to other email

Build failed in Jenkins: kafka-trunk-jdk8 #503

See Changes: [me] KAFKA-3483: Restructure ducktape tests to simplify running subsets of -- [...truncated 3139 lines...] kafka.consumer.ZookeeperConsumerConnectorTest > testConsumerDecoder

Re: [RELEASE UPDATE] Postponing the next release candidates and canceling current vote

Just wanted to throw it out there that still double committing when the committer remembers to do so is useful -- daily updates on unit tests (as flaky as they can be) and system tests are still useful to have. Better to catch any branch-specific issues as early as possible. -Ewen On Fri, Apr 1,

[GitHub] kafka pull request: KAFKA-3483: Restructure ducktape tests to simp...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1162 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3483) Restructure ducktape tests to simplify running subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223622#comment-15223622 ] ASF GitHub Bot commented on KAFKA-3483: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-3483) Restructure ducktape tests to simplify running subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3483: - Resolution: Fixed Fix Version/s: 0.10.0.0 0.10.1.0

Build failed in Jenkins: kafka-0.10.0-jdk7 #30

See Changes: [me] MINOR: Fix small typo in design section -- [...truncated 5236 lines...] org.apache.kafka.streams.processor.internals.StreamThreadTest > testMaybeClean PASSED

Build failed in Jenkins: kafka-trunk-jdk7 #1171

See Changes: [me] MINOR: Fix small typo in design section -- [...truncated 1589 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > testCleanShutdownFile PASSED

[jira] [Commented] (KAFKA-3496) Add reconnect attemps policies for client

[ https://issues.apache.org/jira/browse/KAFKA-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223607#comment-15223607 ] Ewen Cheslack-Postava commented on KAFKA-3496: -- [~fhussonnois] Despite a small # of options,

[GitHub] kafka pull request: MINOR: server stop scripts don't work correctl...

Github user sasakitoa closed the pull request at: https://github.com/apache/kafka/pull/1145 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-52 - Add Connector Control APIs

Ismael, Great point. Pause and resume should be idempotent and actually represent updating a resource that gets written to Kafka (although I must admit I don't know if the use of 202/Accepted should affect this at all), the restart endpoints seem a bit different as they are one-off immediate

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223604#comment-15223604 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- I don't know, this strikes me as a solution

[GitHub] kafka pull request: [Minor]: Clean up of SourceTaskOffsetCommiter

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [VOTE] KIP-4 Metadata Schema

Responding to a few of the other comments: it seems that you propagated > the topic deletion marker by having the replicaManager read from ZK > directly. It seems that it would be simpler/consistent if the controller > propagates that information directly through UpdateMetaRequest. I was told

Re: [VOTE] KIP-4 Metadata Schema

Hi Jun and Ismael, Initially I had 2 booleans used to indicate if a topic was internal and if a topic was marked for deletion. To save space on large deployments, Ismael suggested I break out the internal topics and deleted topics into their own lists. The idea was that instead of 2 bytes added

Re: [VOTE] KIP-4 Metadata Schema

Hi Jun, A couple of comments inline. On Sun, Apr 3, 2016 at 5:55 PM, Jun Rao wrote: > 1. It seems a bit weird to return just a list of internal topics w/o the > corresponding metadata. It also seems a bit weird to return the internal > topics even if the client doesn't ask

Build failed in Jenkins: kafka-trunk-jdk7 #1170

See Changes: [junrao] KAFKA-3495; NetworkClient.blockingSendAndReceive` should rely on -- [...truncated 1583 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest >

[jira] [Updated] (KAFKA-3495) `NetworkClient.blockingSendAndReceive` should rely on requestTimeout

[ https://issues.apache.org/jira/browse/KAFKA-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3495: --- Fix Version/s: 0.10.0.0 > `NetworkClient.blockingSendAndReceive` should rely on requestTimeout >

Re: KIP-4 Wiki Update

Hi Jun, Yes, that makes it a bit challenging to do this in a manner that is fully compatible. Having said that, most (all?) languages have Perl-derived regexes (including Java), so it seems like it should be possible to define a subset for the protocol. But perhaps that's a bigger change that

Re: KIP-4 Wiki Update

Ismael, The reason that we only apply the regex for topics on the client side is that regex is java specific. So, for non-java clients, it would be a bit weird for them to include java specific stuff in the wire protocol. Thanks, Jun On Sun, Apr 3, 2016 at 12:36 AM, Ismael Juma

[jira] [Commented] (KAFKA-3477) Add customizable StreamPartition into #to functions of Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223553#comment-15223553 ] ASF GitHub Bot commented on KAFKA-3477: --- GitHub user mjsax opened a pull request:

[GitHub] kafka pull request: [KAFKA-3477] [Kafka Streams] extended KStream/...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/1180 [KAFKA-3477] [Kafka Streams] extended KStream/KTable API to specify c… …ustom partitioner for sinks You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (KAFKA-3495) `NetworkClient.blockingSendAndReceive` should rely on requestTimeout

[ https://issues.apache.org/jira/browse/KAFKA-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223550#comment-15223550 ] ASF GitHub Bot commented on KAFKA-3495: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-3495; `NetworkClient.blockingSendAndRece...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1177 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka-trunk-jdk7 #1169

See Changes: [wangguoz] KAFKA-3419: clarify difference between topic subscription and partition -- [...truncated 1589 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest >

[jira] [Updated] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

[ https://issues.apache.org/jira/browse/KAFKA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3497: - Issue Type: Sub-task (was: Improvement) Parent: KAFKA-2590 > Streams ProcessorContext

[jira] [Commented] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

[ https://issues.apache.org/jira/browse/KAFKA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223533#comment-15223533 ] Guozhang Wang commented on KAFKA-3497: -- Thanks [~kawamuray]. I think we can consider even replacing

[jira] [Assigned] (KAFKA-3477) Add customizable StreamPartition into #to functions of Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-3477: -- Assignee: Matthias J. Sax > Add customizable StreamPartition into #to functions of

Build failed in Jenkins: kafka-trunk-jdk8 #500

See Changes: [wangguoz] KAFKA-3419: clarify difference between topic subscription and partition -- [...truncated 1588 lines...] kafka.consumer.ZookeeperConsumerConnectorTest >

[jira] [Commented] (KAFKA-3338) Add print and writeAsText functions to the Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223510#comment-15223510 ] Guozhang Wang commented on KAFKA-3338: -- Sounds good! > Add print and writeAsText functions to the

[jira] [Updated] (KAFKA-3419) New consumer javadoc unclear on topic subscription vs partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3419: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-3419) New consumer javadoc unclear on topic subscription vs partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223509#comment-15223509 ] ASF GitHub Bot commented on KAFKA-3419: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-3419: clarify difference between topic s...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1158 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-3498) Transient failure in kafka.api.SslConsumerTest.testSimpleConsumption

Guozhang Wang created KAFKA-3498: Summary: Transient failure in kafka.api.SslConsumerTest.testSimpleConsumption Key: KAFKA-3498 URL: https://issues.apache.org/jira/browse/KAFKA-3498 Project: Kafka

[jira] [Commented] (KAFKA-725) Broker Exception: Attempt to read with a maximum offset less than start offset

[ https://issues.apache.org/jira/browse/KAFKA-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223457#comment-15223457 ] ASF GitHub Bot commented on KAFKA-725: -- GitHub user srdo reopened a pull request:

[jira] [Commented] (KAFKA-725) Broker Exception: Attempt to read with a maximum offset less than start offset

[ https://issues.apache.org/jira/browse/KAFKA-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223456#comment-15223456 ] ASF GitHub Bot commented on KAFKA-725: -- Github user srdo closed the pull request at:

[GitHub] kafka pull request: KAFKA-725: Change behavior of Log/LogSegment w...

Github user srdo closed the pull request at: https://github.com/apache/kafka/pull/1178 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [VOTE] KIP-4 Metadata Schema

Grant, Thanks for the writeup. A few comments. 1. It seems a bit weird to return just a list of internal topics w/o the corresponding metadata. It also seems a bit weird to return the internal topics even if the client doesn't ask for it. Would it be better to just add a flag in topic_metadata

[jira] [Created] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

Yuto Kawamura created KAFKA-3497: Summary: Streams ProcessorContext should support forward() based on child name Key: KAFKA-3497 URL: https://issues.apache.org/jira/browse/KAFKA-3497 Project: Kafka

[jira] [Updated] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

[ https://issues.apache.org/jira/browse/KAFKA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura updated KAFKA-3497: - Description: Currently {{ProcessorContext}} only supports {{forward(K, V)}} which forwards KV to

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

Ashish, For the two benefits that you listed, fail fast can be done just in the implementation w/o getSupportedPrincipalTypes(). For avoiding the same check in all implementations, I think Grant is thinking of adding some ACL RPC request/response btw the client and broker directly, instead of

[jira] [Commented] (KAFKA-3480) Autogenerate metrics documentation

[ https://issues.apache.org/jira/browse/KAFKA-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223239#comment-15223239 ] Otis Gospodnetic commented on KAFKA-3480: - This would be great. I just hope the output can be

Jenkins build is back to normal : kafka-0.10.0-jdk7 #29

See

Build failed in Jenkins: kafka-trunk-jdk7 #1168

See Changes: [me] KAFKA-3486: fix autocommit when partitions assigned manually -- [...truncated 1589 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest >

Re: KIP-4 Wiki Update

Hi Grant, One question that occurred to me is whether we want to take the chance to make it possible to pass a regex pattern for the desired topics in the metadata request. This would potentially improve the efficiency of `KafkaConsumer.subscribe` significantly for cases where there are large

Build failed in Jenkins: kafka-trunk-jdk8 #499

See Changes: [me] KAFKA-3486: fix autocommit when partitions assigned manually -- [...truncated 3139 lines...] kafka.consumer.ZookeeperConsumerConnectorTest > testConsumerDecoder PASSED

[jira] [Updated] (KAFKA-3486) Autocommit does not execute periodically when partitions are manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3486: - Resolution: Fixed Fix Version/s: 0.10.1.0 Status: Resolved

[jira] [Commented] (KAFKA-3486) Autocommit does not execute periodically when partitions are manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223132#comment-15223132 ] ASF GitHub Bot commented on KAFKA-3486: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-3486: fix autocommit when partitions ass...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1169 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is