Build failed in Jenkins: kafka-0.10.0-jdk7 #32

See Changes: [me] KAFKA-3483: Restructure ducktape tests to simplify running subsets of -- [...truncated 5276 lines...] org.apache.kafka.streams.processor.internals.StreamThreadTest > testMaybeCl

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223705#comment-15223705 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- If we can't provide a way tha

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223699#comment-15223699 ] Guozhang Wang commented on KAFKA-3491: -- So do you think that we probably should recon

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223693#comment-15223693 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- Automatic offset commit. Unle

Build failed in Jenkins: kafka-trunk-jdk7 #1173

See Changes: [me] KAFKA-3483: Restructure ducktape tests to simplify running subsets of -- [...truncated 1591 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > testCleanS

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223685#comment-15223685 ] Guozhang Wang commented on KAFKA-3491: -- "if it's actually a good feature to provide":

Re: KIP-4 Wiki Update

Hi Grant, Thanks for the write-ups. I had some comments besides the debate of metadata request principles: 1. One minor thing: "Note there is a special use case - automatic topic creation for TopicMetadataRequest, to trigger it user should set client_id=consumer and define only topic name:" I th

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223682#comment-15223682 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- [~guozhang] Comparing to othe

Re: [VOTE] KIP-4 Metadata Schema

Grant, 2. Hmm, I am not sure about the listener approach. It ignores configs like enable.topic.deletion and also opens the door for potential ordering issues since now there are two separate paths for propagating the metadata to the brokers. Could we just leave out markedForDeletion for now? In th

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223656#comment-15223656 ] Guozhang Wang commented on KAFKA-3491: -- If we do not want to enforce users to keep tr

Build failed in Jenkins: kafka-trunk-jdk7 #1172

See Changes: [me] MINOR: Clean up of SourceTaskOffsetCommiter -- [...truncated 2819 lines...] kafka.integration.SaslPlaintextTopicMetadataTest > testAliveBrokersListWithNoTopicsAfterABrokerShu

Re: offsets.retention.minutes

This is a legitimate question. Anyone have an opinion about changing the default behavior? If not, I can just submit a patch that does this. On Sat, Apr 2, 2016 at 10:59 PM, William Grim wrote: > ? > > On Fri, Apr 1, 2016 at 11:29 PM, William Grim wrote: > >> Hi, >> >> I read the reasoning ab

Kafka Mailing List Filters

The Kafka development mailing list can be a lot to keep up with. To cope, I have created a few filters to separate the most important communication from the noise. I have found using just a few filters help a lot. Below are my most reliable Gmail filters (they should be able to map to other email c

Build failed in Jenkins: kafka-0.10.0-jdk7 #31

See Changes: [me] MINOR: Clean up of SourceTaskOffsetCommiter -- [...truncated 3145 lines...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[15] PASSED kafka.log.BrokerCompressionTe

Build failed in Jenkins: kafka-trunk-jdk8 #503

See Changes: [me] KAFKA-3483: Restructure ducktape tests to simplify running subsets of -- [...truncated 3139 lines...] kafka.consumer.ZookeeperConsumerConnectorTest > testConsumerDecoder PASSED

Re: [RELEASE UPDATE] Postponing the next release candidates and canceling current vote

Just wanted to throw it out there that still double committing when the committer remembers to do so is useful -- daily updates on unit tests (as flaky as they can be) and system tests are still useful to have. Better to catch any branch-specific issues as early as possible. -Ewen On Fri, Apr 1,

[GitHub] kafka pull request: KAFKA-3483: Restructure ducktape tests to simp...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1162 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3483) Restructure ducktape tests to simplify running subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223622#comment-15223622 ] ASF GitHub Bot commented on KAFKA-3483: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3483) Restructure ducktape tests to simplify running subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3483: - Resolution: Fixed Fix Version/s: 0.10.0.0 0.10.1.0

Build failed in Jenkins: kafka-0.10.0-jdk7 #30

See Changes: [me] MINOR: Fix small typo in design section -- [...truncated 5236 lines...] org.apache.kafka.streams.processor.internals.StreamThreadTest > testMaybeClean PASSED org.apache.kafka

Build failed in Jenkins: kafka-trunk-jdk7 #1171

See Changes: [me] MINOR: Fix small typo in design section -- [...truncated 1589 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > testCleanShutdownFile PASSED kafka.log.

[jira] [Commented] (KAFKA-3496) Add reconnect attemps policies for client

[ https://issues.apache.org/jira/browse/KAFKA-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223607#comment-15223607 ] Ewen Cheslack-Postava commented on KAFKA-3496: -- [~fhussonnois] Despite a smal

[GitHub] kafka pull request: MINOR: server stop scripts don't work correctl...

Github user sasakitoa closed the pull request at: https://github.com/apache/kafka/pull/1145 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

Re: [DISCUSS] KIP-52 - Add Connector Control APIs

Ismael, Great point. Pause and resume should be idempotent and actually represent updating a resource that gets written to Kafka (although I must admit I don't know if the use of 202/Accepted should affect this at all), the restart endpoints seem a bit different as they are one-off immediate comma

[jira] [Commented] (KAFKA-3491) Issue with consumer close() in finally block with 'enable.auto.commit=true'

[ https://issues.apache.org/jira/browse/KAFKA-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223604#comment-15223604 ] Ewen Cheslack-Postava commented on KAFKA-3491: -- I don't know, this strikes me

[GitHub] kafka pull request: [Minor]: Clean up of SourceTaskOffsetCommiter

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-4 Metadata Schema

Responding to a few of the other comments: it seems that you propagated > the topic deletion marker by having the replicaManager read from ZK > directly. It seems that it would be simpler/consistent if the controller > propagates that information directly through UpdateMetaRequest. I was told th

Jenkins build is back to normal : kafka-trunk-jdk8 #501

See

[GitHub] kafka pull request: [documentation] Fix small typo in design secti...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-4 Metadata Schema

Hi Jun and Ismael, Initially I had 2 booleans used to indicate if a topic was internal and if a topic was marked for deletion. To save space on large deployments, Ismael suggested I break out the internal topics and deleted topics into their own lists. The idea was that instead of 2 bytes added pe

Re: [VOTE] KIP-4 Metadata Schema

Hi Jun, A couple of comments inline. On Sun, Apr 3, 2016 at 5:55 PM, Jun Rao wrote: > 1. It seems a bit weird to return just a list of internal topics w/o the > corresponding metadata. It also seems a bit weird to return the internal > topics even if the client doesn't ask for it. Good point.

Build failed in Jenkins: kafka-trunk-jdk7 #1170

See Changes: [junrao] KAFKA-3495; NetworkClient.blockingSendAndReceive` should rely on -- [...truncated 1583 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > testCleanSh

[jira] [Updated] (KAFKA-3495) `NetworkClient.blockingSendAndReceive` should rely on requestTimeout

[ https://issues.apache.org/jira/browse/KAFKA-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3495: --- Fix Version/s: 0.10.0.0 > `NetworkClient.blockingSendAndReceive` should rely on requestTimeout > -

Re: KIP-4 Wiki Update

Hi Jun, Yes, that makes it a bit challenging to do this in a manner that is fully compatible. Having said that, most (all?) languages have Perl-derived regexes (including Java), so it seems like it should be possible to define a subset for the protocol. But perhaps that's a bigger change that requ

Re: KIP-4 Wiki Update

Ismael, The reason that we only apply the regex for topics on the client side is that regex is java specific. So, for non-java clients, it would be a bit weird for them to include java specific stuff in the wire protocol. Thanks, Jun On Sun, Apr 3, 2016 at 12:36 AM, Ismael Juma wrote: > Hi Gr

[jira] [Commented] (KAFKA-3477) Add customizable StreamPartition into #to functions of Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223553#comment-15223553 ] ASF GitHub Bot commented on KAFKA-3477: --- GitHub user mjsax opened a pull request:

[GitHub] kafka pull request: [KAFKA-3477] [Kafka Streams] extended KStream/...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/1180 [KAFKA-3477] [Kafka Streams] extended KStream/KTable API to specify c… …ustom partitioner for sinks You can merge this pull request into a Git repository by running: $ git pull https://github

[jira] [Updated] (KAFKA-3495) `NetworkClient.blockingSendAndReceive` should rely on requestTimeout

[ https://issues.apache.org/jira/browse/KAFKA-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3495: --- Resolution: Fixed Fix Version/s: (was: 0.10.0.0) 0.10.1.0 Status: Reso

[jira] [Commented] (KAFKA-3495) `NetworkClient.blockingSendAndReceive` should rely on requestTimeout

[ https://issues.apache.org/jira/browse/KAFKA-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223550#comment-15223550 ] ASF GitHub Bot commented on KAFKA-3495: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3495; `NetworkClient.blockingSendAndRece...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1177 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk7 #1169

See Changes: [wangguoz] KAFKA-3419: clarify difference between topic subscription and partition -- [...truncated 1589 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > t

[jira] [Updated] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

[ https://issues.apache.org/jira/browse/KAFKA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3497: - Issue Type: Sub-task (was: Improvement) Parent: KAFKA-2590 > Streams ProcessorContext sho

[jira] [Commented] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

[ https://issues.apache.org/jira/browse/KAFKA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223533#comment-15223533 ] Guozhang Wang commented on KAFKA-3497: -- Thanks [~kawamuray]. I think we can consider

[jira] [Assigned] (KAFKA-3477) Add customizable StreamPartition into #to functions of Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-3477: -- Assignee: Matthias J. Sax > Add customizable StreamPartition into #to functions of Stre

Build failed in Jenkins: kafka-trunk-jdk8 #500

See Changes: [wangguoz] KAFKA-3419: clarify difference between topic subscription and partition -- [...truncated 1588 lines...] kafka.consumer.ZookeeperConsumerConnectorTest > testConsumerDecod

[jira] [Commented] (KAFKA-3338) Add print and writeAsText functions to the Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223510#comment-15223510 ] Guozhang Wang commented on KAFKA-3338: -- Sounds good! > Add print and writeAsText fun

[jira] [Updated] (KAFKA-3419) New consumer javadoc unclear on topic subscription vs partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3419: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-3419) New consumer javadoc unclear on topic subscription vs partition assignment

[ https://issues.apache.org/jira/browse/KAFKA-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223509#comment-15223509 ] ASF GitHub Bot commented on KAFKA-3419: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3419: clarify difference between topic s...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1158 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Created] (KAFKA-3498) Transient failure in kafka.api.SslConsumerTest.testSimpleConsumption

Guozhang Wang created KAFKA-3498: Summary: Transient failure in kafka.api.SslConsumerTest.testSimpleConsumption Key: KAFKA-3498 URL: https://issues.apache.org/jira/browse/KAFKA-3498 Project: Kafka

[jira] [Commented] (KAFKA-725) Broker Exception: Attempt to read with a maximum offset less than start offset

[ https://issues.apache.org/jira/browse/KAFKA-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223457#comment-15223457 ] ASF GitHub Bot commented on KAFKA-725: -- GitHub user srdo reopened a pull request:

[jira] [Commented] (KAFKA-725) Broker Exception: Attempt to read with a maximum offset less than start offset

[ https://issues.apache.org/jira/browse/KAFKA-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223456#comment-15223456 ] ASF GitHub Bot commented on KAFKA-725: -- Github user srdo closed the pull request at:

[GitHub] kafka pull request: KAFKA-725: Change behavior of Log/LogSegment w...

GitHub user srdo reopened a pull request: https://github.com/apache/kafka/pull/1178 KAFKA-725: Change behavior of Log/LogSegment when attempting read on an offset that's above high watermark. This should make Log.read act the same when startOffset is larger than maxOffset as it wou

[GitHub] kafka pull request: KAFKA-725: Change behavior of Log/LogSegment w...

Github user srdo closed the pull request at: https://github.com/apache/kafka/pull/1178 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

Re: [VOTE] KIP-4 Metadata Schema

Grant, Thanks for the writeup. A few comments. 1. It seems a bit weird to return just a list of internal topics w/o the corresponding metadata. It also seems a bit weird to return the internal topics even if the client doesn't ask for it. Would it be better to just add a flag in topic_metadata to

[jira] [Created] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

Yuto Kawamura created KAFKA-3497: Summary: Streams ProcessorContext should support forward() based on child name Key: KAFKA-3497 URL: https://issues.apache.org/jira/browse/KAFKA-3497 Project: Kafka

[jira] [Updated] (KAFKA-3497) Streams ProcessorContext should support forward() based on child name

[ https://issues.apache.org/jira/browse/KAFKA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura updated KAFKA-3497: - Description: Currently {{ProcessorContext}} only supports {{forward(K, V)}} which forwards KV to

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

Ashish, For the two benefits that you listed, fail fast can be done just in the implementation w/o getSupportedPrincipalTypes(). For avoiding the same check in all implementations, I think Grant is thinking of adding some ACL RPC request/response btw the client and broker directly, instead of writ

[jira] [Commented] (KAFKA-3480) Autogenerate metrics documentation

[ https://issues.apache.org/jira/browse/KAFKA-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223239#comment-15223239 ] Otis Gospodnetic commented on KAFKA-3480: - This would be great. I just hope the o

Jenkins build is back to normal : kafka-0.10.0-jdk7 #29

See

Build failed in Jenkins: kafka-trunk-jdk7 #1168

See Changes: [me] KAFKA-3486: fix autocommit when partitions assigned manually -- [...truncated 1589 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > testCleanShutdownFi

Re: KIP-4 Wiki Update

Hi Grant, One question that occurred to me is whether we want to take the chance to make it possible to pass a regex pattern for the desired topics in the metadata request. This would potentially improve the efficiency of `KafkaConsumer.subscribe` significantly for cases where there are large numb

Build failed in Jenkins: kafka-trunk-jdk8 #499

See Changes: [me] KAFKA-3486: fix autocommit when partitions assigned manually -- [...truncated 3139 lines...] kafka.consumer.ZookeeperConsumerConnectorTest > testConsumerDecoder PASSED kafka.c