Re: [VOTE] 0.11.0.0 RC2

Hi, One super minor issue (that can be fixed without a new RC): The big exactly-once stuff (KIP-98) doesn't actually show up as new features in the release notes. Most chunks appear as sub-tasks, but the new feature itself (KAFKA-4815) is marked as 0.11.1.0 so this is missing. I get that this is c

Re: [kafka-clients] Re: [VOTE] 0.11.0.0 RC2

Hi, Ismael, Thanks for running the release. +1. Verified quickstart on the 2.11 binary. Jun On Mon, Jun 26, 2017 at 3:53 PM, Ismael Juma wrote: > Hi Vahid, > > There are a few known issues when running Kafka on Windows. A PR with some > fixes is: https://github.com/apache/kafka/pull/3283. The

[GitHub] kafka pull request #3441: MINOR: Enable the TransactionsBounceTest

GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/3441 MINOR: Enable the TransactionsBounceTest I'll let this have multiple runs on the branch builder to see if it fails, and investigate if so. You can merge this pull request into a Git repository by r

[DISCUSS] KIP-172 Add regular-expression topic support for sink connector

Hi all, I have raised a new KIP at https://cwiki.apache.org/confluence/display/KAFKA/KIP+172%3A+Add+regular-expression+topic+support+for+sink+connector The corresponding JIRA is at https://issues.apache.org/jira/browse/KAFKA-3073 I look forward to your feedback. Thanks, Kenji Hayashida

Build failed in Jenkins: kafka-trunk-jdk8 #1762

See -- [...truncated 972.54 KB...] kafka.tools.MirrorMakerTest > testDefaultMirrorMakerMessageHandlerWithNoTimestampInSourceMessage STARTED kafka.tools.MirrorMakerTest > testDefaultMi

[GitHub] kafka pull request #3440: MINOR: Close producer in all KafkaProducer tests

GitHub user 10110346 opened a pull request: https://github.com/apache/kafka/pull/3440 MINOR: Close producer in all KafkaProducer tests After running KafkaProducer tests, we should close producer for free resource You can merge this pull request into a Git repository by running:

Jenkins build is back to normal : kafka-trunk-jdk8 #1761

See

Jenkins build is back to normal : kafka-0.10.1-jdk7 #121

See

[jira] [Created] (KAFKA-5522) ListOffset should take LSO into account when searching by timestamp

Jason Gustafson created KAFKA-5522: -- Summary: ListOffset should take LSO into account when searching by timestamp Key: KAFKA-5522 URL: https://issues.apache.org/jira/browse/KAFKA-5522 Project: Kafka

[GitHub] kafka pull request #3437: MINOR: Make JmxMixin wait for the monitored proces...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3437 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [Vote] KIP-150 - Kafka-Streams Cogroup

Kyle, It seems you already have enough committer votes (me, Jay, Sriram, Damian). Guozhang On Mon, Jun 26, 2017 at 11:06 AM, Kyle Winkelman wrote: > Bumping this so it is easy to find now that the discussions have died down. > > Thanks, > Kyle > > On Jun 9, 2017 6:32 PM, "Sriram Subramanian"

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

Hmm... I'm not sure how this can achieve "closing the store before restoration, re-open it with a slight different config, and then close-and-reopen store for query" pattern? You need to be able to access the store object in order to do this right? Guozhang On Mon, Jun 26, 2017 at 7:40 AM, Bill

[jira] [Created] (KAFKA-5521) Support replicas movement between log directories (KIP-113)

Dong Lin created KAFKA-5521: --- Summary: Support replicas movement between log directories (KIP-113) Key: KAFKA-5521 URL: https://issues.apache.org/jira/browse/KAFKA-5521 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-5367) Producer should not expiry topic from metadata cache if accumulator still has data for this topic

[ https://issues.apache.org/jira/browse/KAFKA-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-5367. - Resolution: Invalid > Producer should not expiry topic from metadata cache if accumulator still has > dat

[GitHub] kafka pull request #3439: KAFKA-5464: StreamsKafkaClient should not use Stre...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3439 KAFKA-5464: StreamsKafkaClient should not use StreamsConfig.POLL_MS_CONFIG You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka kafka-5464-

Re: [VOTE] 0.11.0.0 RC2

Hi Vahid, There are a few known issues when running Kafka on Windows. A PR with some fixes is: https://github.com/apache/kafka/pull/3283. The fact that the index cannot be accessed indicates that it may be a similar issue. I suggest we move this discussion to the relevant JIRAs instead of the rele

Re: Open PRs

Hi Tom, The team is really overloaded, we can see it from the activity in this dev group. Let's wait a bit while the team is finishing their awesome work. With best wishes, A guy awaiting review for about 2 months @tashoyan 2017-06-26 22:27 GMT+03:00 Paolo Patierno : > Hi Ismael, > > thanks for

Re: [VOTE] 0.11.0.0 RC2

Hi Ismael, This is the output of core tests from the start until the first failed test. kafka.admin.AdminRackAwareTest > testAssignmentWithRackAwareWithUnevenRacks PASSED kafka.admin.AdminRackAwareTest > testAssignmentWith2ReplicasRackAware PASSED kafka.admin.AdminRackAwareTest > testAssign

Re: [DISCUSS] KIP-171: Extend Consumer Group Reset Offset for Stream Application

Jorge, thanks a lot for this KIP. Allowing the reset streams applications with arbitrary start offset is something we got multiple requests already. Couple of clarification question: - why do you want to deprecate the current tool instead of extending the current tool with the stuff the offset

Re: [VOTE] 0.11.0.0 RC2

+1 from me. I verified the artifacts and tested basic producing and consuming. I also verified the procedure for upgrading to the new message format. -Jason On Mon, Jun 26, 2017 at 1:53 PM, Ismael Juma wrote: > Hi Vahid, > > Can you please check which test fails first? The errors you mentioned

Re: [VOTE] 0.11.0.0 RC2

Hi Vahid, Can you please check which test fails first? The errors you mentioned can happen if a test fails and doesn't clean-up properly. Ismael On Mon, Jun 26, 2017 at 8:41 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Ismael, > > To answer your questions: > > 1. Yes, the issu

Re: [VOTE] 0.11.0.0 RC2

Hi, Heroku has been doing additional performance testing on (1) log compaction and, separately (2) Go clients with older message format against 0.11-rc2 brokers with new message format. For log compaction, we've tested with messages using a single key, messages using unique keys, and messages wit

[GitHub] kafka pull request #3438: KAFKA-3465: Clarify warning message of ConsumerOff...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/3438 KAFKA-3465: Clarify warning message of ConsumerOffsetChecker Add that the tool works with the old consumer only. You can merge this pull request into a Git repository by running: $ git p

[DISCUSS] KIP-171: Extend Consumer Group Reset Offset for Stream Application

Hi all, I'd like to start the discussion to add reset offset tooling for Stream applications. The KIP can be found here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-171+-+Extend+Consumer+Group+Reset+Offset+for+Stream+Application Thanks, Jorge.

[jira] [Created] (KAFKA-5520) Extend Consumer Group Reset Offset tool for Stream Applications

Jorge Quilcate created KAFKA-5520: - Summary: Extend Consumer Group Reset Offset tool for Stream Applications Key: KAFKA-5520 URL: https://issues.apache.org/jira/browse/KAFKA-5520 Project: Kafka

Re: [VOTE] 0.11.0.0 RC2

Hi Ismael, To answer your questions: 1. Yes, the issues exists in trunk too. 2. I haven't checked with Cygwin, but I can give it a try. And thanks for addressing this issue. I can confirm with your PR I no longer see it. But now that the tests progress I see quite a few errors like this in co

Re: Open PRs

Hi Ismael, thanks for replying to Tom's comment because it is the same situation here with some PRs opened even 2 weeks ago. I was pretty sure that the reason was the 0.11.0 release and you have just confirmed it. Looking forward to see the new Kafka version out, pushing more on the next one

[jira] [Created] (KAFKA-5519) Support for multiple certificates in a single keystore

Alla Tumarkin created KAFKA-5519: Summary: Support for multiple certificates in a single keystore Key: KAFKA-5519 URL: https://issues.apache.org/jira/browse/KAFKA-5519 Project: Kafka Issue Ty

[GitHub] kafka pull request #3437: MINOR: Make JmxMixing wait for the monitored proce...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3437 MINOR: Make JmxMixing wait for the monitored process to be listening on the JMX port before launching JmxTool You can merge this pull request into a Git repository by running: $ git pull https

Re: [Vote] KIP-150 - Kafka-Streams Cogroup

Bumping this so it is easy to find now that the discussions have died down. Thanks, Kyle On Jun 9, 2017 6:32 PM, "Sriram Subramanian" wrote: > +1 > > On Fri, Jun 9, 2017 at 2:24 PM, Jay Kreps wrote: > > > +1 > > > > -Jay > > > > On Thu, Jun 8, 2017 at 11:16 AM, Guozhang Wang > wrote: > > > >

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

The KIP and PR have been updated please go take a look and vote. For those worried about the [DISCUSS] Streams DSL/StateStore Refactoring email thread affecting this I believe the cogroup methods fit well into the streams dsl and won't need to change. We can update the aggregate methods in the sam

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

+1 on the original KIP I actually prefer TotalTopicCount because it makes it clearer that it is a cluster-wide count. OfflinePartitionsCount is global to the cluster (but it is fairly clear that the controller is SoT on that). TopicCount on the other hand could be misread as a local count since Par

[jira] [Created] (KAFKA-5518) General Kafka connector performanc workload

Chen He created KAFKA-5518: -- Summary: General Kafka connector performanc workload Key: KAFKA-5518 URL: https://issues.apache.org/jira/browse/KAFKA-5518 Project: Kafka Issue Type: Bug Compo

Jenkins build is back to normal : kafka-0.10.2-jdk7 #177

See

Re: Open PRs

Hey Tom, Thanks for the PRs you have submitted. As you suggested, most of us are busy testing the upcoming release. It's a major release with many exciting features and we want to make sure it's a high quality release. I understand that it can be frustrating to wait. At the same time, there are n

Open PRs

I realise that 0.11.0.0 is imminent and so the committers are rightly going to be rather focussed on that, but I opened some PRs nearly a week ago and they don't seem to have been looked at. Even a comment on the PR to the effect of "We'll look at this right after 0.11.0.0" would at least reassure

[GitHub] kafka pull request #3436: KAFKA-5517: Add id to config HTML tables to allow ...

GitHub user tombentley opened a pull request: https://github.com/apache/kafka/pull/3436 KAFKA-5517: Add id to config HTML tables to allow linking You can merge this pull request into a Git repository by running: $ git pull https://github.com/tombentley/kafka KAFKA-5517 Altern

[jira] [Created] (KAFKA-5517) Support linking to particular configuration parameters

Tom Bentley created KAFKA-5517: -- Summary: Support linking to particular configuration parameters Key: KAFKA-5517 URL: https://issues.apache.org/jira/browse/KAFKA-5517 Project: Kafka Issue Type:

Re: [VOTE] KIP-160: Augment KStream.print() to allow users pass in extra parameters in the printed string

+1 On Sat, Jun 24, 2017 at 4:37 AM, Matthias J. Sax wrote: > +1 > > On 6/23/17 9:43 PM, Guozhang Wang wrote: > > +1 > > > > On Fri, Jun 23, 2017 at 3:16 AM, Eno Thereska > > wrote: > > > >> +1 thanks! > >> > >> Eno > >>> On 23 Jun 2017, at 05:29, James Chain > wrote: > >>> > >>> Hi all, > >>>

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

Thinking about this some more, I have another approach. Leave the first parameter of as String in the StateRestoreListener interface. But we'll provide 2 default abstract classes one implementing StateRestoreCallback and the other implementing the BatchingStateRestoreCallback. Both abstract clas

Re: [DISCUSS] KIP-167: Add a restoreAll method to StateRestoreCallback

Guozhang, Thanks for the comments. I think that will work, but my concern is it might not be as clear to users that want to receive external notification of the restore progress separately (say for reporting purposes) and still send separate signals to the state store for resource management task

[GitHub] kafka pull request #3435: KAFKA-4388 Recommended values for converters from ...

GitHub user evis opened a pull request: https://github.com/apache/kafka/pull/3435 KAFKA-4388 Recommended values for converters from plugins Questions to reviewers: 1. Should we cache `converterRecommenders.validValues()`, `SinkConnectorConfig.configDef()` and `SourceConnectorCon

[jira] [Created] (KAFKA-5516) Formatting verifiable producer/consumer output in a similar fashion

Paolo Patierno created KAFKA-5516: - Summary: Formatting verifiable producer/consumer output in a similar fashion Key: KAFKA-5516 URL: https://issues.apache.org/jira/browse/KAFKA-5516 Project: Kafka

[GitHub] kafka pull request #3434: KAFKA-5516: Formatting verifiable producer/consume...

GitHub user ppatierno opened a pull request: https://github.com/apache/kafka/pull/3434 KAFKA-5516: Formatting verifiable producer/consumer output in a similar fashion You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppatierno/kaf

[jira] [Created] (KAFKA-5515) Consider removing date formatting from Segments class

Bill Bejeck created KAFKA-5515: -- Summary: Consider removing date formatting from Segments class Key: KAFKA-5515 URL: https://issues.apache.org/jira/browse/KAFKA-5515 Project: Kafka Issue Type: I

[jira] [Created] (KAFKA-5514) KafkaConsumer ignores default values in Properties object because of incorrect use of Properties object.

Geert Schuring created KAFKA-5514: - Summary: KafkaConsumer ignores default values in Properties object because of incorrect use of Properties object. Key: KAFKA-5514 URL: https://issues.apache.org/jira/browse/KAFK

[GitHub] kafka pull request #3433: MINOR: typo in variable name "unkownInfo"

GitHub user evis opened a pull request: https://github.com/apache/kafka/pull/3433 MINOR: typo in variable name "unkownInfo" You can merge this pull request into a Git repository by running: $ git pull https://github.com/evis/kafka unkown_info_typo Alternatively you can review

[GitHub] kafka pull request #3432: KAFKA-5372: fixes to state transitions

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/3432 KAFKA-5372: fixes to state transitions You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka KAFKA-5372-state-transitions Alter

Re: [VOTE] 0.11.0.0 RC2

Hi Vahid, Sorry for not replying to the previous email, I had missed it. A couple of questions: 1. Is this also happening in trunk? Seems like it should be the case for months and seemingly no-one reported it until the RC stage. 2. Is it correct that this only happens when compiling on Windows wi

[jira] [Resolved] (KAFKA-4563) State transitions error PARTITIONS_REVOKED to NOT_RUNNING

[ https://issues.apache.org/jira/browse/KAFKA-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-4563. - Resolution: Duplicate This will be fixed as part of KAFKA-5372 > State transitions error PARTITIO

[GitHub] kafka pull request #3431: MINOR: Adjust checkstyle suppression paths to work...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3431 MINOR: Adjust checkstyle suppression paths to work on Windows Use the file name whenever possible and replace / with [/\\] when it's not. Also remove unnecessary suppresions. You can merg

[jira] [Created] (KAFKA-5513) Contradicting scalaDoc for AdminUtils.assignReplicasToBrokers

Charly Molter created KAFKA-5513: Summary: Contradicting scalaDoc for AdminUtils.assignReplicasToBrokers Key: KAFKA-5513 URL: https://issues.apache.org/jira/browse/KAFKA-5513 Project: Kafka

[jira] [Created] (KAFKA-5512) KafkaConsumer: High memory allocation rate when idle

Stephane Roset created KAFKA-5512: - Summary: KafkaConsumer: High memory allocation rate when idle Key: KAFKA-5512 URL: https://issues.apache.org/jira/browse/KAFKA-5512 Project: Kafka Issue Ty

[GitHub] kafka pull request #3430: KAFKA-4750: RocksDBStore always deletes null value...

GitHub user evis opened a pull request: https://github.com/apache/kafka/pull/3430 KAFKA-4750: RocksDBStore always deletes null values @guozhangwang @mjsax You can merge this pull request into a Git repository by running: $ git pull https://github.com/evis/kafka rocksdbstore_nu

[jira] [Created] (KAFKA-5511) ConfigDef.define() overloads take too many parameters

Evgeny Veretennikov created KAFKA-5511: -- Summary: ConfigDef.define() overloads take too many parameters Key: KAFKA-5511 URL: https://issues.apache.org/jira/browse/KAFKA-5511 Project: Kafka