Running system tests on mac

Hi. Has anyone had luck running Kafka system tests on a Mac. I have a MacOS Mojave 10.14.6. I got Python 3.6.9 using pyenv. However, the command *ducktape tests/kafkatest/tests* yields the following error, making it look like some Python incompatibility issue. $ ducktape tests/kafkatest/tests Tra

Re: Running system tests on mac

Hello, I successfully run system tests on Mac with Docker. I followed the instruction on [1] and it works like a charm. [1] https://github.com/apache/kafka/tree/trunk/tests#running-tests-using-docker > 29 июня 2020 г., в 12:14, Gokul Ramanan Subramanian > написал(а): > > Hi. > > Has anyone

[jira] [Created] (KAFKA-10210) Use templated log message for LogContext

Fabian Lange created KAFKA-10210: Summary: Use templated log message for LogContext Key: KAFKA-10210 URL: https://issues.apache.org/jira/browse/KAFKA-10210 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

Hi, Does anyone have any comments about this? If not I'll likely start a vote in a couple of days. Kind regards, Tom On Mon, Jun 8, 2020 at 4:56 PM Tom Bentley wrote: > Hi all, > > I've opened a small KIP seeking to deprecate and replace a couple of > methods of DescribeLogDirsResult which re

Re: Running system tests on mac

Thanks. This worked. I had to use the Docker desktop app, not docker-machine, with which it gave errors around permissions setting up /etc/hosts. On Mon, Jun 29, 2020 at 10:24 AM Nikolay Izhikov wrote: > Hello, > > I successfully run system tests on Mac with Docker. > I followed the instruction

[DISCUSS] KIP-632: Add DirectoryConfigProvider

Hi all, I would like to start discussion on a small KIP to make using the config provider mechanism more ergonomic on Kubernetes: https://cwiki.apache.org/confluence/display/KAFKA/KIP-632%3A+Add+DirectoryConfigProvider If you have a few minutes to have a look I'd be grateful for any feedback. M

Re: [DISCUSS] KIP-617: Allow Kafka Streams State Stores to be iterated backwards

Thanks, John! Make sense to reconsider the current approach. I was heading in a similar direction while drafting the implementation. Metered, Caching, and other layers will also have to get duplicated to build up new methods in `Stores` factory, and class casting issues would appear on stores crea

Re: [VOTE] KIP-418: A method-chaining way to branch KStream

This will be a great addition. Thanks Ivan! +1 (non-binding) On Fri, Jun 26, 2020 at 7:07 PM John Roesler wrote: > Thanks, Ivan! > > I’m +1 (binding) > > -John > > On Thu, May 28, 2020, at 17:24, Ivan Ponomarev wrote: > > Hello all! > > > > I'd like to start the vote for KIP-418 which proposes

Jenkins build is back to normal : kafka-trunk-jdk11 #1601

See

[jira] [Created] (KAFKA-10211) Add DirectoryConfigProvider

Tom Bentley created KAFKA-10211: --- Summary: Add DirectoryConfigProvider Key: KAFKA-10211 URL: https://issues.apache.org/jira/browse/KAFKA-10211 Project: Kafka Issue Type: Improvement

Build failed in Jenkins: kafka-trunk-jdk14 #249

See Changes: [github] MINOR: Iterate over partitions only once when building -- [...truncated 6.36 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldCapture

Jenkins build is back to normal : kafka-trunk-jdk8 #4673

See

Announcing ApacheCon @Home 2020

Hi, Apache enthusiast! (You’re receiving this because you’re subscribed to one or more dev or user mailing lists for an Apache Software Foundation project.) The ApacheCon Planners and the Apache Software Foundation are pleased to announce that ApacheCon @Home will be held online, September 29

[jira] [Created] (KAFKA-10212) Describing a topic with the TopicCommand fails if unauthorised to use ListPartitionReassignments API

David Jacot created KAFKA-10212: --- Summary: Describing a topic with the TopicCommand fails if unauthorised to use ListPartitionReassignments API Key: KAFKA-10212 URL: https://issues.apache.org/jira/browse/KAFKA-10212

Re: [DISCUSS] Apache Kafka 2.6.0 release

Hi Randall, We have discovered an annoying issue that we introduced in 2.5: Describing topics with the command line tool fails if the user does not have the privileges to access the ListPartitionReassignments API. I believe that this is the case for most non-admin users. I propose to include the

[jira] [Created] (KAFKA-10213) Prefer --bootstrap-server in ducktape tests for Kafka clients

Vinoth Chandar created KAFKA-10213: -- Summary: Prefer --bootstrap-server in ducktape tests for Kafka clients Key: KAFKA-10213 URL: https://issues.apache.org/jira/browse/KAFKA-10213 Project: Kafka

[jira] [Created] (KAFKA-10214) fix flaky zookeeper_tls_test.py

Chia-Ping Tsai created KAFKA-10214: -- Summary: fix flaky zookeeper_tls_test.py Key: KAFKA-10214 URL: https://issues.apache.org/jira/browse/KAFKA-10214 Project: Kafka Issue Type: Bug

Re: [DISCUSS] Apache Kafka 2.6.0 release

Thanks for raising this, David. I agree it makes sense to include this fix in 2.6, so I've adjusted the "Fix Version(s)" field to include '2.6.0'. Best regards, Randall On Mon, Jun 29, 2020 at 8:25 AM David Jacot wrote: > Hi Randall, > > We have discovered an annoying issue that we introduced

Re: Running system tests on mac

I have been running the tests for over 5 hours now, and only 60 or so out of the 600 tests are through. Many tests are timing out. For example: [INFO:2020-06-29 07:52:54,606]: RunnerClient: kafkatest.tests.streams.streams_cooperative_rebalance_upgrade_test.StreamsCooperativeRebalanceUpgradeTest.te

Re: Running system tests on mac

Hello. > Is this common? At this rate, the tests would take about 2 days to complete > and there'd probably be lots of failures. Yes. System tests are heavy and resource expensive. You can run a single test with the TC_PATH environment variable. ``` TC_PATHS="tests/kafkatest/tests/client/plugga

[jira] [Created] (KAFKA-10215) MockProcessorContext doesn't work with SessionStores

John Roesler created KAFKA-10215: Summary: MockProcessorContext doesn't work with SessionStores Key: KAFKA-10215 URL: https://issues.apache.org/jira/browse/KAFKA-10215 Project: Kafka Issue Ty

[jira] [Created] (KAFKA-10217) Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetrics

John Roesler created KAFKA-10217: Summary: Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetrics Key: KAFKA-10217 URL: https://issues.apache.org/jira/browse/KAFKA-10217

[jira] [Created] (KAFKA-10216) Loosen permission for listing reassignments

Jason Gustafson created KAFKA-10216: --- Summary: Loosen permission for listing reassignments Key: KAFKA-10216 URL: https://issues.apache.org/jira/browse/KAFKA-10216 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-trunk-jdk14 #250

See Changes: [github] MINOR: Upgrade jetty to 9.4.30.v20200611 (#8893) -- [...truncated 3.18 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldFailIfValueI

Build failed in Jenkins: kafka-trunk-jdk8 #4674

See Changes: [github] MINOR: Upgrade jetty to 9.4.30.v20200611 (#8893) -- [...truncated 3.16 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldNotUpdateSto

Build failed in Jenkins: kafka-trunk-jdk11 #1602

See Changes: [github] MINOR: Upgrade jetty to 9.4.30.v20200611 (#8893) -- [...truncated 3.18 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldCre

Re: [VOTE] KIP-629: Use racially neutral terms in our codebase

Thank you Everyone, KIP-629 has been adopted with 7 binding +1 and 5 non-binding +1 votes. Xavier On Sun, Jun 28, 2020 at 10:10 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > +1 (binding) > > On Fri, Jun 26, 2020 at 3:10 AM Benny Lee wrote: > > > > > > > ___

[jira] [Resolved] (KAFKA-6508) Look into optimizing StreamPartitionAssignor StandbyTask Assignment

[ https://issues.apache.org/jira/browse/KAFKA-6508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-6508. Resolution: Fixed This should be addressed via KIP-441 > Look into optimizing Stre

[jira] [Created] (KAFKA-10218) DistributedHerder's canReadConfigs field is never reset to true

Chris Egerton created KAFKA-10218: - Summary: DistributedHerder's canReadConfigs field is never reset to true Key: KAFKA-10218 URL: https://issues.apache.org/jira/browse/KAFKA-10218 Project: Kafka

Build failed in Jenkins: kafka-2.6-jdk8 #64

See Changes: [konstantine] KAFKA-9509: Increase timeout when consuming records to fix flaky test in -- [...truncated 3.15 MB...] org.apache.kafka.streams.test.ConsumerRecord

Build failed in Jenkins: kafka-trunk-jdk14 #251

See Changes: [github] KAFKA-9509: Increase timeout when consuming records to fix flaky test in [github] MINOR: re-enable EosBetaUpgradeIntegrationTest (#8953) -- [...tr

Build failed in Jenkins: kafka-trunk-jdk11 #1603

See Changes: [github] KAFKA-9509: Increase timeout when consuming records to fix flaky test in [github] MINOR: re-enable EosBetaUpgradeIntegrationTest (#8953) -- [...t

Build failed in Jenkins: kafka-trunk-jdk8 #4675

See Changes: [github] KAFKA-9509: Increase timeout when consuming records to fix flaky test in [github] MINOR: re-enable EosBetaUpgradeIntegrationTest (#8953) -- [...tr

Re: [VOTE] KIP-418: A method-chaining way to branch KStream

+1 (binding). Thanks Ivan! Guozhang On Mon, Jun 29, 2020 at 3:55 AM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > This will be a great addition. Thanks Ivan! > > +1 (non-binding) > > On Fri, Jun 26, 2020 at 7:07 PM John Roesler wrote: > > > Thanks, Ivan! > > > > I’m +1 (bin

Re: [VOTE] KIP-623: Add "internal-topics" option to streams application reset tool

Thanks Joel, the KIP lgtm. A minor suggestion is to explain where users can get the list of internal topics of a given application, and maybe also add it as part of the helper scripts, for example via topology description. Overall, I'm +1 as well (binding). Guozhang On Sat, Jun 27, 2020 at 4:3

[DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

Hi everyone, I would like to start the discussion for KIP-634:https://cwiki.apache.org/confluence/display/KAFKA/KIP-634%3A+Complementary+support+for+headers+in+Kafka+Streams+DSL Looking forward to your feedback. Thanks! Jorge.

Re: [VOTE] KIP-623: Add "internal-topics" option to streams application reset tool

Thanks for the KIP, Joel! It seems like a good pattern to document would be to first run with —dry-run and without —internal-topics to list all potential topics, and then to use —internal-topics if needed to limit the internal topics to delete. Just to make sure, would we have a sanity check t

Re: [VOTE] KIP-623: Add "internal-topics" option to streams application reset tool

Oh, I meant to say, if that’s the case, then I’m +1 (binding) also :) Thanks again, John On Mon, Jun 29, 2020, at 19:09, John Roesler wrote: > Thanks for the KIP, Joel! > > It seems like a good pattern to document would be to first run with > —dry-run and without —internal-topics to list all po

Build failed in Jenkins: kafka-trunk-jdk11 #1604

See Changes: [github] add toString for RocksDbWindowBytesStoreSupplier (#8952) -- [...truncated 3.18 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest > sh

Re: [VOTE] KIP-418: A method-chaining way to branch KStream

Thanks for the KIP Ivan, +1 (binding). -Bill On Mon, Jun 29, 2020 at 7:22 PM Guozhang Wang wrote: > +1 (binding). Thanks Ivan! > > > Guozhang > > On Mon, Jun 29, 2020 at 3:55 AM Jorge Esteban Quilcate Otoya < > quilcate.jo...@gmail.com> wrote: > > > This will be a great addition. Thanks Ivan! >

[jira] [Created] (KAFKA-10219) KStream API support for multiple cluster broker

Sachin Kurle created KAFKA-10219: Summary: KStream API support for multiple cluster broker Key: KAFKA-10219 URL: https://issues.apache.org/jira/browse/KAFKA-10219 Project: Kafka Issue Type: B

Build failed in Jenkins: kafka-trunk-jdk14 #252

See Changes: [github] add toString for RocksDbWindowBytesStoreSupplier (#8952) -- [...truncated 3.19 MB...] org.apache.kafka.streams.internals.WindowStoreFacadeTest > sh

Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

Hi Daniel, Thanks for working on this KIP. Proposed changes looks good to me, minor comments: 1. We use "command-config" option name in most of the cmdline tools to pass config properties file. We can use the same name here. 2. Not sure, if we need a separate option to pass an consumer property

[jira] [Resolved] (KAFKA-10194) run the reset tool between stopping StreamsOptimizedTest and starting the new one

[ https://issues.apache.org/jira/browse/KAFKA-10194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10194. Resolution: Duplicate > run the reset tool between stopping StreamsOptimizedTest and start

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

Hi Sagar, That’s a good observation; yes, it should go in the ReadOnlyKeyValueStore interface. Thanks again for the great work, John On Sun, Jun 28, 2020, at 23:54, Sagar wrote: > Hi John, > > Thank you for the positive feedback! The meaningful discussions we had on > the mailing list helped m

Build failed in Jenkins: kafka-trunk-jdk8 #4676

See Changes: [github] add toString for RocksDbWindowBytesStoreSupplier (#8952) -- [...truncated 3.16 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest > sho

Build failed in Jenkins: kafka-trunk-jdk11 #1605

See Changes: [mkumar] KAFKA-10212: Describing a topic with the TopicCommand fails if -- [...truncated 3.18 MB...] org.apache.kafka.streams.scala.kstream.MaterializedTest

Re: [VOTE] KIP-623: Add "internal-topics" option to streams application reset tool

Thanks John for the great suggestion. +1 for enforcing the prefix check for the `--internal-topics` list. On Mon, Jun 29, 2020 at 5:11 PM John Roesler wrote: > Oh, I meant to say, if that’s the case, then I’m +1 (binding) also :) > > Thanks again, > John > > On Mon, Jun 29, 2020, at 19:09, John

Build failed in Jenkins: kafka-trunk-jdk14 #253

See Changes: [mkumar] KAFKA-10212: Describing a topic with the TopicCommand fails if -- [...truncated 3.18 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shou

Build failed in Jenkins: kafka-trunk-jdk8 #4677

See Changes: [mkumar] KAFKA-10212: Describing a topic with the TopicCommand fails if -- [...truncated 3.16 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shoul

Jenkins build is back to normal : kafka-2.6-jdk8 #65

See