[jira] [Created] (KAFKA-16845) Migrate ReplicationQuotasTestRig to new test infra
Chia-Ping Tsai created KAFKA-16845: -- Summary: Migrate ReplicationQuotasTestRig to new test infra Key: KAFKA-16845 URL: https://issues.apache.org/jira/browse/KAFKA-16845 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai as title -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16843) Remove preAppendErrors from createPutCacheCallback
Chia-Ping Tsai created KAFKA-16843: -- Summary: Remove preAppendErrors from createPutCacheCallback Key: KAFKA-16843 URL: https://issues.apache.org/jira/browse/KAFKA-16843 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai origin discussion: [https://github.com/apache/kafka/pull/16072#pullrequestreview-2077368462] The method `createPutCacheCallback` has a input argument `preAppendErrors` [0]. It is used to keep the "error" happens before appending. However, the pre-append error is handled before by calling `responseCallback` [1]. Hence, we can remove `preAppendErrors`. [0] https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L387 [1] https://github.com/apache/kafka/blob/4f55786a8a86fe228a0b10a2f28529f5128e5d6f/core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala#L927C15-L927C84 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10234) The key/value deserializer used by ConsoleConsumer is not closed
[ https://issues.apache.org/jira/browse/KAFKA-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10234. Resolution: Invalid > The key/value deserializer used by ConsoleConsumer is not closed > > > Key: KAFKA-10234 > URL: https://issues.apache.org/jira/browse/KAFKA-10234 > Project: Kafka > Issue Type: Bug > Reporter: Chia-Ping Tsai > Assignee: Chia-Ping Tsai >Priority: Minor > > We instantiate, configure and use them but them are never closed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16828) RackAwareTaskAssignorTest failed
[ https://issues.apache.org/jira/browse/KAFKA-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16828. Fix Version/s: 3.8.0 Resolution: Fixed > RackAwareTaskAssignorTest failed > > > Key: KAFKA-16828 > URL: https://issues.apache.org/jira/browse/KAFKA-16828 > Project: Kafka > Issue Type: Test >Reporter: Luke Chen >Assignee: Kuan Po Tseng >Priority: Major > Fix For: 3.8.0 > > > Found in the latest trunk build. > It fails many tests in `RackAwareTaskAssignorTest` suite. > > https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-15951/7/#showFailuresLink -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16830) Remove the scala version formatters support
Chia-Ping Tsai created KAFKA-16830: -- Summary: Remove the scala version formatters support Key: KAFKA-16830 URL: https://issues.apache.org/jira/browse/KAFKA-16830 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Kuan Po Tseng Fix For: 4.0.0 [https://github.com/apache/kafka/blob/trunk/tools/src/main/java/org/apache/kafka/tools/consumer/ConsoleConsumerOptions.java#L72] Those deprecated formatters "strings" should be removed from 4.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16795) Fix broken compatibility in kafka.tools.NoOpMessageFormatter, kafka.tools.DefaultMessageFormatter, and kafka.tools.LoggingMessageFormatter
[
https://issues.apache.org/jira/browse/KAFKA-16795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16795.
Resolution: Fixed
> Fix broken compatibility in kafka.tools.NoOpMessageFormatter,
> kafka.tools.DefaultMessageFormatter, and kafka.tools.LoggingMessageFormatter
> --
>
> Key: KAFKA-16795
> URL: https://issues.apache.org/jira/browse/KAFKA-16795
> Project: Kafka
> Issue Type: Sub-task
>Reporter: Kuan Po Tseng
>Assignee: Kuan Po Tseng
>Priority: Major
> Fix For: 3.8.0
>
>
> [{{0bf830f}}|https://github.com/apache/kafka/commit/0bf830fc9c3915bc99b6e487e6083dabd593c5d3]
> moved NoOpMessageFormatter, DefaultMessageFormatter and
> LoggingMessageFormatter package from {{kafka.tools}} to
> {{{}org.apache.kafka.tools.consumer{}}}{{{}{}}}
> These classes could be used via cmd kafka-console-consumer.sh. We should have
> a dependency cycle before 3.8.0 comes out.
>
> {code:java}
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 \
> --topic streams-wordcount-output \
> --from-beginning \
> --formatter kafka.tools.DefaultMessageFormatter \
> --property print.key=true \
> --property print.value=true \
> --property
> key.deserializer=org.apache.kafka.common.serialization.StringDeserializer \
> --property
> value.deserializer=org.apache.kafka.common.serialization.LongDeserializer{code}
> The goal in this Jira is to allow user to keep using
> {{{}kafka.tools.NoOpMessageFormatter{}}},
> {{{}kafka.tools.DefaultMessageFormatter{}}}, and
> {{{}kafka.tools.LoggingMessageFormatter{}}}, but we also display warning
> messages to say those "strings" will be removed in 4.0.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [VOTE] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder
+1 Thanks for Yang to take over this! > Frank Yang 於 2024年5月24日 凌晨12:27 寫道： > > Hi all, > > I would like to start a vote on KIP-1047: Introduce new > org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder. > > KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1047+Introduce+new+org.apache.kafka.tools.api.Decoder+to+replace+kafka.serializer.Decoder > > Discussion thread: > https://lists.apache.org/thread/n3k6vb4vddl1s5nopcyglnddtvzp4j63 > > Thanks and regards, > PoAn
[jira] [Resolved] (KAFKA-16829) Consider removing delegation.token.master.key
[ https://issues.apache.org/jira/browse/KAFKA-16829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16829. Resolution: Duplicate see KAFKA-12601 > Consider removing delegation.token.master.key > - > > Key: KAFKA-16829 > URL: https://issues.apache.org/jira/browse/KAFKA-16829 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai > Assignee: Chia-Ping Tsai >Priority: Minor > > It was marked as deprecated since 2020 [0], and maybe we should remove it now. > [0] > https://cwiki.apache.org/confluence/display/KAFKA/KIP-681:+Rename+master+key+in+delegation+token+feature -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16829) Consider removing delegation.token.master.key
Chia-Ping Tsai created KAFKA-16829: -- Summary: Consider removing delegation.token.master.key Key: KAFKA-16829 URL: https://issues.apache.org/jira/browse/KAFKA-16829 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai It was marked as deprecated since 2020 [0], and maybe we should remove it now. [0] https://cwiki.apache.org/confluence/display/KAFKA/KIP-681:+Rename+master+key+in+delegation+token+feature -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder
Thanks for Josep's response > We can add this to 3.8.0, but keep in mind the KIP is not voted yet (as far as I can see), so I would highly encourage to start the vote thread ASAP and strat with the implementation right after. sure. We will file a draft PR at the same time! Josep Prat 於 2024年5月23日 週四 上午12:31寫道： > Hi all, > > We can add this to 3.8.0, but keep in mind the KIP is not voted yet (as far > as I can see), so I would highly encourage to start the vote thread ASAP > and strat with the implementation right after. > > Best, > > - > Josep Prat > Open Source Engineering Director, aivenjosep.p...@aiven.io | > +491715557497 | aiven.io > Aiven Deutschland GmbH > Alexanderufer 3-7, 10117 Berlin > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > Amtsgericht Charlottenburg, HRB 209739 B > > On Wed, May 22, 2024, 17:06 Chia-Ping Tsai wrote: > > > > One issue I also noted is that some of the existing Decoder > > implementations (StringDecoder for example) can accept configurations > > but currently DumpLogSegments does not provide a way to pass any > > configurations, it creates an empty VerifiableProperties object each > > time it instantiates a Decoder instance. If we were to use > > Deserializer we would also need a way to provide configurations. > > > > BTW, if the known bug gets fixed, we have to make new interface extend > > `configurable`. > > > > Or we can just ignore the known issue as `DumpLogSegments` has no options > > to take custom configs for `Decoder`. That allow the `Decoder` more > simple > > > > > > Chia-Ping Tsai 於 2024年5月22日 週三 下午10:58寫道： > > > > > > > > Thanks for Mickael response! > > > > > > >I'm wondering whether we need to introduce a new Decoder interface and > > > instead if we could reuse Deserializer. We could deprecate the > > > key-decoder-class and value-decoder-class flags and introduce new > > > flags like key-deserializer-class and value-deserializer-class. One > > > benefit is that we already have many existing deserializer > > > implementations. WDYT? > > > > > > I prefer to use different interface, since using the same interface > > > (Deserializer) may obstruct us from enhancing the interface used by > > > DumpLogSegments only in the future. > > > > > > > One issue I also noted is that some of the existing Decoder > > > implementations (StringDecoder for example) can accept configurations > > > but currently DumpLogSegments does not provide a way to pass any > > > configurations, it creates an empty VerifiableProperties object each > > > time it instantiates a Decoder instance. If we were to use > > > Deserializer we would also need a way to provide configurations. > > > > > > yep, that is a known issue: > > > https://issues.apache.org/jira/browse/KAFKA-12311 > > > > > > We will file PR to fix it > > > > > > Mickael Maison 於 2024年5月22日 週三 下午10:51寫道： > > > > > >> Hi, > > >> > > >> Thanks for the KIP. Sorting this out in 3.8.0 would be really nice as > > >> it would allow us to migrate this tool in 4.0.0. We're unfortunately > > >> past the KIP deadline but maybe this is small enough to have an > > >> exception. > > >> > > >> I'm wondering whether we need to introduce a new Decoder interface and > > >> instead if we could reuse Deserializer. We could deprecate the > > >> key-decoder-class and value-decoder-class flags and introduce new > > >> flags like key-deserializer-class and value-deserializer-class. One > > >> benefit is that we already have many existing deserializer > > >> implementations. WDYT? > > >> > > >> One issue I also noted is that some of the existing Decoder > > >> implementations (StringDecoder for example) can accept configurations > > >> but currently DumpLogSegments does not provide a way to pass any > > >> configurations, it creates an empty VerifiableProperties object each > > >> time it instantiates a Decoder instance. If we were to use > > >> Deserializer we would also need a way to provide configurations. > > >> > > >> Thanks, > > >> Mickael > > >> > > >> On Wed, May 22, 2024 at 4:12 PM Chia-Ping Tsai > > >> wrote: > > >> > > > >> > Dear all, > > >> > > > >> > We know that 3.8.0 KIP is already frozen, but this is a small KIP > and > > >> we need to ship it to 3.8.0 so as to remove the deprecated scala > > interface > > >> from 4.0. > > >> > > > >> > Best, > > >> > Chia-Ping > > >> > > > >> > On 2024/05/22 14:05:16 Frank Yang wrote: > > >> > > Hi team, > > >> > > > > >> > > Chia-Ping Tsai and I would like to propose KIP-1047 to migrate > > >> kafka.serializer.Decoder from core module (scala) to tools module > > (java). > > >> > > > > >> > > Feedback and comments are welcome. > > >> > > > > >> > > KIP-1047: > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1047+Introduce+new+org.apache.kafka.tools.api.Decoder+to+replace+kafka.serializer.Decoder > > >> > > JIRA: https://issues.apache.org/jira/browse/KAFKA-16796 > > >> > > > > >> > > Thank you. > > >> > > PoAn > > >> > > > > > >
Re: [DISCUSS] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder
> One issue I also noted is that some of the existing Decoder implementations (StringDecoder for example) can accept configurations but currently DumpLogSegments does not provide a way to pass any configurations, it creates an empty VerifiableProperties object each time it instantiates a Decoder instance. If we were to use Deserializer we would also need a way to provide configurations. BTW, if the known bug gets fixed, we have to make new interface extend `configurable`. Or we can just ignore the known issue as `DumpLogSegments` has no options to take custom configs for `Decoder`. That allow the `Decoder` more simple Chia-Ping Tsai 於 2024年5月22日 週三 下午10:58寫道： > > Thanks for Mickael response! > > >I'm wondering whether we need to introduce a new Decoder interface and > instead if we could reuse Deserializer. We could deprecate the > key-decoder-class and value-decoder-class flags and introduce new > flags like key-deserializer-class and value-deserializer-class. One > benefit is that we already have many existing deserializer > implementations. WDYT? > > I prefer to use different interface, since using the same interface > (Deserializer) may obstruct us from enhancing the interface used by > DumpLogSegments only in the future. > > > One issue I also noted is that some of the existing Decoder > implementations (StringDecoder for example) can accept configurations > but currently DumpLogSegments does not provide a way to pass any > configurations, it creates an empty VerifiableProperties object each > time it instantiates a Decoder instance. If we were to use > Deserializer we would also need a way to provide configurations. > > yep, that is a known issue: > https://issues.apache.org/jira/browse/KAFKA-12311 > > We will file PR to fix it > > Mickael Maison 於 2024年5月22日 週三 下午10:51寫道： > >> Hi, >> >> Thanks for the KIP. Sorting this out in 3.8.0 would be really nice as >> it would allow us to migrate this tool in 4.0.0. We're unfortunately >> past the KIP deadline but maybe this is small enough to have an >> exception. >> >> I'm wondering whether we need to introduce a new Decoder interface and >> instead if we could reuse Deserializer. We could deprecate the >> key-decoder-class and value-decoder-class flags and introduce new >> flags like key-deserializer-class and value-deserializer-class. One >> benefit is that we already have many existing deserializer >> implementations. WDYT? >> >> One issue I also noted is that some of the existing Decoder >> implementations (StringDecoder for example) can accept configurations >> but currently DumpLogSegments does not provide a way to pass any >> configurations, it creates an empty VerifiableProperties object each >> time it instantiates a Decoder instance. If we were to use >> Deserializer we would also need a way to provide configurations. >> >> Thanks, >> Mickael >> >> On Wed, May 22, 2024 at 4:12 PM Chia-Ping Tsai >> wrote: >> > >> > Dear all, >> > >> > We know that 3.8.0 KIP is already frozen, but this is a small KIP and >> we need to ship it to 3.8.0 so as to remove the deprecated scala interface >> from 4.0. >> > >> > Best, >> > Chia-Ping >> > >> > On 2024/05/22 14:05:16 Frank Yang wrote: >> > > Hi team, >> > > >> > > Chia-Ping Tsai and I would like to propose KIP-1047 to migrate >> kafka.serializer.Decoder from core module (scala) to tools module (java). >> > > >> > > Feedback and comments are welcome. >> > > >> > > KIP-1047: >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1047+Introduce+new+org.apache.kafka.tools.api.Decoder+to+replace+kafka.serializer.Decoder >> > > JIRA: https://issues.apache.org/jira/browse/KAFKA-16796 >> > > >> > > Thank you. >> > > PoAn >> >
Re: [DISCUSS] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder
Thanks for Mickael response! >I'm wondering whether we need to introduce a new Decoder interface and instead if we could reuse Deserializer. We could deprecate the key-decoder-class and value-decoder-class flags and introduce new flags like key-deserializer-class and value-deserializer-class. One benefit is that we already have many existing deserializer implementations. WDYT? I prefer to use different interface, since using the same interface (Deserializer) may obstruct us from enhancing the interface used by DumpLogSegments only in the future. > One issue I also noted is that some of the existing Decoder implementations (StringDecoder for example) can accept configurations but currently DumpLogSegments does not provide a way to pass any configurations, it creates an empty VerifiableProperties object each time it instantiates a Decoder instance. If we were to use Deserializer we would also need a way to provide configurations. yep, that is a known issue: https://issues.apache.org/jira/browse/KAFKA-12311 We will file PR to fix it Mickael Maison 於 2024年5月22日 週三 下午10:51寫道： > Hi, > > Thanks for the KIP. Sorting this out in 3.8.0 would be really nice as > it would allow us to migrate this tool in 4.0.0. We're unfortunately > past the KIP deadline but maybe this is small enough to have an > exception. > > I'm wondering whether we need to introduce a new Decoder interface and > instead if we could reuse Deserializer. We could deprecate the > key-decoder-class and value-decoder-class flags and introduce new > flags like key-deserializer-class and value-deserializer-class. One > benefit is that we already have many existing deserializer > implementations. WDYT? > > One issue I also noted is that some of the existing Decoder > implementations (StringDecoder for example) can accept configurations > but currently DumpLogSegments does not provide a way to pass any > configurations, it creates an empty VerifiableProperties object each > time it instantiates a Decoder instance. If we were to use > Deserializer we would also need a way to provide configurations. > > Thanks, > Mickael > > On Wed, May 22, 2024 at 4:12 PM Chia-Ping Tsai > wrote: > > > > Dear all, > > > > We know that 3.8.0 KIP is already frozen, but this is a small KIP and > we need to ship it to 3.8.0 so as to remove the deprecated scala interface > from 4.0. > > > > Best, > > Chia-Ping > > > > On 2024/05/22 14:05:16 Frank Yang wrote: > > > Hi team, > > > > > > Chia-Ping Tsai and I would like to propose KIP-1047 to migrate > kafka.serializer.Decoder from core module (scala) to tools module (java). > > > > > > Feedback and comments are welcome. > > > > > > KIP-1047: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1047+Introduce+new+org.apache.kafka.tools.api.Decoder+to+replace+kafka.serializer.Decoder > > > JIRA: https://issues.apache.org/jira/browse/KAFKA-16796 > > > > > > Thank you. > > > PoAn >
Re: [DISCUSS] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder
Dear all, We know that 3.8.0 KIP is already frozen, but this is a small KIP and we need to ship it to 3.8.0 so as to remove the deprecated scala interface from 4.0. Best, Chia-Ping On 2024/05/22 14:05:16 Frank Yang wrote: > Hi team, > > Chia-Ping Tsai and I would like to propose KIP-1047 to migrate > kafka.serializer.Decoder from core module (scala) to tools module (java). > > Feedback and comments are welcome. > > KIP-1047: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1047+Introduce+new+org.apache.kafka.tools.api.Decoder+to+replace+kafka.serializer.Decoder > JIRA: https://issues.apache.org/jira/browse/KAFKA-16796 > > Thank you. > PoAn
[jira] [Created] (KAFKA-16813) Add global timeout for "@Test" and "@TestTemplate"
Chia-Ping Tsai created KAFKA-16813:
--
Summary: Add global timeout for "@Test" and "@TestTemplate"
Key: KAFKA-16813
URL: https://issues.apache.org/jira/browse/KAFKA-16813
Project: Kafka
Issue Type: Improvement
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
in code base `@Test` is used by unit test and `@TestTemplate` is used by
integration test. The later includes `ParameterizedTest`, `ClusterTest`,
`ClusterTests`, and `ClusterTemplate`. Hence, we can add two different timeout
for `@Test` and `@TestTemplate`. For example:
junit.jupiter.execution.timeout.default = 30s
junit.jupiter.execution.timeout.testtemplate.method.default = 120s
The accurate timeout value may need more discussion, but we can try it in small
junit5 module first. For example: tools module and storage module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16812) The tools-related tests are slow
Chia-Ping Tsai created KAFKA-16812: -- Summary: The tools-related tests are slow Key: KAFKA-16812 URL: https://issues.apache.org/jira/browse/KAFKA-16812 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai see https://ci-builds.apache.org/job/Kafka/job/kafka/job/trunk/2923/testReport/org.apache.kafka.tools/ Maybe we run too many cluster types (5), and we can remove some unrelated types for those tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra
[ https://issues.apache.org/jira/browse/KAFKA-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16784. Fix Version/s: 3.8.0 Resolution: Fixed > Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new > test infra > - > > Key: KAFKA-16784 > URL: https://issues.apache.org/jira/browse/KAFKA-16784 > Project: Kafka > Issue Type: Test > Reporter: Chia-Ping Tsai >Assignee: PoAn Yang >Priority: Minor > Labels: storage_test > Fix For: 3.8.0 > > > as title -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16654) Refactor kafka.test.annotation.Type and ClusterTestExtensions
[ https://issues.apache.org/jira/browse/KAFKA-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16654. Fix Version/s: 3.8.0 Resolution: Fixed > Refactor kafka.test.annotation.Type and ClusterTestExtensions > - > > Key: KAFKA-16654 > URL: https://issues.apache.org/jira/browse/KAFKA-16654 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: TaiJuWu >Priority: Minor > Fix For: 3.8.0 > > > It seems to me the refactor could include following tasks. > 1. change `invocationContexts`, method invoked by `ClusterTemplate`, and > generate-related methods in `ClusterTestExtensions` to return a > java.util.Collection instead of accepting a `java.util.function.Consumer`. > That can brings two benefit. 1) more simple in production: we don't need to > create a List and then pass it to be a function to collect stuff. 2) more > easy to write unit test. > 2. separate `provideTestTemplateInvocationContexts` to multi methods to > handle each annotation. That can help us to write tests, and make core more > readable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1031: Control offset translation in MirrorSourceConnector
> Which SourceRecord are you referring to here? I have added comments to your PR ( https://github.com/apache/kafka/pull/15999#pullrequestreview-2066823538) in short, `sourcePartition` and `sourceOffset` are unused if emit.offset-syncs.enabled=false BTW, I'm +1 to this KIP, and I noticed my previous comments are related to code. Hence, please feel free to open votes. We can have discussion about the code later. Omnia Ibrahim 於 2024年5月21日 週二 下午9:10寫道： > Hi Chia-Ping > > It seems we can disable the sync of idle consumers by setting > `sync.group.offsets.interval.seconds` to -1, so the fail-fast should > include sync.group.offsets.interval.seconds too. For another, maybe we > should do fail-fast for MirrorCheckpointConnector even though we don't have > this KIP > I don’t think we need to fail fast with > `sync.group.offsets.interval.seconds` to -1, as `MirrorCheckpointConnector` > runs two functionality based on offset-syncs topic that can run separately > 1. Write group offset to checkpoints internal topic can be disabled with > `emit.checkpoints.interval.seconds` -1 > 2. Schedule syncing the group offset to __consumer_offsets later can be > disabled with `sync.group.offsets.interval.seconds` to -1 > > So technically `MirrorCheckpointConnector` can run if only one of these > intervals is set to -1 however, if we want to fail fast we should check > both `sync.group.offsets.interval.seconds` and > `emit.checkpoints.interval.seconds` not set to -1 as this would be useless. > > > > 2) Should we do similar fail-fast for MirrorSourceConnector if user set > custom producer configs with emit.offset-syncs.enabled=false? I assume the > producer which sending records to offset-syncs topic won't be created if > emit.offset-syncs.enabled=false > This is a good point I’ll update MirrorSourceConnector’s validate method > to address this. I think we should also address > `offset-syncs.topic.location` and `offset-syncs.topic.replication.factor` > as well as custom consumer, and admin client configs. > > > > 3) Should we simplify the SourceRecord if > emit.offset-syncs.enabled=false? Maybe that can get a bit performance > improvement. > Which SourceRecord are you referring to here? > > Omnia > > On 20 May 2024, at 16:58, Chia-Ping Tsai wrote: > > > > Nice KIP. some minor comments/questions are listed below. > > > > 1) It seems we can disable the sync of idle consumers by setting > `sync.group.offsets.interval.seconds` to -1, so the fail-fast should > include sync.group.offsets.interval.seconds too. For another, maybe we > should do fail-fast for MirrorCheckpointConnector even though we don't have > this KIP > > > > 2) Should we do similar fail-fast for MirrorSourceConnector if user set > custom producer configs with emit.offset-syncs.enabled=false? I assume the > producer which sending records to offset-syncs topic won't be created if > emit.offset-syncs.enabled=false > > > > 3) Should we simplify the SourceRecord if > emit.offset-syncs.enabled=false? Maybe that can get a bit performance > improvement. > > > > Best, > > Chia-Ping > > > > On 2024/04/08 10:03:50 Omnia Ibrahim wrote: > >> Hi Chris, > >> Validation method is a good call. I updated the KIP to state that the > checkpoint connector will fail if the configs aren’t correct. And updated > the description of the new config to explain the impact of it on checkpoint > connector as well. > >> > >> If there is no any other feedback from anyone I would like to start the > voting thread in few days. > >> Thanks > >> Omnia > >> > >>> On 8 Apr 2024, at 06:31, Chris Egerton > wrote: > >>> > >>> Hi Omnia, > >>> > >>> Ah, good catch. I think failing to start the checkpoint connector if > offset > >>> syncs are disabled is fine. We'd probably want to do that via the > >>> Connector::validate [1] method in order to be able to catch invalid > configs > >>> during preflight validation, but it's not necessary to get that > specific in > >>> the KIP (especially since we may add other checks as well). > >>> > >>> [1] - > >>> > https://kafka.apache.org/37/javadoc/org/apache/kafka/connect/connector/Connector.html#validate(java.util.Map) > >>> > >>> Cheers, > >>> > >>> Chris > >>> > >>> On Thu, Apr 4, 2024 at 8:07 PM Omnia Ibrahim > >>> wrote: > >>> > >>>> Thanks Chris for the feedback > >>>>> 1. It'd be nice to mention that increasing the max offset lag to > INT_MAX > >>>
Re: DescribeLogDirs in Kafka 3.3.1 returns all topics instead of one provided in the request. Bug or "bad user error"?
Dear all,
I file https://issues.apache.org/jira/browse/KAFKA-16807 to fix it.
Thanks to Maxim for this nice finding. Also, thanks to Gaurav for the quick
response/dig-in
Cheers,
Chia-Ping
Gaurav Narula 於 2024年5月21日 週二 下午2:56寫道：
> Hello Maxim,
>
> Thanks for sharing this.
>
> I had a look and it seems like the behaviour on the wire changed with
> KAFKA-9435. I believe this change [0] in ReplicaManager causes all topics
> in
> online log dirs to be a part of the response inadvertently. We do however
> set
> partition information only for the queried topic. I'd suggest creating a
> JIRA
> for this.
>
> As for an option to specify ALL_PARTITIONS, I reckon that would require a
> KIP
> since it would be a change to the public interface.
>
> [0]:
>
> https://github.com/apache/kafka/pull/7972/files#diff-78812e247ffeae6f8c49b1b22506434701b1e1bafe7f92ef8f8708059e292bf0R674
>
> Regards,
> Gaurav
>
> On Mon, May 20, 2024 at 10:35 PM Maxim Senin
> wrote:
>
> > Hello.
> >
> > I’m having a problem with Kafka protocol API.
> >
> > Requests:
> > DescribeLogDirs Request (Version: 0) => [topics]
> > topics => topic [partitions]
> > topic => STRING
> > partitions => INT32
> >
> > My request contains `[{topic: “blah”, partitions:
> > [0,1,2,3,4,5,6,7,8,9]}]`, but the result
> >
> > Responses:
> > DescribeLogDirs Response (Version: 0) => throttle_time_ms [results]
> > throttle_time_ms => INT32
> > results => error_code log_dir [topics]
> > error_code => INT16
> > log_dir => STRING
> > topics => name [partitions]
> > name => STRING
> > partitions => partition_index partition_size offset_lag
> is_future_key
> > partition_index => INT32
> > partition_size => INT64
> > offset_lag => INT64
> > is_future_key => BOOLEAN
> >
> >
> >
> > contains entries for *all* topics. My workaround had been to filter the
> > returned list by topic name to find the one I was requesting the data
> for,
> > but I don’t understand why it’s not limiting the results to just the
> topic
> > I requested in the first place.
> >
> > Also, I think there should be an option to just specify ALL_PARTITIONS
> > because that would save me from having to retrieve topic metadata from
> the
> > broker to count the number of partitions. Kafka server would probably
> have
> > means to do that more efficiently.
> >
> > Is this a bug or am I doing something wrong?
> >
> > Thanks,
> > Maxim
> >
> >
> >
> > COGILITY SOFTWARE CORPORATION LEGAL DISCLAIMER: The information in this
> > email is confidential and is intended solely for the addressee. Access to
> > this email by anyone else is unauthorized. If you are not the intended
> > recipient, any disclosure, copying, distribution or any action taken or
> > omitted to be taken in reliance on it, is prohibited and may be unlawful.
> >
>
[jira] [Created] (KAFKA-16807) DescribeLogDirsResponseData#results#topics have unexpected topics having empty partitions
Chia-Ping Tsai created KAFKA-16807: -- Summary: DescribeLogDirsResponseData#results#topics have unexpected topics having empty partitions Key: KAFKA-16807 URL: https://issues.apache.org/jira/browse/KAFKA-16807 Project: Kafka Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai ReplicaManager [0] could generate a response having unexpected topics which have empty partitions. The root cause is it always generate the topic collection even though they have no matched partitions. That is not a issue to Kafka clients, since we loop the "partitions" to fill all future responses [1]. Hence, those unexpected topics won't be existent in the final results. However, that could be a issue to the users who implement Kafka client based on Kafka protocol [2] [0] https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/ReplicaManager.scala#L1252 [1] https://github.com/apache/kafka/blob/b5a013e4564ad93026b9c61431e4573a39bec766/clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java#L3145 [2] https://lists.apache.org/thread/lp7ktmm17pbg7nqk7p4s904lcn3mrvhy -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16794) Can't open videos in streams documentation
[
https://issues.apache.org/jira/browse/KAFKA-16794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16794.
Fix Version/s: 3.8.0
Resolution: Fixed
> Can't open videos in streams documentation
> --
>
> Key: KAFKA-16794
> URL: https://issues.apache.org/jira/browse/KAFKA-16794
> Project: Kafka
> Issue Type: Bug
> Components: docs, streams
>Reporter: Kuan Po Tseng
>Assignee: 黃竣陽
>Priority: Minor
> Fix For: 3.8.0
>
> Attachments: IMG_4445.png, image.png
>
>
> Can't open videos in page [https://kafka.apache.org/documentation/streams/]
> Open console in chrome browser and it shows error message:
> {{Refused to frame 'https://www.youtube.com/' because it violates the
> following Content Security Policy directive: "frame-src 'self'".}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [DISCUSS] KIP-1031: Control offset translation in MirrorSourceConnector
Nice KIP. some minor comments/questions are listed below. 1) It seems we can disable the sync of idle consumers by setting `sync.group.offsets.interval.seconds` to -1, so the fail-fast should include sync.group.offsets.interval.seconds too. For another, maybe we should do fail-fast for MirrorCheckpointConnector even though we don't have this KIP 2) Should we do similar fail-fast for MirrorSourceConnector if user set custom producer configs with emit.offset-syncs.enabled=false? I assume the producer which sending records to offset-syncs topic won't be created if emit.offset-syncs.enabled=false 3) Should we simplify the SourceRecord if emit.offset-syncs.enabled=false? Maybe that can get a bit performance improvement. Best, Chia-Ping On 2024/04/08 10:03:50 Omnia Ibrahim wrote: > Hi Chris, > Validation method is a good call. I updated the KIP to state that the > checkpoint connector will fail if the configs aren’t correct. And updated the > description of the new config to explain the impact of it on checkpoint > connector as well. > > If there is no any other feedback from anyone I would like to start the > voting thread in few days. > Thanks > Omnia > > > On 8 Apr 2024, at 06:31, Chris Egerton wrote: > > > > Hi Omnia, > > > > Ah, good catch. I think failing to start the checkpoint connector if offset > > syncs are disabled is fine. We'd probably want to do that via the > > Connector::validate [1] method in order to be able to catch invalid configs > > during preflight validation, but it's not necessary to get that specific in > > the KIP (especially since we may add other checks as well). > > > > [1] - > > https://kafka.apache.org/37/javadoc/org/apache/kafka/connect/connector/Connector.html#validate(java.util.Map) > > > > Cheers, > > > > Chris > > > > On Thu, Apr 4, 2024 at 8:07 PM Omnia Ibrahim > > wrote: > > > >> Thanks Chris for the feedback > >>> 1. It'd be nice to mention that increasing the max offset lag to INT_MAX > >>> could work as a partial workaround for users on existing versions (though > >>> of course this wouldn't prevent creation of the syncs topic). > >> I updated the KIP > >> > >>> 2. Will it be illegal to disable offset syncs if other features that rely > >>> on offset syncs are explicitly enabled in the connector config? If > >> they're > >>> not explicitly enabled then it should probably be fine to silently > >> disable > >>> them, but I'd be interested in your thoughts. > >> The rest of the features that relays on this is controlled by > >> emit.checkpoints.enabled (enabled by default) and > >> sync.group.offsets.enabled (disabled by default) which are part of > >> MirrorCheckpointConnector config not MirrorSourceConnector, I was thinking > >> that MirrorCheckpointConnector should fail to start if > >> emit.offset-syncs.enabled is disabled while emit.checkpoints.enabled and/or > >> sync.group.offsets.enabled are enabled as no point of creating this > >> connector if the main part is disabled. WDYT? > >> > >> Thanks > >> Omnia > >> > >>> On 3 Apr 2024, at 12:45, Chris Egerton wrote: > >>> > >>> Hi Omnia, > >>> > >>> Thanks for the KIP! Two small things come to mind: > >>> > >>> 1. It'd be nice to mention that increasing the max offset lag to INT_MAX > >>> could work as a partial workaround for users on existing versions (though > >>> of course this wouldn't prevent creation of the syncs topic). > >>> > >>> 2. Will it be illegal to disable offset syncs if other features that rely > >>> on offset syncs are explicitly enabled in the connector config? If > >> they're > >>> not explicitly enabled then it should probably be fine to silently > >> disable > >>> them, but I'd be interested in your thoughts. > >>> > >>> Cheers, > >>> > >>> Chris > >>> > >>> On Wed, Apr 3, 2024, 20:41 Luke Chen wrote: > >>> > Hi Omnia, > > Thanks for the KIP! > It LGTM! > But I'm not an expert of MM2, it would be good to see if there is any > >> other > comment from MM2 experts. > > Thanks. > Luke > > On Thu, Mar 14, 2024 at 6:08 PM Omnia Ibrahim > wrote: > > > Hi everyone, I would like to start a discussion thread for KIP-1031: > > Control offset translation in MirrorSourceConnector > > > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1031%3A+Control+offset+translation+in+MirrorSourceConnector > > > > Thanks > > Omnia > > > > >> > >> > >
[jira] [Resolved] (KAFKA-16797) A bit cleanup of FeatureControlManager
[ https://issues.apache.org/jira/browse/KAFKA-16797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16797. Fix Version/s: 3.8.0 Resolution: Fixed > A bit cleanup of FeatureControlManager > -- > > Key: KAFKA-16797 > URL: https://issues.apache.org/jira/browse/KAFKA-16797 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: 黃竣陽 >Priority: Trivial > Fix For: 3.8.0 > > > [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java#L62] > > that can be replaced by `Collections.emptyIterator()` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16797) A bit cleanup of FeatureControlManager
Chia-Ping Tsai created KAFKA-16797: -- Summary: A bit cleanup of FeatureControlManager Key: KAFKA-16797 URL: https://issues.apache.org/jira/browse/KAFKA-16797 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java#L62] that can be replaced by `Collections.emptyIterator()` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16796) Introduce new org.apache.kafka.tools.api.MessageParser to replace kafka.tools.DumpLogSegments.MessageParser
Chia-Ping Tsai created KAFKA-16796: -- Summary: Introduce new org.apache.kafka.tools.api.MessageParser to replace kafka.tools.DumpLogSegments.MessageParser Key: KAFKA-16796 URL: https://issues.apache.org/jira/browse/KAFKA-16796 Project: Kafka Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai We need a replacement in order to complete https://issues.apache.org/jira/browse/KAFKA-14579 in kafak 4.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15723) KRaft support in ListOffsetsRequestTest
[
https://issues.apache.org/jira/browse/KAFKA-15723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-15723.
Fix Version/s: 3.8.0
Resolution: Fixed
> KRaft support in ListOffsetsRequestTest
> ---
>
> Key: KAFKA-15723
> URL: https://issues.apache.org/jira/browse/KAFKA-15723
> Project: Kafka
> Issue Type: Task
> Components: core
>Reporter: Sameer Tejani
>Assignee: Mickael Maison
>Priority: Minor
> Labels: kraft, kraft-test, newbie
> Fix For: 3.8.0
>
>
> The following tests in ListOffsetsRequestTest in
> core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala need to be
> updated to support KRaft
> 37 : def testListOffsetsErrorCodes(): Unit = {
> 84 : def testListOffsetsMaxTimeStampOldestVersion(): Unit = {
> 112 : def testCurrentEpochValidation(): Unit = {
> 173 : def testResponseIncludesLeaderEpoch(): Unit = {
> 210 : def testResponseDefaultOffsetAndLeaderEpochForAllVersions(): Unit = {
> Scanned 261 lines. Found 0 KRaft tests out of 5 tests
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-16544) DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE
[
https://issues.apache.org/jira/browse/KAFKA-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16544.
Fix Version/s: 3.8
Resolution: Fixed
> DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames
> should return null instead of throwing NPE
> --
>
> Key: KAFKA-16544
> URL: https://issues.apache.org/jira/browse/KAFKA-16544
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
>Assignee: Kuan Po Tseng
>Priority: Major
> Fix For: 3.8
>
>
> {code:java}
> * @return A future map from topic names to descriptions which can be
> used to check
> * the status of individual description if the describe topic
> request used
> * topic names, otherwise return null, this request succeeds only
> if all the
> * topic descriptions succeed
> {code}
> According the docs, it should return null if we try to get the result
> unmatched to the request. For example, we call `allTopicNames` in passing
> `TopicIdCollection`. However, the current implementation will throw NPE
> directly
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16791) Add thread detection to ClusterTestExtensions
Chia-Ping Tsai created KAFKA-16791: -- Summary: Add thread detection to ClusterTestExtensions Key: KAFKA-16791 URL: https://issues.apache.org/jira/browse/KAFKA-16791 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai `ClusterTestExtensions` should implement `BeforeAllCallback` and `AfterAllCallback` by `TestUtils.verifyNoUnexpectedThreads` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16763) Upgrade to scala 2.12.19 and scala 2.13.14
[ https://issues.apache.org/jira/browse/KAFKA-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16763. Fix Version/s: 3.8.0 Resolution: Fixed > Upgrade to scala 2.12.19 and scala 2.13.14 > -- > > Key: KAFKA-16763 > URL: https://issues.apache.org/jira/browse/KAFKA-16763 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: 黃竣陽 >Priority: Minor > Fix For: 3.8.0 > > > scala 2.12.19 (https://github.com/scala/scala/releases/tag/v2.12.19) > > scala 2.13.14 (https://github.com/scala/scala/releases/tag/v2.13.14) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16785) Migrate TopicBasedRemoteLogMetadataManagerRestartTest to new test infra
Chia-Ping Tsai created KAFKA-16785: -- Summary: Migrate TopicBasedRemoteLogMetadataManagerRestartTest to new test infra Key: KAFKA-16785 URL: https://issues.apache.org/jira/browse/KAFKA-16785 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai as title -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra
Chia-Ping Tsai created KAFKA-16784: -- Summary: Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra Key: KAFKA-16784 URL: https://issues.apache.org/jira/browse/KAFKA-16784 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai as title -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16783) Migrate RemoteLogMetadataManagerTest to new test infra
Chia-Ping Tsai created KAFKA-16783: -- Summary: Migrate RemoteLogMetadataManagerTest to new test infra Key: KAFKA-16783 URL: https://issues.apache.org/jira/browse/KAFKA-16783 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai as title `TopicBasedRemoteLogMetadataManagerWrapperWithHarness` could be replaced by `RemoteLogMetadataManagerTestUtils#builder` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16668) Enable to set tags by `ClusterTest`
[ https://issues.apache.org/jira/browse/KAFKA-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16668. Fix Version/s: 3.8.0 Resolution: Fixed > Enable to set tags by `ClusterTest` > > > Key: KAFKA-16668 > URL: https://issues.apache.org/jira/browse/KAFKA-16668 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: Johnny Hsu >Priority: Minor > Fix For: 3.8.0 > > > Currently, the display name can be customized by only `name` > (https://github.com/apache/kafka/blob/trunk/core/src/test/java/kafka/test/annotation/ClusterTest.java#L42). > However, the "key" is hard-code to "name=xxx". Also, it is impossible to set > more "tags" for display name. > https://github.com/apache/kafka/pull/15766 is a example that we want to add > "xxx=bbb" to display name. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16539) Can't update specific broker configs in pre-migration mode
[ https://issues.apache.org/jira/browse/KAFKA-16539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16539. Resolution: Fixed > Can't update specific broker configs in pre-migration mode > -- > > Key: KAFKA-16539 > URL: https://issues.apache.org/jira/browse/KAFKA-16539 > Project: Kafka > Issue Type: Bug > Components: config, kraft >Affects Versions: 3.6.0, 3.7.0, 3.6.1, 3.6.2 >Reporter: David Arthur >Assignee: David Arthur >Priority: Major > Fix For: 3.8.0, 3.7.1 > > > In migration mode, ZK brokers will have a forwarding manager configured. This > is used to forward requests to the KRaft controller once we get to that part > of the migration. However, prior to KRaft taking over as the controller > (known as pre-migration mode), the ZK brokers are still attempting to forward > IncrementalAlterConfigs to the controller. > This works fine for cluster level configs (e.g., "-entity-type broker > --entity-default"), but this fails for specific broker configs (e.g., > "-entity-type broker --entity-id 1"). > This affects BROKER and BROKER_LOGGER config types. > To workaround this bug, you can either disable migrations on the brokers > (assuming no migration has taken place), or proceed with the migration and > get to the point where KRaft is the controller. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] KIP-950: Tiered Storage Disablement
+1 but I prefer to ship it to KRaft only. I do concern that community have enough time to accept more feature in 3.8 :( Best, Chia-Ping On 2024/05/14 15:20:50 Christo Lolov wrote: > Heya! > > I would like to start a vote on KIP-950: Tiered Storage Disablement in > order to catch the last Kafka release targeting Zookeeper - > https://cwiki.apache.org/confluence/display/KAFKA/KIP-950%3A++Tiered+Storage+Disablement > > Best, > Christo >
Re: [DISCUSS] Apache Kafka 3.7.1 release
+1 Our production will be upgraded to your 3.7.1 regardless of how many paper works I have to handle :) Lucas Brutschy 於 2024年5月16日 週四 下午3:25寫道： > Hi Igor, > > thanks a lot! > +1 > > Lucas > > On Thu, May 16, 2024 at 5:21 AM Luke Chen wrote: > > > > Hi Igor, > > > > Thanks for volunteering! > > +1 > > > > Luke > > > > On Wed, May 15, 2024 at 11:15 PM Mickael Maison < > mickael.mai...@gmail.com> > > wrote: > > > > > Hi Igor, > > > > > > Thanks for volunteering, +1 > > > > > > Mickael > > > > > > On Thu, Apr 25, 2024 at 11:09 AM Igor Soarez wrote: > > > > > > > > Hi everyone, > > > > > > > > I'd like to volunteer to be the release manager for a 3.7.1 release. > > > > > > > > Please keep in mind, this would be my first release, so I might have > > > some questions, > > > > and it might also take me a bit longer to work through the release > > > process. > > > > So I'm thinking a good target would be toward the end of May. > > > > > > > > Please let me know your thoughts and if there are any objections or > > > concerns. > > > > > > > > Thanks, > > > > > > > > -- > > > > Igor > > > >
[jira] [Created] (KAFKA-16775) Fix flaky PlaintextAdminIntegrationTest#testCreateExistingTopicsThrowTopicExistsException
Chia-Ping Tsai created KAFKA-16775: -- Summary: Fix flaky PlaintextAdminIntegrationTest#testCreateExistingTopicsThrowTopicExistsException Key: KAFKA-16775 URL: https://issues.apache.org/jira/browse/KAFKA-16775 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai org.opentest4j.AssertionFailedError: timed out waiting for topics at app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38) at app//org.junit.jupiter.api.Assertions.fail(Assertions.java:138) at app//kafka.api.BaseAdminIntegrationTest.waitForTopics(BaseAdminIntegrationTest.scala:236) at app//kafka.api.PlaintextAdminIntegrationTest.testCreateExistingTopicsThrowTopicExistsException(PlaintextAdminIntegrationTest.scala:140) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled
Chia-Ping Tsai created KAFKA-16774: -- Summary: fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled Key: KAFKA-16774 URL: https://issues.apache.org/jira/browse/KAFKA-16774 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai java.util.ConcurrentModificationException at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1720) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) at org.apache.kafka.streams.processor.internals.TaskManager.allTasks(TaskManager.java:1686) at org.apache.kafka.streams.processor.internals.TaskManager.releaseLockedUnassignedTaskDirectories(TaskManager.java:1364) at org.apache.kafka.streams.processor.internals.TaskManager.handleRebalanceComplete(TaskManager.java:208) at org.apache.kafka.streams.processor.internals.StreamsRebalanceListener.onPartitionsAssigned(StreamsRebalanceListener.java:79) at org.apache.kafka.streams.processor.internals.StreamThreadTest.shouldCloseAllTaskProducersOnCloseIfEosEnabled(StreamThreadTest.java:1408) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16773) Fix flaky QuorumControllerTest#testDelayedConfigurationOperations
Chia-Ping Tsai created KAFKA-16773: -- Summary: Fix flaky QuorumControllerTest#testDelayedConfigurationOperations Key: KAFKA-16773 URL: https://issues.apache.org/jira/browse/KAFKA-16773 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai org.apache.kafka.server.fault.FaultHandlerException: fatalFaultHandler: exception while renouncing leadership: Attempt to resign from epoch 1 which is larger than the current epoch 0 at org.apache.kafka.metalog.LocalLogManager.resign(LocalLogManager.java:788) at org.apache.kafka.controller.QuorumController.renounce(QuorumController.java:1267) at org.apache.kafka.controller.QuorumController.handleEventException(QuorumController.java:546) at org.apache.kafka.controller.QuorumController.access$800(QuorumController.java:179) at org.apache.kafka.controller.QuorumController$ControllerWriteEvent.complete(QuorumController.java:878) at org.apache.kafka.controller.QuorumController$ControllerWriteEvent.handleException(QuorumController.java:868) at org.apache.kafka.queue.KafkaEventQueue$EventContext.completeWithException(KafkaEventQueue.java:149) at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:138) at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:211) at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:182) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.IllegalArgumentException: Attempt to resign from epoch 1 which is larger than the current epoch 0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16671) Revisit SessionedProtocolIntegrationTest.ensureInternalEndpointIsSecured
[ https://issues.apache.org/jira/browse/KAFKA-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16671. Fix Version/s: 3.8.0 Resolution: Fixed > Revisit SessionedProtocolIntegrationTest.ensureInternalEndpointIsSecured > > > Key: KAFKA-16671 > URL: https://issues.apache.org/jira/browse/KAFKA-16671 > Project: Kafka > Issue Type: Test > Reporter: Chia-Ping Tsai >Assignee: PoAn Yang >Priority: Minor > Fix For: 3.8.0 > > > loop 1000times on my local, and all pass. Let's enable the test to see what > happens in our CI -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16763) Upgrade to scala 2.12.19 and scala 2.13.14
Chia-Ping Tsai created KAFKA-16763: -- Summary: Upgrade to scala 2.12.19 and scala 2.13.14 Key: KAFKA-16763 URL: https://issues.apache.org/jira/browse/KAFKA-16763 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai scala 2.12.19 (https://github.com/scala/scala/releases/tag/v2.12.19) scala 2.13.14 (https://github.com/scala/scala/releases/tag/v2.13.14) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16761) ZkClusterInstance#start does not create ClusterConfigurableIntegrationHarness
Chia-Ping Tsai created KAFKA-16761: -- Summary: ZkClusterInstance#start does not create ClusterConfigurableIntegrationHarness Key: KAFKA-16761 URL: https://issues.apache.org/jira/browse/KAFKA-16761 Project: Kafka Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai [https://github.com/apache/kafka/blob/440f5f6c09720bb9414524781342bbf35973c281/core/src/test/java/kafka/test/junit/ZkClusterInvocationContext.java#L103] `ClusterConfigurableIntegrationHarness` is created only in "beforeEach" phase, and that makes `ZkClusterInstance#start` does not work as it could cause NPE. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16677) Replace ClusterType#ALL and ClusterType#DEFAULT by Array
[
https://issues.apache.org/jira/browse/KAFKA-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16677.
Fix Version/s: 3.8.0
Resolution: Fixed
> Replace ClusterType#ALL and ClusterType#DEFAULT by Array
>
>
> Key: KAFKA-16677
> URL: https://issues.apache.org/jira/browse/KAFKA-16677
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
>Assignee: PoAn Yang
>Priority: Minor
> Fix For: 3.8.0
>
>
> Both ClusterType#ALL and ClusterType#DEFAULT are a kind of "tag" instead of
> true "type". It seems to me they can be removed by using Array. For example:
> ClusterType#ALL -> {Type.ZK, Type.KRAFT, Type.CO_KRAFT}
> ClusterType#DEFAULT -> {}
> There are two benefits
> 1. That is more readable for "ALL type".
> 2. We don't throw the awkward "exception" when seeing "DEFAULT".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16705) the flag "started" of RaftClusterInstance is false even though the cluster is started
Chia-Ping Tsai created KAFKA-16705: -- Summary: the flag "started" of RaftClusterInstance is false even though the cluster is started Key: KAFKA-16705 URL: https://issues.apache.org/jira/browse/KAFKA-16705 Project: Kafka Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai we should set `started` to true after https://github.com/apache/kafka/blob/643db430a707479c9e87eec1ad67e1d4f43c9268/core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java#L113 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16679) Merge unit test down to the class of integration test
[ https://issues.apache.org/jira/browse/KAFKA-16679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16679. Fix Version/s: 3.8.0 Resolution: Fixed > Merge unit test down to the class of integration test > - > > Key: KAFKA-16679 > URL: https://issues.apache.org/jira/browse/KAFKA-16679 > Project: Kafka > Issue Type: Test > Reporter: Chia-Ping Tsai >Assignee: Cheng-Kai, Zhang >Priority: Minor > Fix For: 3.8.0 > > > Normally, we don't put multi test classes into single file. Those test > classes can be extracted into a new class file. Or we can merge them into > single class by using "@Test" annotation. That can make those test cases run > without embedded cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16704) Fix flaky kafka.log.remote.RemoteIndexCacheTest#testIndexFileAlreadyExistOnDiskButNotInCache
Chia-Ping Tsai created KAFKA-16704: -- Summary: Fix flaky kafka.log.remote.RemoteIndexCacheTest#testIndexFileAlreadyExistOnDiskButNotInCache Key: KAFKA-16704 URL: https://issues.apache.org/jira/browse/KAFKA-16704 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai java.io.UncheckedIOException: java.nio.file.NoSuchFileException: /tmp/kafka-RemoteIndexCacheTest3690189103734187552/R68MBnutRfmqJY66XXFoOA:foo-0/remote-log-index-cache/2147584984_Ma8JCqucS7mqKIHfSSDeow.txnindex.deleted at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87) at java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at kafka.log.remote.RemoteIndexCacheTest.renameRemoteCacheIndexFileFromDisk$1(RemoteIndexCacheTest.scala:832) at kafka.log.remote.RemoteIndexCacheTest.testIndexFileAlreadyExistOnDiskButNotInCache(RemoteIndexCacheTest.scala:851) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16698) Fix flaky kafka.network.DynamicConnectionQuotaTest#testDynamicIpConnectionRateQuota
Chia-Ping Tsai created KAFKA-16698:
--
Summary: Fix flaky
kafka.network.DynamicConnectionQuotaTest#testDynamicIpConnectionRateQuota
Key: KAFKA-16698
URL: https://issues.apache.org/jira/browse/KAFKA-16698
Project: Kafka
Issue Type: Test
Reporter: Chia-Ping Tsai
{code:java}
org.opentest4j.AssertionFailedError: Timed out waiting for connection rate
update to propagate at
app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38) at
app//org.junit.jupiter.api.Assertions.fail(Assertions.java:138) at
app//kafka.network.DynamicConnectionQuotaTest.updateIpConnectionRate(DynamicConnectionQuotaTest.scala:279)
at
app//kafka.network.DynamicConnectionQuotaTest.testDynamicIpConnectionRateQuota(DynamicConnectionQuotaTest.scala:255)
at
java.base@21.0.2/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base@21.0.2/java.lang.reflect.Method.invoke(Method.java:580)at
app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at
app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16697) Fix flaky kafka.api.SaslGssapiSslEndToEndAuthorizationTest#testProduceConsumeWithWildcardAcls
Chia-Ping Tsai created KAFKA-16697:
--
Summary: Fix flaky
kafka.api.SaslGssapiSslEndToEndAuthorizationTest#testProduceConsumeWithWildcardAcls
Key: KAFKA-16697
URL: https://issues.apache.org/jira/browse/KAFKA-16697
Project: Kafka
Issue Type: Test
Reporter: Chia-Ping Tsai
{code:java}
org.opentest4j.AssertionFailedError: Should have been zero expired connections
killed: 1(total=0.0) ==> expected: <0> but was: <1> at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at
app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) at
app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166) at
app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:664) at
app//kafka.api.EndToEndAuthorizationTest.$anonfun$confirmReauthenticationMetrics$1(EndToEndAuthorizationTest.scala:202)
at
app//kafka.api.EndToEndAuthorizationTest.$anonfun$confirmReauthenticationMetrics$1$adapted(EndToEndAuthorizationTest.scala:200)
at app//scala.collection.IterableOnceOps.foreach(IterableOnce.scala:576)
at app//scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:574)
at app//scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at
app//kafka.api.EndToEndAuthorizationTest.confirmReauthenticationMetrics(EndToEndAuthorizationTest.scala:200)
at
app//kafka.api.EndToEndAuthorizationTest.testProduceConsumeWithWildcardAcls(EndToEndAuthorizationTest.scala:236)
at
java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)at
java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568)
at
app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at
app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChai
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-16660) reduce the check interval to speedup DelegationTokenRequestsTest
[ https://issues.apache.org/jira/browse/KAFKA-16660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16660. Fix Version/s: 3.8.0 Resolution: Fixed > reduce the check interval to speedup DelegationTokenRequestsTest > > > Key: KAFKA-16660 > URL: https://issues.apache.org/jira/browse/KAFKA-16660 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: Kuan Po Tseng >Priority: Minor > Fix For: 3.8.0 > > > the check interval is 1 minute > (https://github.com/apache/kafka/blob/trunk/core/src/test/scala/unit/kafka/server/DelegationTokenRequestsTest.scala#L49), > and `DelegationTokenRequestsTest` waits 2 minutes before running the check > (https://github.com/apache/kafka/blob/trunk/core/src/test/scala/unit/kafka/server/DelegationTokenRequestsTest.scala#L159) > ... > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-12947) Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ...
[ https://issues.apache.org/jira/browse/KAFKA-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12947. Resolution: Duplicate this is fixed by https://github.com/apache/kafka/pull/14623 > Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ... > -- > > Key: KAFKA-12947 > URL: https://issues.apache.org/jira/browse/KAFKA-12947 > Project: Kafka > Issue Type: Sub-task >Reporter: YI-CHEN WANG >Assignee: Dalibor Plavcic >Priority: Major > > For Kafka-7438 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16484) Support to define per broker/controller property by ClusterConfigProperty
[ https://issues.apache.org/jira/browse/KAFKA-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16484. Fix Version/s: 3.8.0 Resolution: Fixed > Support to define per broker/controller property by ClusterConfigProperty > - > > Key: KAFKA-16484 > URL: https://issues.apache.org/jira/browse/KAFKA-16484 > Project: Kafka > Issue Type: Test > Reporter: Chia-Ping Tsai >Assignee: Kuan Po Tseng >Priority: Major > Fix For: 3.8.0 > > > the property set to `ClusterConfigProperty` gets applied to all brokers, and > hence we can't have individual props for each broker to test racks. > > It seems to me we can add new field "id" to `ClusterConfigProperty` to > declare the property should be applied to specific broker (or controller). > the default value is -1 and it should be applied to all nodes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16640) Replace TestUtils#resource by scala.util.Using
[ https://issues.apache.org/jira/browse/KAFKA-16640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16640. Fix Version/s: 3.8.0 Resolution: Fixed > Replace TestUtils#resource by scala.util.Using > -- > > Key: KAFKA-16640 > URL: https://issues.apache.org/jira/browse/KAFKA-16640 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: TengYao Chi >Priority: Minor > Fix For: 3.8.0 > > > `scala.util.Using` is in both scala 2.13 and scala-collection-compat so we > don't need to have custom try-resource function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16678) Remove unimplementedquorum from EndToEndAuthorizationTest
[ https://issues.apache.org/jira/browse/KAFKA-16678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16678. Fix Version/s: 3.8.0 Resolution: Fixed > Remove unimplementedquorum from EndToEndAuthorizationTest > - > > Key: KAFKA-16678 > URL: https://issues.apache.org/jira/browse/KAFKA-16678 > Project: Kafka > Issue Type: Test > Reporter: Chia-Ping Tsai >Assignee: TengYao Chi >Priority: Minor > Fix For: 3.8.0 > > > `unimplementedquorum`[0] is used to skip test cases if they don't support to > run by kraft. However, KAFKA-15219 , KAFKA-14765 and KAFKA-14776 make related > tests support to run by kraft. > In short, it is time to remove the unused variable :) > [0] > [https://github.com/apache/kafka/blob/d76352e2151178521dc447e3406dabb8fcd4c57c/core/src/test/scala/integration/kafka/api/EndToEndAuthorizationTest.scala#L146] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16689) Move LogValidatorTest to storage module
Chia-Ping Tsai created KAFKA-16689: -- Summary: Move LogValidatorTest to storage module Key: KAFKA-16689 URL: https://issues.apache.org/jira/browse/KAFKA-16689 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai `LogValidator` is moved to storage module already but its unit test is still in core module. That is a bit weird. We ought to rewrite it by java and then move it to storage module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16684) FetchResponse#responseData could return incorrect data
Chia-Ping Tsai created KAFKA-16684: -- Summary: FetchResponse#responseData could return incorrect data Key: KAFKA-16684 URL: https://issues.apache.org/jira/browse/KAFKA-16684 Project: Kafka Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai [https://github.com/apache/kafka/commit/2b8aff58b575c199ee8372e5689420c9d77357a5] make it accept input to return "partial" data. The content of output is based on the input but we cache the output ... It will return same output even though we pass different input. That is a potential bug. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16683) Extract security-related helpers from scala.TestUtils to java class
Chia-Ping Tsai created KAFKA-16683: -- Summary: Extract security-related helpers from scala.TestUtils to java class Key: KAFKA-16683 URL: https://issues.apache.org/jira/browse/KAFKA-16683 Project: Kafka Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai We can merge them into `JaasTestUtils and then rename `JaasTestUtils` to `SecurityTestUtils. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16682) Rewrite JassTestUtils by Java
Chia-Ping Tsai created KAFKA-16682: -- Summary: Rewrite JassTestUtils by Java Key: KAFKA-16682 URL: https://issues.apache.org/jira/browse/KAFKA-16682 Project: Kafka Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai as title one more thing is that we should change the package name from kafka.utils to kafka.security -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16681) Rewrite MiniKDC by Java
Chia-Ping Tsai created KAFKA-16681: -- Summary: Rewrite MiniKDC by Java Key: KAFKA-16681 URL: https://issues.apache.org/jira/browse/KAFKA-16681 Project: Kafka Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Noted: # we need to move it from scala folder to java folder # don't change the package name since system tests requires it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16680) Make ClusterTestExtensions support SASL
Chia-Ping Tsai created KAFKA-16680: -- Summary: Make ClusterTestExtensions support SASL Key: KAFKA-16680 URL: https://issues.apache.org/jira/browse/KAFKA-16680 Project: Kafka Issue Type: New Feature Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai This is a umbrella issue. In order to migrate more tests to new test infra, we ought to make it support SASL at least. *phase1: reuse/rewrite existent SASL utils by Java* # MiniKdc # JaasTestUtils # Move security-related helpers from scala.TestUtils to java.TestUtils # extract/rewrite non-zk code from SaslSetup to new java class *phase2: make `ClusterTest#securityProtocol` works. It does not work for kraft mode :(* # add client-related helper to generate consumer/producer/admin class with security configs # configure kraft server with security settings # migrate tests of tools to use new test infra with security -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16679) Merge `DeleteRecordsCommandUnitTest` into `DeleteRecordsCommandTest`, `FeatureCommandUnitTest` into
Chia-Ping Tsai created KAFKA-16679: -- Summary: Merge `DeleteRecordsCommandUnitTest` into `DeleteRecordsCommandTest`, `FeatureCommandUnitTest` into Key: KAFKA-16679 URL: https://issues.apache.org/jira/browse/KAFKA-16679 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Normally, we don't put multi test classes into single file. Those test classes can be extracted into a new class file. Or we can merge them into single class by using "@Test" annotation. That can make those test cases run without embedded cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16678) Remove unimplementedquorum from EndToEndAuthorizationTest
Chia-Ping Tsai created KAFKA-16678: -- Summary: Remove unimplementedquorum from EndToEndAuthorizationTest Key: KAFKA-16678 URL: https://issues.apache.org/jira/browse/KAFKA-16678 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai `unimplementedquorum`[0] is used to skip test cases if they don't support to run by kraft. However, KAFKA-15219 , KAFKA-14765 and KAFKA-14776 make related tests support to run by kraft. In short, it is time to remove the unused variable :) [0] [https://github.com/apache/kafka/blob/d76352e2151178521dc447e3406dabb8fcd4c57c/core/src/test/scala/integration/kafka/api/EndToEndAuthorizationTest.scala#L146] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16593) Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions
[ https://issues.apache.org/jira/browse/KAFKA-16593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16593. Fix Version/s: 3.8.0 Resolution: Fixed > Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions > -- > > Key: KAFKA-16593 > URL: https://issues.apache.org/jira/browse/KAFKA-16593 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: TengYao Chi >Priority: Minor > Fix For: 3.8.0 > > > as title. the test is in tools module and it does not need to depend on test > code of core module directly -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16677) Replace ClusterType#ALL and ClusterType#DEFAULT by Array
Chia-Ping Tsai created KAFKA-16677:
--
Summary: Replace ClusterType#ALL and ClusterType#DEFAULT by Array
Key: KAFKA-16677
URL: https://issues.apache.org/jira/browse/KAFKA-16677
Project: Kafka
Issue Type: Improvement
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Both ClusterType#ALL and ClusterType#DEFAULT are a kind of "tag" instead of
true "type". It seems to me they can be removed by using Array. For example:
ClusterType#ALL -> {Type.ZK, Type.KRAFT, Type.CO_KRAFT}
ClusterType#DEFAULT -> {}
There are two benefits
1. That is more readable for "ALL type".
2. We don't throw the awkward "exception" when seeing "DEFAULT".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-16470) kafka-dump-log --offsets-decoder should support new records
[ https://issues.apache.org/jira/browse/KAFKA-16470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16470. Fix Version/s: 3.8.0 Resolution: Fixed > kafka-dump-log --offsets-decoder should support new records > --- > > Key: KAFKA-16470 > URL: https://issues.apache.org/jira/browse/KAFKA-16470 > Project: Kafka > Issue Type: Sub-task >Reporter: David Jacot >Assignee: David Jacot >Priority: Major > Fix For: 3.8.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16608) AsyncKafkaConsumer doesn't honor interrupted thread status on KafkaConsumer.poll(Duration)
[ https://issues.apache.org/jira/browse/KAFKA-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16608. Fix Version/s: 3.8.0 Resolution: Fixed > AsyncKafkaConsumer doesn't honor interrupted thread status on > KafkaConsumer.poll(Duration) > -- > > Key: KAFKA-16608 > URL: https://issues.apache.org/jira/browse/KAFKA-16608 > Project: Kafka > Issue Type: Bug > Components: clients, consumer >Affects Versions: 3.8.0 >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Minor > Fix For: 3.8.0 > > > The behaviour for KafkaConsumer.poll(Duration) when the calling thread is in > interrupted state is to throw InterruptException. The AsyncKafkaConsumer > doesn't do this. It only throws that exception if the interruption occurs > while it is waiting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16393) SslTransportLayer doesn't implement write(ByteBuffer[], int, int) correctly
[
https://issues.apache.org/jira/browse/KAFKA-16393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16393.
Fix Version/s: 3.8.0
Resolution: Fixed
> SslTransportLayer doesn't implement write(ByteBuffer[], int, int) correctly
> ---
>
> Key: KAFKA-16393
> URL: https://issues.apache.org/jira/browse/KAFKA-16393
> Project: Kafka
> Issue Type: Improvement
>Reporter: Haruki Okada
>Assignee: Haruki Okada
>Priority: Minor
> Fix For: 3.8.0
>
>
> As of Kafka 3.7.0, SslTransportLayer.write(ByteBuffer[], int, int) is
> implemented like below:
> {code:java}
> public long write(ByteBuffer[] srcs, int offset, int length) throws
> IOException {
> ...
> int i = offset;
> while (i < length) {
> if (srcs[i].hasRemaining() || hasPendingWrites()) {
>
> {code}
> The loop index starts at `offset` and ends with `length`.
> However this isn't correct because end-index should be `offset + length`.
> Let's say we have the array of ByteBuffer with length = 5 and try calling
> this method with offset = 3, length = 1.
> In current code, `write(srcs, 3, 1)` doesn't attempt any write because the
> loop condition is immediately false.
> For now, seems this method is only called with args offset = 0, length =
> srcs.length in Kafka code base so not causing any problem though, we should
> fix this because this could introduce subtle bug if use this method with
> different args in the future.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16672) Fix flaky DedicatedMirrorIntegrationTest.testMultiNodeCluster
Chia-Ping Tsai created KAFKA-16672: -- Summary: Fix flaky DedicatedMirrorIntegrationTest.testMultiNodeCluster Key: KAFKA-16672 URL: https://issues.apache.org/jira/browse/KAFKA-16672 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai It is flaky on my jenkins, and sometimes it fails in Kafka [CI|https://ge.apache.org/scans/tests?search.buildOutcome=failure=gradle=P28D=kafka=Asia%2FTaipei=org.apache.kafka.connect.mirror.integration.DedicatedMirrorIntegrationTest=testMultiNodeCluster()] The error happens in virtue of race condition. `KafkaBasedLog` loads records from topic via thread, so `RebalanceNeededException` will be thrown if we check the task configs too soon. It seems to me `RebalanceNeededException` is a temporary exception so we should treat it as a retryable exception in waiting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16671) Revisit SessionedProtocolIntegrationTest.ensureInternalEndpointIsSecured
Chia-Ping Tsai created KAFKA-16671: -- Summary: Revisit SessionedProtocolIntegrationTest.ensureInternalEndpointIsSecured Key: KAFKA-16671 URL: https://issues.apache.org/jira/browse/KAFKA-16671 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai loop 1000times on my local, and all pass. Let's enable the test to see what happens in our CI -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16659) KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER
[
https://issues.apache.org/jira/browse/KAFKA-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16659.
Fix Version/s: 3.8.0
Resolution: Fixed
> KafkaConsumer#position() does not respect wakup when group protocol is
> CONSUMER
> ---
>
> Key: KAFKA-16659
> URL: https://issues.apache.org/jira/browse/KAFKA-16659
> Project: Kafka
> Issue Type: Bug
> Reporter: Chia-Ping Tsai
>Assignee: PoAn Yang
>Priority: Minor
> Fix For: 3.8.0
>
>
> see following test
> {code:scala}
> @ParameterizedTest(name =
> TestInfoUtils.TestWithParameterizedQuorumAndGroupProtocolNames)
> @MethodSource(Array("getTestQuorumAndGroupProtocolParametersAll"))
> @Timeout(15)
> def testPositionRespectsWakeup(quorum: String, groupProtocol: String): Unit
> = {
> val topicPartition = new TopicPartition(topic, 15)
> val consumer = createConsumer()
> consumer.assign(List(topicPartition).asJava)
> CompletableFuture.runAsync { () =>
> TimeUnit.SECONDS.sleep(1)
> consumer.wakeup()
> }
> assertThrows(classOf[WakeupException], () =>
> consumer.position(topicPartition, Duration.ofSeconds(3)))
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16669) Remove extra collection copy when genrating DescribeAclsResource
Chia-Ping Tsai created KAFKA-16669: -- Summary: Remove extra collection copy when genrating DescribeAclsResource Key: KAFKA-16669 URL: https://issues.apache.org/jira/browse/KAFKA-16669 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai There are three collection copy happening in generating DescribeAclsResource 1. Iterable -> HashSet (https://github.com/apache/kafka/blob/25118cec145b1a70a7b1709ca4a7ac367f066c6c/core/src/main/scala/kafka/server/AclApis.scala#L72) 2. HashSet -> Map (https://github.com/apache/kafka/blob/25118cec145b1a70a7b1709ca4a7ac367f066c6c/clients/src/main/java/org/apache/kafka/common/requests/DescribeAclsResponse.java#L141) 3. Map -> List (https://github.com/apache/kafka/blob/25118cec145b1a70a7b1709ca4a7ac367f066c6c/clients/src/main/java/org/apache/kafka/common/requests/DescribeAclsResponse.java#L146) We can do two small optimization: 1. remove the first collection copy. This optimization needs two steps: a) change `aclsResources` input type from `Collection` to `Iterable`. b) de-duplicate in second collection copy: HashSet -> Map. We use `Set` to replace the `List` 2. set the array size. https://github.com/apache/kafka/blob/25118cec145b1a70a7b1709ca4a7ac367f066c6c/clients/src/main/java/org/apache/kafka/common/requests/DescribeAclsResponse.java#L148 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16668) Enable to set tags by `ClusterTest`
Chia-Ping Tsai created KAFKA-16668: -- Summary: Enable to set tags by `ClusterTest` Key: KAFKA-16668 URL: https://issues.apache.org/jira/browse/KAFKA-16668 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Currently, the display name can be customized by only `name` (https://github.com/apache/kafka/blob/trunk/core/src/test/java/kafka/test/annotation/ClusterTest.java#L42). However, the "key" is hard-code to "name=xxx". Also, it is impossible to set more "tags" for display name. https://github.com/apache/kafka/pull/15766 is a example that we want to add "xxx=bbb" to display name. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16666) Remove unused class `GroupMetadataMessageFormatter` and `OffsetsMessageFormatter`
Chia-Ping Tsai created KAFKA-1: -- Summary: Remove unused class `GroupMetadataMessageFormatter` and `OffsetsMessageFormatter` Key: KAFKA-1 URL: https://issues.apache.org/jira/browse/KAFKA-1 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai After https://github.com/apache/kafka/pull/15652, both `GroupMetadataMessageFormatter`[0] and `OffsetsMessageFormatter`[1] get unused. We should remove them. [0] https://github.com/apache/kafka/blob/376e9e20dbf7c7aeb6f6f666d47932c445eb6bd1/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L1269 [1] https://github.com/apache/kafka/blob/376e9e20dbf7c7aeb6f6f666d47932c445eb6bd1/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L1248 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16660) reduce the check interval to speedup DelegationTokenRequestsTest
Chia-Ping Tsai created KAFKA-16660: -- Summary: reduce the check interval to speedup DelegationTokenRequestsTest Key: KAFKA-16660 URL: https://issues.apache.org/jira/browse/KAFKA-16660 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai the check interval is 1 minute (https://github.com/apache/kafka/blob/trunk/core/src/test/scala/unit/kafka/server/DelegationTokenRequestsTest.scala#L49), and `DelegationTokenRequestsTest` wait 2 minutes before running the check (https://github.com/apache/kafka/blob/trunk/core/src/test/scala/unit/kafka/server/DelegationTokenRequestsTest.scala#L159) ... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16572) allow defining number of disks per broker in ClusterTest
[ https://issues.apache.org/jira/browse/KAFKA-16572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16572. Fix Version/s: 3.8.0 Resolution: Fixed > allow defining number of disks per broker in ClusterTest > > > Key: KAFKA-16572 > URL: https://issues.apache.org/jira/browse/KAFKA-16572 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: PoAn Yang >Priority: Minor > Fix For: 3.8.0 > > > This is a follow-up of KAFKA-16559 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16659) KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER
Chia-Ping Tsai created KAFKA-16659:
--
Summary: KafkaConsumer#position() does not respect wakup when
group protocol is CONSUMER
Key: KAFKA-16659
URL: https://issues.apache.org/jira/browse/KAFKA-16659
Project: Kafka
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
see following test
{code:scala}
@ParameterizedTest(name =
TestInfoUtils.TestWithParameterizedQuorumAndGroupProtocolNames)
@MethodSource(Array("getTestQuorumAndGroupProtocolParametersAll"))
@Timeout(15)
def testPositionRespectsWakeup(quorum: String, groupProtocol: String): Unit =
{
val topicPartition = new TopicPartition(topic, 15)
val consumer = createConsumer()
consumer.assign(List(topicPartition).asJava)
CompletableFuture.runAsync { () =>
TimeUnit.SECONDS.sleep(1)
consumer.wakeup()
}
assertThrows(classOf[WakeupException], () =>
consumer.position(topicPartition, Duration.ofSeconds(3)))
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-10804) Tune travis system tests to avoid timeouts
[ https://issues.apache.org/jira/browse/KAFKA-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10804. Resolution: Won't Fix > Tune travis system tests to avoid timeouts > -- > > Key: KAFKA-10804 > URL: https://issues.apache.org/jira/browse/KAFKA-10804 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson > Assignee: Chia-Ping Tsai >Priority: Major > > Thanks to https://github.com/apache/kafka/pull/9652, we are now running > system tests for PRs. However, it looks like we need some tuning because many > of the subsets are timing out. For example: > https://travis-ci.com/github/apache/kafka/jobs/453241933. This might just be > a matter of adding more subsets or changing the timeout, but we should > probably also consider whether we want to run all system tests or if there is > a more useful subset of them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10809) Make all system tests stably run on either Travis CI or local
[ https://issues.apache.org/jira/browse/KAFKA-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10809. Resolution: Fixed > Make all system tests stably run on either Travis CI or local > - > > Key: KAFKA-10809 > URL: https://issues.apache.org/jira/browse/KAFKA-10809 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai > Assignee: Chia-Ping Tsai >Priority: Major > > It can encourage us to write more system tests if Travis CI can run a subset > of system tests automatically. > This initial work is https://github.com/apache/kafka/pull/9652 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10290) fix flaky core/compatibility_test_new_broker_test.py
[
https://issues.apache.org/jira/browse/KAFKA-10290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-10290.
Resolution: Won't Fix
> fix flaky core/compatibility_test_new_broker_test.py
>
>
> Key: KAFKA-10290
> URL: https://issues.apache.org/jira/browse/KAFKA-10290
> Project: Kafka
> Issue Type: Sub-task
> Components: core, system tests
> Reporter: Chia-Ping Tsai
>Priority: Major
>
> {quote}
> Module: kafkatest.tests.core.compatibility_test_new_broker_test
> Class: ClientCompatibilityTestNewBroker
> Method: test_compatibility
> Arguments:
> {
> "compression_types": [
> "none"
> ],
> "consumer_version": "1.0.2",
> "producer_version": "1.0.2",
> "timestamp_type": "CreateTime"
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-14874) Unable to create > 5000 topics for once when using Kraft
[ https://issues.apache.org/jira/browse/KAFKA-14874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-14874. Resolution: Won't Fix > Unable to create > 5000 topics for once when using Kraft > > > Key: KAFKA-14874 > URL: https://issues.apache.org/jira/browse/KAFKA-14874 > Project: Kafka > Issue Type: Bug > Reporter: Chia-Ping Tsai > Assignee: Chia-Ping Tsai >Priority: Minor > > the error happens due to > [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/QuorumController.java#L779] > I encountered this error when creating >5000 topics for mirroring the cluster > from zk to Kraft. The operation of creating a bunch of topics is allowed by > zk-based kafka. > It seems to me there are two improvements for this issue. > 1) add more precise error message for such case. > 2) make `maxRecordsPerBatch` configurable (there is already a setter > [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/QuorumController.java#L272]) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10805) More useful reporting from travis system tests
[ https://issues.apache.org/jira/browse/KAFKA-10805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10805. Resolution: Won't Fix > More useful reporting from travis system tests > -- > > Key: KAFKA-10805 > URL: https://issues.apache.org/jira/browse/KAFKA-10805 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Priority: Major > > Inspecting system test output from travis is a bit painful at the moment > because you have to check the build logs to find the tests that failed. > Additionally, there is no logging available from the workers which is often > essential to debug a failure. We should look into how we can improve the > build so that the output is more convenient and useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10288) fix flaky client/client_compatibility_features_test.py
[
https://issues.apache.org/jira/browse/KAFKA-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-10288.
Resolution: Won't Fix
> fix flaky client/client_compatibility_features_test.py
> --
>
> Key: KAFKA-10288
> URL: https://issues.apache.org/jira/browse/KAFKA-10288
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, system tests
> Reporter: Chia-Ping Tsai
>Priority: Major
>
> {quote}
> Module: kafkatest.tests.client.client_compatibility_features_test
> Class: ClientCompatibilityFeaturesTest
> Method: run_compatibility_test
> Arguments:
> {
> "broker_version": "0.10.0.1"
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-10291) fix flaky tools/log4j_appender_test.py
[
https://issues.apache.org/jira/browse/KAFKA-10291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-10291.
Resolution: Won't Fix
> fix flaky tools/log4j_appender_test.py
> --
>
> Key: KAFKA-10291
> URL: https://issues.apache.org/jira/browse/KAFKA-10291
> Project: Kafka
> Issue Type: Sub-task
> Components: system tests, tools
> Reporter: Chia-Ping Tsai
>Priority: Major
>
> {quote}
> Module: kafkatest.tests.tools.log4j_appender_test
> Class: Log4jAppenderTest
> Method: test_log4j_appender
> Arguments:
> {
> "security_protocol": "SSL"
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-10822) Force some stdout from system tests for Travis
[ https://issues.apache.org/jira/browse/KAFKA-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10822. Resolution: Won't Fix > Force some stdout from system tests for Travis > -- > > Key: KAFKA-10822 > URL: https://issues.apache.org/jira/browse/KAFKA-10822 > Project: Kafka > Issue Type: Sub-task > Reporter: Chia-Ping Tsai > Assignee: Chia-Ping Tsai >Priority: Minor > > downgrade_test.py/upgrade_test.py does upgrade/downgrade for each tests. the > upgrade/downgrade tasks take 10+ mins in Travis env so we ought to print > something in order to avoid timeout caused by Travis. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10696) Replace ProduceResponse.PartitionResponse by auto-generated PartitionProduceResponse
[ https://issues.apache.org/jira/browse/KAFKA-10696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10696. Resolution: Won't Fix > Replace ProduceResponse.PartitionResponse by auto-generated > PartitionProduceResponse > > > Key: KAFKA-10696 > URL: https://issues.apache.org/jira/browse/KAFKA-10696 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai > Assignee: Chia-Ping Tsai >Priority: Minor > > This is a follow-up of KAFKA-9628. > related discussion: > https://github.com/apache/kafka/pull/9401#discussion_r518976605 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-7014) Guarantee the byte buffer returned by Serializer is reusable
[ https://issues.apache.org/jira/browse/KAFKA-7014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-7014. --- Resolution: Won't Fix > Guarantee the byte buffer returned by Serializer is reusable > > > Key: KAFKA-7014 > URL: https://issues.apache.org/jira/browse/KAFKA-7014 > Project: Kafka > Issue Type: Bug > Reporter: Chia-Ping Tsai >Priority: Minor > > The byte array returned by Serializer is copied to another ByteBuffer by > KafkaProducer so the byte array is reusable actually. If kafka can guarantee > this behavior (perhaps doc it on the Serializer), user can design the > size-fixed message and then reuse the byte array. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-6992) Configuring the batch.size to zero won't disable the batching entirely
[
https://issues.apache.org/jira/browse/KAFKA-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-6992.
---
Resolution: Won't Fix
> Configuring the batch.size to zero won't disable the batching entirely
> --
>
> Key: KAFKA-6992
> URL: https://issues.apache.org/jira/browse/KAFKA-6992
> Project: Kafka
> Issue Type: Bug
> Components: documentation
> Reporter: Chia-Ping Tsai
>Priority: Minor
>
> In MAGIC_VALUE_V2, the varint is used and the estimated size won't be equal
> with the actual size. So it has chance to accept more records in a single
> produce batch.
> Perhaps we should revise the docs of batch.size.
> {code:java}
> public static final String BATCH_SIZE_CONFIG = "batch.size";
> private static final String BATCH_SIZE_DOC = "The producer will attempt to
> batch records together into fewer requests whenever multiple records are
> being sent"
> + " to the same partition. This
> helps performance on both the client and the server. This configuration
> controls the "
> + "default batch size in bytes. "
> + ""
> + "No attempt will be made to
> batch records larger than this size. "
> + ""
> + "Requests sent to brokers will
> contain multiple batches, one for each partition with data available to be
> sent. "
> + ""
> + "A small batch size will make
> batching less common and may reduce throughput (a batch size of zero will
> disable "
> + "batching entirely). A very
> large batch size may use memory a bit more wastefully as we will always
> allocate a "
> + "buffer of the specified batch
> size in anticipation of additional records.";{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-16647) Remove setMetadataDirectory from BrokerNode/ControllerNode
[ https://issues.apache.org/jira/browse/KAFKA-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16647. Fix Version/s: 3.8.0 Resolution: Fixed > Remove setMetadataDirectory from BrokerNode/ControllerNode > -- > > Key: KAFKA-16647 > URL: https://issues.apache.org/jira/browse/KAFKA-16647 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: Kuan Po Tseng >Priority: Minor > Fix For: 3.8.0 > > > `TestKitNodes` does not enable callers to define the location of "base > folder". That makes sense to me since callers should not care for it. That > means the location of metadata folder shoud be transparent to callers. Hence, > the setter of metadata folder is useless. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16614) Disallow `@ClusterTemplate("")`
[
https://issues.apache.org/jira/browse/KAFKA-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-16614.
Fix Version/s: 3.8.0
Resolution: Fixed
> Disallow `@ClusterTemplate("")`
> ---
>
> Key: KAFKA-16614
> URL: https://issues.apache.org/jira/browse/KAFKA-16614
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
>Assignee: TaiJuWu
>Priority: Minor
> Fix For: 3.8.0
>
>
> `@ClusterTemplate` enable us to create dynamic configs, and it expect to
> accept a method name which can create server configs at runtime. It throws
> error when we pass a nonexistent method name, but it works if we pass an
> empty name
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16654) Refactor kafka.test.annotation.Type and ClusterTestExtensions
Chia-Ping Tsai created KAFKA-16654: -- Summary: Refactor kafka.test.annotation.Type and ClusterTestExtensions Key: KAFKA-16654 URL: https://issues.apache.org/jira/browse/KAFKA-16654 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai It seems to me the refactor could include following tasks. 1. change `invocationContexts`, method invoked by `ClusterTemplate`, and generate-related methods in `ClusterTestExtensions` to return a java.util.Collection instead of accepting a `java.util.function.Consumer`. That can brings two benefit. 1) more simple in production: we don't need to create a List and then pass it to be a function to collect stuff. 2) more easy to write unit test. 2. separate `provideTestTemplateInvocationContexts` to multi methods to handle each annotation. That can help us to write tests, and make core more readable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16588) broker shutdown hangs when `log.segment.delete.delay.ms` is zero
[ https://issues.apache.org/jira/browse/KAFKA-16588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16588. Fix Version/s: 3.8.0 Resolution: Fixed > broker shutdown hangs when `log.segment.delete.delay.ms` is zero > - > > Key: KAFKA-16588 > URL: https://issues.apache.org/jira/browse/KAFKA-16588 > Project: Kafka > Issue Type: Bug > Reporter: Chia-Ping Tsai >Assignee: PoAn Yang >Priority: Minor > Fix For: 3.8.0 > > > see > [https://github.com/apache/kafka/blob/f22ad6645bfec0b38e820e0090261c9f6b421a74/core/src/main/scala/kafka/log/LogManager.scala#L1154] > If `log.segment.delete.delay.ms` is zero, We call `take` even though the > `logsToBeDeleted` is empty, and `KafkaScheduler#shutdown` call `shutdown` > rather than `shudownNow` > ([https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/util/KafkaScheduler.java#L134)] > Hence, the thread won't be completed forever, and it blocks the shutdown of > broker. > We should replace the `take` by `poll` since we have checked the element > before. > BTW, the zero is allowed > ([https://github.com/apache/kafka/blob/f22ad6645bfec0b38e820e0090261c9f6b421a74/storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java#L258]) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16652) add unit test for
Chia-Ping Tsai created KAFKA-16652: -- Summary: add unit test for Key: KAFKA-16652 URL: https://issues.apache.org/jira/browse/KAFKA-16652 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai https://github.com/apache/kafka/blob/31355ef8f948f369e240ebc203f889f187116d75/core/src/test/java/kafka/test/junit/ClusterTestExtensions.java#L94 If `ClusterTemplate`does not generate any `ClusterConfig`, we will throw exception. However, we don't have UT for such scenario currently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16650) add integration test for Admin#abortTransaction
Chia-Ping Tsai created KAFKA-16650: -- Summary: add integration test for Admin#abortTransaction Key: KAFKA-16650 URL: https://issues.apache.org/jira/browse/KAFKA-16650 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai It seems there are only few unit tests. We should add IT includeing zk, kraft, and new group coordinator for it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16647) Remove setMetadataDirectory from BrokerNode/ControllerNode
Chia-Ping Tsai created KAFKA-16647: -- Summary: Remove setMetadataDirectory from BrokerNode/ControllerNode Key: KAFKA-16647 URL: https://issues.apache.org/jira/browse/KAFKA-16647 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Kuan Po Tseng `TestKitNodes` does not enable callers to define the location of "base folder". That makes sense to me since callers should not care for it. That means the location of metadata folder shoud be transparent to callers. Hence, the setter of metadata folder is useless. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-10859) add @Test annotation to FileStreamSourceTaskTest.testInvalidFile and reduce the loop count to speedup the test
[ https://issues.apache.org/jira/browse/KAFKA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10859. Resolution: Duplicate fixed by https://github.com/apache/kafka/commit/3dacdc5694da5db283524889d2270695defebbaa > add @Test annotation to FileStreamSourceTaskTest.testInvalidFile and reduce > the loop count to speedup the test > -- > > Key: KAFKA-10859 > URL: https://issues.apache.org/jira/browse/KAFKA-10859 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: Tom Bentley >Priority: Major > Labels: newbie > > FileStreamSourceTaskTest.testInvalidFile miss a `@Test` annotation. Also, it > loops 100 times which spend about 2m to complete a unit test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16627) Remove ClusterConfig parameter in BeforeEach and AfterEach
[ https://issues.apache.org/jira/browse/KAFKA-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16627. Fix Version/s: 3.8.0 Resolution: Fixed > Remove ClusterConfig parameter in BeforeEach and AfterEach > -- > > Key: KAFKA-16627 > URL: https://issues.apache.org/jira/browse/KAFKA-16627 > Project: Kafka > Issue Type: Improvement >Reporter: Kuan Po Tseng >Assignee: Kuan Po Tseng >Priority: Minor > Fix For: 3.8.0 > > > In the past we modify configs like server broker properties by modifying the > ClusterConfig reference passed to BeforeEach and AfterEach based on the > requirements of the tests. > While after KAFKA-16560, the ClusterConfig become immutable, modify the > ClusterConfig reference no longer reflects any changes to the test cluster. > Then pass ClusterConfig to BeforeEach and AfterEach become redundant. We > should remove this behavior. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15897) Flaky Test: testWrongIncarnationId() – kafka.server.ControllerRegistrationManagerTest
[
https://issues.apache.org/jira/browse/KAFKA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-15897.
Fix Version/s: 3.8.0
Resolution: Fixed
> Flaky Test: testWrongIncarnationId() –
> kafka.server.ControllerRegistrationManagerTest
> -
>
> Key: KAFKA-15897
> URL: https://issues.apache.org/jira/browse/KAFKA-15897
> Project: Kafka
> Issue Type: Test
>Reporter: Apoorv Mittal
> Assignee: Chia-Ping Tsai
>Priority: Major
> Labels: flaky-test
> Fix For: 3.8.0
>
>
> Build run:
> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14699/21/tests/
>
> {code:java}
> org.opentest4j.AssertionFailedError: expected: <(false,1,0)> but was:
> <(true,0,0)>Stacktraceorg.opentest4j.AssertionFailedError: expected:
> <(false,1,0)> but was: <(true,0,0)>at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
>at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
>at
> app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
> at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
> at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1141)
> at
> app//kafka.server.ControllerRegistrationManagerTest.$anonfun$testWrongIncarnationId$3(ControllerRegistrationManagerTest.scala:228)
>at
> app//org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:379)
> at
> app//kafka.server.ControllerRegistrationManagerTest.testWrongIncarnationId(ControllerRegistrationManagerTest.scala:226)
> at
> java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)at
> java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568)
> at
> app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
> at
> app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> app//org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
>at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
>at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescripto
[jira] [Created] (KAFKA-16640) Replace TestUtils#resource by scala.util.Using
Chia-Ping Tsai created KAFKA-16640: -- Summary: Replace TestUtils#resource by scala.util.Using Key: KAFKA-16640 URL: https://issues.apache.org/jira/browse/KAFKA-16640 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai `scala.util.Using` is in both scala 2.13 and scala-collection-compat so we don't need to have custom try-resource function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16639) AsyncKafkaConsumer#close does not send heartbeat to leave group
Chia-Ping Tsai created KAFKA-16639: -- Summary: AsyncKafkaConsumer#close does not send heartbeat to leave group Key: KAFKA-16639 URL: https://issues.apache.org/jira/browse/KAFKA-16639 Project: Kafka Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai This bug can be reproduced by immediately closing a consumer which is just created. The root cause is that we skip the new heartbeat used to leave group when there is a in-flight heartbeat request ([https://github.com/apache/kafka/blob/5de5d967adffd864bad3ec729760a430253abf38/clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java#L212).] It seems to me the simple solution is that we create a heartbeat request when meeting above situation and then send it by pollOnClose ([https://github.com/apache/kafka/blob/5de5d967adffd864bad3ec729760a430253abf38/clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestManager.java#L62).] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16560) Refactor/cleanup BrokerNode/ControllerNode/ClusterConfig
[ https://issues.apache.org/jira/browse/KAFKA-16560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16560. Fix Version/s: 3.8.0 Resolution: Fixed > Refactor/cleanup BrokerNode/ControllerNode/ClusterConfig > > > Key: KAFKA-16560 > URL: https://issues.apache.org/jira/browse/KAFKA-16560 > Project: Kafka > Issue Type: Improvement > Reporter: Chia-Ping Tsai >Assignee: Kuan Po Tseng >Priority: Minor > Fix For: 3.8.0 > > > origin discussion: > https://github.com/apache/kafka/pull/15715#discussion_r1564660916 > It seems to me this jira should address following tasks. > 1. make them immutable. We have adopted the builder pattern, so all changes > should be completed in the builder phase > 2. make all `Builder#build()` not accept any arguments. Instead, we should > add new setters for those arguments. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-6527) Transient failure in DynamicBrokerReconfigurationTest.testDefaultTopicConfig
[
https://issues.apache.org/jira/browse/KAFKA-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-6527.
---
Resolution: Fixed
It is enabled by https://github.com/apache/kafka/pull/15796
> Transient failure in DynamicBrokerReconfigurationTest.testDefaultTopicConfig
>
>
> Key: KAFKA-6527
> URL: https://issues.apache.org/jira/browse/KAFKA-6527
> Project: Kafka
> Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: TaiJuWu
>Priority: Blocker
> Labels: flakey, flaky-test
> Fix For: 3.8.0
>
>
> {code:java}
> java.lang.AssertionError: Log segment size increase not applied
> at kafka.utils.TestUtils$.fail(TestUtils.scala:355)
> at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:865)
> at
> kafka.server.DynamicBrokerReconfigurationTest.testDefaultTopicConfig(DynamicBrokerReconfigurationTest.scala:348)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
