[jira] [Created] (KAFKA-16412) Uncreated topics is considered as created topics

[gendong1 (Jira)]

gendong1 created KAFKA-16412:


 Summary: Uncreated topics is considered as created topics
 Key: KAFKA-16412
 URL: https://issues.apache.org/jira/browse/KAFKA-16412
 Project: Kafka
  Issue Type: Bug
Affects Versions: 2.8.2
Reporter: gendong1


A client sends topic creation request to broker.

Another client sends the same topic creation request to broker.

The former request does not finish. However, the second client get 
TopicExistsException.

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

[Arpit Goyal]

Hi Team,
Any further comments or suggestions on the possible approaches discussed
above.

On Tue, Mar 19, 2024, 09:55 Arpit Goyal  wrote:

> @Luke Chen @Kamal Chandraprakash   @Greg
> Harris Any suggestion on the above two possible approaches.
> On Sun, Mar 17, 2024, 13:36 Arpit Goyal  wrote:
>
>>
  In summary , There are two possible solution to handle the above
>> scenario when producer snapshot file found to be null
>>
>> 1. *Generate empty producer snapshot file at run time when copying
>> LogSegment*
>>
>>
>>- This will not require any backward compatibility dependencies with
>>the plugin.
>>- It preserves the contract i.e producerSnapshot files should be
>>mandatory.
>>- We could have a metric which helps us to assess how many times
>>empty snapshot files have been created.
>>
>> 2.*  Make producerSnapshot files optional *
>>
>>- This would break the contract with the plugin and would require
>>defining a set of approaches to handle it which is mentioned earlier in 
>> the
>>thread.
>>- If we make producer Snapshot optional , We would   not be  handling
>>the error which @LukeChen mentioned when producerSnapshot
>>accidentally deleted a given segment. But this holds true for
>>TransactionalIndex.
>>- The other question is do we really need to make the field optional.
>>The only case where this problem can occur is only when the topic migrated
>>from < 2.8 version.
>>
>>

[jira] [Resolved] (KAFKA-16381) use volatile to guarantee KafkaMetric#config visibility across threads

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-16381.

Fix Version/s: 3.8.0
   Resolution: Fixed

> use volatile to guarantee KafkaMetric#config visibility across threads
> --
>
> Key: KAFKA-16381
> URL: https://issues.apache.org/jira/browse/KAFKA-16381
> Project: Kafka
>  Issue Type: Bug
>Reporter: Johnny Hsu
>Assignee: Johnny Hsu
>Priority: Minor
> Fix For: 3.8.0
>
>
> In KafkaMetirc.java, the getter is 
> ```
> @Override
> public MetricName metricName() {
> return this.metricName;
> }
> ```
> and there is a setter 
> ```
> public void config(MetricConfig config) {
>     synchronized (lock) {
>     this.config = config;
>   }
> }
> ```
> Since it's possible to set and get in the mean time, we should have lock in 
> the getter as well



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.6.2 RC1

[Chia-Ping Tsai]

hi Manikumar

https://issues.apache.org/jira/browse/KAFKA-16341 has been resolved!

thanks for your patience.

best,
Chia-Ping

Chia-Ping Tsai  於 2024年3月22日 週五 上午10:32寫道：

>
> hi Manikumar
>
> > Pls let me know after merging the PR. I will generate RC2 later today.
>
> Sure. We will complete it ASAP
>
>
> > Manikumar  於 2024年3月22日 上午9:26 寫道：
> >
> > Hi,
> >
> > Thanks. Since we have merged KAFKA-16342
> > , it's probably
> better
> > to take the PR for KAFKA-16341
> >  for consistency.
> > I am canceling the RC1 in favour of including KAFKA-16341
> > .
> >
> > Chia-Ping,
> >
> > Pls let me know after merging the PR. I will generate RC2 later today.
> >
> >
> > Thank you,
> >
> >
> >
> >
> >
> >
> > On Thu, Mar 21, 2024 at 9:10 PM Chia-Ping Tsai 
> wrote:
> >
> >>> Is this a regression from the 3.5.0 release?
> >>
> >> I believe the bug is existent for a while, but it is not a true issue
> until
> >> we allowed users to fetch offset of max timestamp.
> >>
> >>> Can we update the "Affects Version/s" field on JIRA?
> >>
> >> done. I attach the tags for active branches - 3.6.1 and 3.7.0
> >>
> >> Manikumar  於 2024年3月21日 週四 下午11:12寫道：
> >>
> >>> Hi Chia-Ping,
> >>>
> >>> Thanks for letting me know.
> >>>
> >>> Is this a regression from the 3.5.0 release?  Can we update the
> "Affects
> >>> Version/s" field on JIRA?
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> On Thu, Mar 21, 2024 at 5:06 PM Chia-Ping Tsai 
> >> wrote:
> >>>
>  hi Manikumar,
> 
>  There is a bug fix which needs to be backport to 3.6 (
>  https://issues.apache.org/jira/browse/KAFKA-16341)
> 
>  It fixes the incorrect offset of max timestamp in non-compress path.
> >> The
>  other paths are already fixed by
>  https://issues.apache.org/jira/browse/KAFKA-16342.
> 
>  Personally, we should backport both fixes for all paths, and we can
>  complete the backport today.
> 
>  Sorry for bring this news to RC1.
> 
>  Best,
>  Chia-Ping
> 
> 
>  Manikumar  於 2024年3月21日 週四 下午6:11寫道：
> 
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 3.6.2.
> >
> > This is a bugfix release with several fixes, including dependency
> > version bumps for CVEs.
> >
> > Release notes for the 3.6.2 release:
> >
> >> https://home.apache.org/~manikumar/kafka-3.6.2-rc1/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Tuesday, March 26th
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~manikumar/kafka-3.6.2-rc1/
> >
> >
> > * Maven artifacts to be voted upon:
> >
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~manikumar/kafka-3.6.2-rc1/javadoc/
> >
> > * Tag to be voted upon (off 3.6 branch) is the 3.6.2 tag:
> > https://github.com/apache/kafka/releases/tag/3.6.2-rc1
> >
> > * Documentation:
> > https://kafka.apache.org/36/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/36/protocol.html
> >
> > * Successful Jenkins builds for the 3.6 branch:
> > Unit/integration tests:
> >
> >
> 
> >>>
> >>
> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.6/159/tests
> > (with few flaky failures)
> > System tests: I will update system test results
> >
> > Thanks,
> > Manikumar
> >
> 
> >>>
> >>
>

[jira] [Resolved] (KAFKA-16341) Fix un-compressed records

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-16341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-16341.

Resolution: Fixed

> Fix un-compressed records
> -
>
> Key: KAFKA-16341
> URL: https://issues.apache.org/jira/browse/KAFKA-16341
> Project: Kafka
>  Issue Type: Sub-task
>Affects Versions: 3.7.0, 3.6.1
>Reporter: Luke Chen
>Assignee: Johnny Hsu
>Priority: Major
> Fix For: 3.6.2, 3.8.0, 3.7.1
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2743

[Apache Jenkins Server]

See 

Re: [VOTE] 3.6.2 RC1

[Colin McCabe]

I have created a PR to fix this here : 
https://github.com/apache/kafka/pull/15584

best,
Colin

On Fri, Mar 22, 2024, at 13:28, Colin McCabe wrote:
> Sorry but I have to vote -1
>
> I tried verifying that the migration quotas bug described in 
> https://issues.apache.org/jira/browse/KAFKA-16222 was fixed, and it 
> appears to still be an issue with 3.6.2 RC1. The quota on the default 
> resource is still getting translated improperly.
>
> I am looking into what the issue is here.
>
> best,
> Colin
>
>
> On Thu, Mar 21, 2024, at 19:32, Chia-Ping Tsai wrote:
>> hi Manikumar
>>
>>> Pls let me know after merging the PR. I will generate RC2 later today.
>>
>> Sure. We will complete it ASAP
>>
>>
>>> Manikumar  於 2024年3月22日 上午9:26 寫道：
>>> 
>>> Hi,
>>> 
>>> Thanks. Since we have merged KAFKA-16342
>>> , it's probably better
>>> to take the PR for KAFKA-16341
>>>  for consistency.
>>> I am canceling the RC1 in favour of including KAFKA-16341
>>> .
>>> 
>>> Chia-Ping,
>>> 
>>> Pls let me know after merging the PR. I will generate RC2 later today.
>>> 
>>> 
>>> Thank you,
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Thu, Mar 21, 2024 at 9:10 PM Chia-Ping Tsai  wrote:
>>> 
> Is this a regression from the 3.5.0 release?
 
 I believe the bug is existent for a while, but it is not a true issue until
 we allowed users to fetch offset of max timestamp.
 
> Can we update the "Affects Version/s" field on JIRA?
 
 done. I attach the tags for active branches - 3.6.1 and 3.7.0
 
 Manikumar  於 2024年3月21日 週四 下午11:12寫道：
 
> Hi Chia-Ping,
> 
> Thanks for letting me know.
> 
> Is this a regression from the 3.5.0 release?  Can we update the "Affects
> Version/s" field on JIRA?
> 
> Thanks,
> 
> 
> On Thu, Mar 21, 2024 at 5:06 PM Chia-Ping Tsai 
 wrote:
> 
>> hi Manikumar,
>> 
>> There is a bug fix which needs to be backport to 3.6 (
>> https://issues.apache.org/jira/browse/KAFKA-16341)
>> 
>> It fixes the incorrect offset of max timestamp in non-compress path.
 The
>> other paths are already fixed by
>> https://issues.apache.org/jira/browse/KAFKA-16342.
>> 
>> Personally, we should backport both fixes for all paths, and we can
>> complete the backport today.
>> 
>> Sorry for bring this news to RC1.
>> 
>> Best,
>> Chia-Ping
>> 
>> 
>> Manikumar  於 2024年3月21日 週四 下午6:11寫道：
>> 
>>> Hello Kafka users, developers and client-developers,
>>> 
>>> This is the first candidate for release of Apache Kafka 3.6.2.
>>> 
>>> This is a bugfix release with several fixes, including dependency
>>> version bumps for CVEs.
>>> 
>>> Release notes for the 3.6.2 release:
>>> 
 https://home.apache.org/~manikumar/kafka-3.6.2-rc1/RELEASE_NOTES.html
>>> 
>>> *** Please download, test and vote by Tuesday, March 26th
>>> 
>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>>> https://kafka.apache.org/KEYS
>>> 
>>> * Release artifacts to be voted upon (source and binary):
>>> https://home.apache.org/~manikumar/kafka-3.6.2-rc1/
>>> 
>>> 
>>> * Maven artifacts to be voted upon:
>>> 
 https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>> 
>>> * Javadoc:
>>> https://home.apache.org/~manikumar/kafka-3.6.2-rc1/javadoc/
>>> 
>>> * Tag to be voted upon (off 3.6 branch) is the 3.6.2 tag:
>>> https://github.com/apache/kafka/releases/tag/3.6.2-rc1
>>> 
>>> * Documentation:
>>> https://kafka.apache.org/36/documentation.html
>>> 
>>> * Protocol:
>>> https://kafka.apache.org/36/protocol.html
>>> 
>>> * Successful Jenkins builds for the 3.6 branch:
>>> Unit/integration tests:
>>> 
>>> 
>> 
> 
 https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.6/159/tests
>>> (with few flaky failures)
>>> System tests: I will update system test results
>>> 
>>> Thanks,
>>> Manikumar
>>> 
>> 
> 

[jira] [Created] (KAFKA-16411) Correctly migrate default entities in KRaft migration

[Colin McCabe (Jira)]

Colin McCabe created KAFKA-16411:


 Summary: Correctly migrate default entities in KRaft migration
 Key: KAFKA-16411
 URL: https://issues.apache.org/jira/browse/KAFKA-16411
 Project: Kafka
  Issue Type: Bug
Reporter: Colin McCabe
Assignee: Colin McCabe






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.6 #162

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-16410) kafka-leader-election / LeaderElectionCommand doesn't set exit code on error

[Greg Harris (Jira)]

Greg Harris created KAFKA-16410:
---

 Summary: kafka-leader-election / LeaderElectionCommand doesn't set 
exit code on error
 Key: KAFKA-16410
 URL: https://issues.apache.org/jira/browse/KAFKA-16410
 Project: Kafka
  Issue Type: Bug
  Components: tools
Affects Versions: 3.7.0
Reporter: Greg Harris
 Fix For: 3.7.1


The kafka-leader-election command does not set the process exit code to nonzero 
when an unexpected error occurs.
{noformat}
% bin/kafka-leader-election.sh --path-to-json-file /tmp/does-not-exist     
Missing required option(s): bootstrap-server, election-type
org.apache.kafka.server.common.AdminCommandFailedException: Missing required 
option(s): bootstrap-server, election-type
        at 
org.apache.kafka.tools.LeaderElectionCommand$LeaderElectionCommandOptions.validate(LeaderElectionCommand.java:332)
        at 
org.apache.kafka.tools.LeaderElectionCommand.run(LeaderElectionCommand.java:78)
        at 
org.apache.kafka.tools.LeaderElectionCommand.main(LeaderElectionCommand.java:66)
% echo "$?"
0
{noformat}
The exit code is sometimes set properly when other code paths cause the command 
to exit, or in versions < 3.7:
{noformat}
% bin/kafka-leader-election.sh
This tool attempts to elect a new leader for a set of topic partitions. The 
type of elections supported are preferred replicas and unclean replicas.
Option                                  Description                           
--                                  ---                       
...
% echo "$?"
1
{noformat}
This appears to be a regression in 3.7.0, and since a shell script may be 
relying on the return code from this command, this is something we should fix 
in the next release.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-16409) kafka-delete-records / DeleteRecordsCommand should use standard exception handling

[Greg Harris (Jira)]

Greg Harris created KAFKA-16409:
---

 Summary: kafka-delete-records / DeleteRecordsCommand should use 
standard exception handling
 Key: KAFKA-16409
 URL: https://issues.apache.org/jira/browse/KAFKA-16409
 Project: Kafka
  Issue Type: Task
Affects Versions: 3.7.0
Reporter: Greg Harris


When an exception is thrown in kafka-delete-records, it propagates through 
`main` to the JVM, producing the following message:
{noformat}
bin/kafka-delete-records.sh --bootstrap-server localhost:9092 
--offset-json-file /tmp/does-not-exist
Exception in thread "main" java.io.IOException: Unable to read file 
/tmp/does-not-exist
        at org.apache.kafka.common.utils.Utils.readFileAsString(Utils.java:787)
        at 
org.apache.kafka.tools.DeleteRecordsCommand.execute(DeleteRecordsCommand.java:105)
        at 
org.apache.kafka.tools.DeleteRecordsCommand.main(DeleteRecordsCommand.java:64)
Caused by: java.nio.file.NoSuchFileException: /tmp/does-not-exist
        at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at 
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
        at java.nio.file.Files.newByteChannel(Files.java:361)
        at java.nio.file.Files.newByteChannel(Files.java:407)
        at java.nio.file.Files.readAllBytes(Files.java:3152)
        at org.apache.kafka.common.utils.Utils.readFileAsString(Utils.java:784)
        ... 2 more{noformat}
This is in contrast to the error handling used in other tools, such as the 
kafka-log-dirs:
{noformat}
bin/kafka-log-dirs.sh --bootstrap-server localhost:9092 --describe 
--command-config /tmp/does-not-exist
/tmp/does-not-exist
java.nio.file.NoSuchFileException: /tmp/does-not-exist
        at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at 
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
        at java.nio.file.Files.newByteChannel(Files.java:361)
        at java.nio.file.Files.newByteChannel(Files.java:407)
        at 
java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
        at java.nio.file.Files.newInputStream(Files.java:152)
        at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:686)
        at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:673)
        at 
org.apache.kafka.tools.LogDirsCommand.createAdminClient(LogDirsCommand.java:149)
        at org.apache.kafka.tools.LogDirsCommand.execute(LogDirsCommand.java:68)
        at 
org.apache.kafka.tools.LogDirsCommand.mainNoExit(LogDirsCommand.java:54)
        at 
org.apache.kafka.tools.LogDirsCommand.main(LogDirsCommand.java:49){noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

[Jun Rao]

Hi, Jose,

Thanks for the reply. A few more comments.

54. Admin.addMetadataVoter: It seems that Endpoint shouldn't include
securityProtocol since it's not in DescribeQuorumResponse.

55. Metrics:
55.1 It would be useful to be clear whether they are reported by the
controller leader, all controllers or all controllers and brokers.
55.2 IsObserver, type=KafkaController: Should we use the dash convention to
be consistent with the rest of the metrics?

56. kafka-storage : "If the --release-version flag is not specified, the
IBP in the configuration is used."
  kafka-storage takes controller.properties as the input parameter and IBP
is not a controller property, right?

57. To be consistent with kafka-storage, should we make the
--release-version flag in kafka-features optional too? If this is not
specified, the default associated with the tool will be used.

58. Typo: when the voter ID and UUID doesn't match
  doesn't => don't

Jun

On Fri, Mar 22, 2024 at 9:21 AM José Armando García Sancio
 wrote:

> Hi Claude,
>
> On Fri, Mar 22, 2024 at 4:36 AM Claude Warren  wrote:
> > Is there test code, or initial POC code for this KIP somewhere?  I would
> like to help move this forward but need a few pointers to associated
> resources.  I have read KIP-853 and it is beginning to sink in, but code
> would be nice.
>
> Thanks for your interest and I would appreciate the help with the
> implementation. I don't have a lot of code to show at the moment. The
> existing KRaft implementation is in the "raft" Java module in the
> apache/kafka repo.
>
> I am planning to create a set of sub-tasks under KAFKA-14094 soon, to
> give a rough outline of what needs implementing.
>
> Thanks,
> --
> -José
>

Re: [VOTE] 3.6.2 RC1

[Colin McCabe]

Sorry but I have to vote -1

I tried verifying that the migration quotas bug described in 
https://issues.apache.org/jira/browse/KAFKA-16222 was fixed, and it appears to 
still be an issue with 3.6.2 RC1. The quota on the default resource is still 
getting translated improperly.

I am looking into what the issue is here.

best,
Colin


On Thu, Mar 21, 2024, at 19:32, Chia-Ping Tsai wrote:
> hi Manikumar
>
>> Pls let me know after merging the PR. I will generate RC2 later today.
>
> Sure. We will complete it ASAP
>
>
>> Manikumar  於 2024年3月22日 上午9:26 寫道：
>> 
>> Hi,
>> 
>> Thanks. Since we have merged KAFKA-16342
>> , it's probably better
>> to take the PR for KAFKA-16341
>>  for consistency.
>> I am canceling the RC1 in favour of including KAFKA-16341
>> .
>> 
>> Chia-Ping,
>> 
>> Pls let me know after merging the PR. I will generate RC2 later today.
>> 
>> 
>> Thank you,
>> 
>> 
>> 
>> 
>> 
>> 
>> On Thu, Mar 21, 2024 at 9:10 PM Chia-Ping Tsai  wrote:
>> 
 Is this a regression from the 3.5.0 release?
>>> 
>>> I believe the bug is existent for a while, but it is not a true issue until
>>> we allowed users to fetch offset of max timestamp.
>>> 
 Can we update the "Affects Version/s" field on JIRA?
>>> 
>>> done. I attach the tags for active branches - 3.6.1 and 3.7.0
>>> 
>>> Manikumar  於 2024年3月21日 週四 下午11:12寫道：
>>> 
 Hi Chia-Ping,
 
 Thanks for letting me know.
 
 Is this a regression from the 3.5.0 release?  Can we update the "Affects
 Version/s" field on JIRA?
 
 Thanks,
 
 
 On Thu, Mar 21, 2024 at 5:06 PM Chia-Ping Tsai 
>>> wrote:
 
> hi Manikumar,
> 
> There is a bug fix which needs to be backport to 3.6 (
> https://issues.apache.org/jira/browse/KAFKA-16341)
> 
> It fixes the incorrect offset of max timestamp in non-compress path.
>>> The
> other paths are already fixed by
> https://issues.apache.org/jira/browse/KAFKA-16342.
> 
> Personally, we should backport both fixes for all paths, and we can
> complete the backport today.
> 
> Sorry for bring this news to RC1.
> 
> Best,
> Chia-Ping
> 
> 
> Manikumar  於 2024年3月21日 週四 下午6:11寫道：
> 
>> Hello Kafka users, developers and client-developers,
>> 
>> This is the first candidate for release of Apache Kafka 3.6.2.
>> 
>> This is a bugfix release with several fixes, including dependency
>> version bumps for CVEs.
>> 
>> Release notes for the 3.6.2 release:
>> 
>>> https://home.apache.org/~manikumar/kafka-3.6.2-rc1/RELEASE_NOTES.html
>> 
>> *** Please download, test and vote by Tuesday, March 26th
>> 
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>> 
>> * Release artifacts to be voted upon (source and binary):
>> https://home.apache.org/~manikumar/kafka-3.6.2-rc1/
>> 
>> 
>> * Maven artifacts to be voted upon:
>> 
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> 
>> * Javadoc:
>> https://home.apache.org/~manikumar/kafka-3.6.2-rc1/javadoc/
>> 
>> * Tag to be voted upon (off 3.6 branch) is the 3.6.2 tag:
>> https://github.com/apache/kafka/releases/tag/3.6.2-rc1
>> 
>> * Documentation:
>> https://kafka.apache.org/36/documentation.html
>> 
>> * Protocol:
>> https://kafka.apache.org/36/protocol.html
>> 
>> * Successful Jenkins builds for the 3.6 branch:
>> Unit/integration tests:
>> 
>> 
> 
 
>>> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.6/159/tests
>> (with few flaky failures)
>> System tests: I will update system test results
>> 
>> Thanks,
>> Manikumar
>> 
> 
 
>>>

[jira] [Created] (KAFKA-16408) kafka-get-offsets / GetOffsetShell doesn't handle --version or --help

[Greg Harris (Jira)]

Greg Harris created KAFKA-16408:
---

 Summary: kafka-get-offsets / GetOffsetShell doesn't handle 
--version or --help
 Key: KAFKA-16408
 URL: https://issues.apache.org/jira/browse/KAFKA-16408
 Project: Kafka
  Issue Type: Bug
  Components: tools
Affects Versions: 3.7.0
Reporter: Greg Harris


The kafka-get-offsets script was moved to Java for the 3.7.0 release. It now 
inherits the CommandDefaultOptions, including "--help" and "–version". However, 
it does not process these arguments like other commands, and instead prints the 
following error:
{noformat}
./bin/kafka-get-offsets.sh --version
Missing required argument "[broker-list]"
...
--help                                   Print usage information.           ... 
  
--version                                Display Kafka version.{noformat}
followed by the usage/help.

It should follow the behavior of other commands, which is to print the 
help/version without complaining about missing arguments. It should call 
CommandLineUtils.maybePrintHelpOrVersion somewhere during argument parsing.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2742

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-16407) ForeignKey INNER join ignores FK change when its previous value is null

[Ayoub Omari (Jira)]

Ayoub Omari created KAFKA-16407:
---

 Summary: ForeignKey INNER join ignores FK change when its previous 
value is null
 Key: KAFKA-16407
 URL: https://issues.apache.org/jira/browse/KAFKA-16407
 Project: Kafka
  Issue Type: Bug
  Components: streams
Affects Versions: 3.7.0
Reporter: Ayoub Omari


We have two topics : _left-topic[String, LeftRecord]_ and _right-topic[String, 
String]_

where _LeftRecord_ :
{code:scala}
 case class LeftRecord(foreignKey: String, name: String){code}
we do a simple *INNER* foreign key join on left-topic's foreignKey field. The 
resulting join value is the value in right-topic.

 

*Scenario: Primary key pk1 gets mapped to a new FK after having a null FK*
{code:scala}
rightTopic.pipeInput("fk", "1")

leftTopic.pipeInput("pk1", LeftRecord(null, "pk1"))
leftTopic.pipeInput("pk1", LeftRecord("fk", "pk1")) {code}
 

*+Expected result+*
{code:scala}
KeyValue(pk1, 1){code}
 

*+Actual result+*
{code:scala}
# No output !

# Logs:

20:14:29,723 WARN  
org.apache.kafka.streams.kstream.internals.foreignkeyjoin.ForeignJoinSubscriptionSendProcessorSupplier
  - Skipping record due to null foreign key. value=[LeftRecord(null,pk1)] 
topic=[left-topic] partition=[0] offset=[0]

20:14:29,728 WARN  
org.apache.kafka.streams.kstream.internals.foreignkeyjoin.ForeignJoinSubscriptionSendProcessorSupplier
  - Skipping record due to null foreign key. value=[LeftRecord(null,pk1)] 
topic=[left-topic] partition=[0] offset=[1]
{code}
 

After looking into the code, I believe this is the line behind the issue : 
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/foreignkeyjoin/SubscriptionSendProcessorSupplier.java#L147



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-16354) FinalizedFeatureChangeListenerTest should use mocked latches

[Greg Harris (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Harris resolved KAFKA-16354.
-
Resolution: Won't Fix

> FinalizedFeatureChangeListenerTest should use mocked latches
> 
>
> Key: KAFKA-16354
> URL: https://issues.apache.org/jira/browse/KAFKA-16354
> Project: Kafka
>  Issue Type: Test
>  Components: core
>Affects Versions: 3.7.0
>Reporter: Greg Harris
>Priority: Trivial
>  Labels: newbie
>
> testCacheUpdateWaitFailsForUnreachableVersion takes 30 seconds, and 
> testInitFailureDueToFeatureIncompatibility takes 5 seconds. This appears to 
> be caused by FeatureCacheUpdater#awaitUpdateOrThrow waiting for a real 
> CountDownLatch with a real-time timeout.
> Instead, a mocked latch should be used to exit the await immediately.
> This should be done both for CPU-independence, and for test execution speed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-16406) Split long-running consumer integration test

[Lianet Magrans (Jira)]

Lianet Magrans created KAFKA-16406:
--

 Summary: Split long-running consumer integration test
 Key: KAFKA-16406
 URL: https://issues.apache.org/jira/browse/KAFKA-16406
 Project: Kafka
  Issue Type: Task
Reporter: Lianet Magrans
Assignee: Lianet Magrans


PlaintextConsumerTest contains integration tests for the consumer. Since the 
introduction of the new consumer group protocol (KIP-848) and the new 
KafkaConsumer, this test has been parametrized to run with multiple 
combinations, making sure we test the logic for the old and new coordinator, as 
well as for the legacy and new KafkaConsumer. 

This led to this being one of the longest-running integration tests, so in the 
aim of reducing the impact on the build times we could split it to allow for 
parallelization.  The tests covers multiple areas of the consumer logic, in a 
single file, so splitting based on the high-level features being tested would 
be sensible and achieve the result wanted.   




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-16405) Mismatch assignment error when running consumer rolling upgrade system tests

[Philip Nee (Jira)]

Philip Nee created KAFKA-16405:
--

 Summary: Mismatch assignment error when running consumer rolling 
upgrade system tests
 Key: KAFKA-16405
 URL: https://issues.apache.org/jira/browse/KAFKA-16405
 Project: Kafka
  Issue Type: Task
  Components: consumer, system tests
Reporter: Philip Nee


relevant to [https://github.com/apache/kafka/pull/15578]

 

We are seeing:
{code:java}

SESSION REPORT (ALL TESTS)
ducktape version: 0.11.4
session_id:   2024-03-21--001
run time: 3 minutes 24.632 seconds
tests run:7
passed:   5
flaky:0
failed:   2
ignored:  0

test_id:
kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=COMBINED_KRAFT.use_new_coordinator=True.group_protocol=classic
status: PASS
run time:   24.599 seconds

test_id:
kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=COMBINED_KRAFT.use_new_coordinator=True.group_protocol=consumer
status: FAIL
run time:   26.638 seconds


AssertionError("Mismatched assignment: {frozenset(), 
frozenset({TopicPartition(topic='test_topic', partition=3), 
TopicPartition(topic='test_topic', partition=0), 
TopicPartition(topic='test_topic', partition=1), 
TopicPartition(topic='test_topic', partition=2)})}")
Traceback (most recent call last):
  File 
"/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 
186, in _do_run
data = self.run_test()
  File 
"/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 
246, in run_test
return self.test_context.function(self.test)
  File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
433, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File 
"/opt/kafka-dev/tests/kafkatest/tests/client/consumer_rolling_upgrade_test.py", 
line 77, in rolling_update_test
self._verify_range_assignment(consumer)
  File 
"/opt/kafka-dev/tests/kafkatest/tests/client/consumer_rolling_upgrade_test.py", 
line 38, in _verify_range_assignment
assert assignment == set([
AssertionError: Mismatched assignment: {frozenset(), 
frozenset({TopicPartition(topic='test_topic', partition=3), 
TopicPartition(topic='test_topic', partition=0), 
TopicPartition(topic='test_topic', partition=1), 
TopicPartition(topic='test_topic', partition=2)})}


test_id:
kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=ISOLATED_KRAFT.use_new_coordinator=False
status: PASS
run time:   29.815 seconds

test_id:
kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=ISOLATED_KRAFT.use_new_coordinator=True
status: PASS
run time:   29.766 seconds

test_id:
kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=ISOLATED_KRAFT.use_new_coordinator=True.group_protocol=classic
status: PASS
run time:   30.086 seconds

test_id:
kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=ISOLATED_KRAFT.use_new_coordinator=True.group_protocol=consumer
status: FAIL
run time:   35.965 seconds


AssertionError("Mismatched assignment: {frozenset(), 
frozenset({TopicPartition(topic='test_topic', partition=3), 
TopicPartition(topic='test_topic', partition=0), 
TopicPartition(topic='test_topic', partition=1), 
TopicPartition(topic='test_topic', partition=2)})}")
Traceback (most recent call last):
  File 
"/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 
186, in _do_run
data = self.run_test()
  File 
"/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 
246, in run_test
return self.test_context.function(self.test)
  File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
433, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File 
"/opt/kafka-dev/tests/kafkatest/tests/client/consumer_rolling_upgrade_test.py", 
line 77, in rolling_update_test
self._verify_range_assignment(consumer)
  File 

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

[José Armando García Sancio]

Hi Claude,

On Fri, Mar 22, 2024 at 4:36 AM Claude Warren  wrote:
> Is there test code, or initial POC code for this KIP somewhere?  I would like 
> to help move this forward but need a few pointers to associated resources.  I 
> have read KIP-853 and it is beginning to sink in, but code would be nice.

Thanks for your interest and I would appreciate the help with the
implementation. I don't have a lot of code to show at the moment. The
existing KRaft implementation is in the "raft" Java module in the
apache/kafka repo.

I am planning to create a set of sub-tasks under KAFKA-14094 soon, to
give a rough outline of what needs implementing.

Thanks,
-- 
-José

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

[Bruno Cadonna]

Hi Walker,

A couple of follow-up questions.

1.
Why do you propose to explicitly pass a parameter "storeName" in 
StreamsBuilder#addGlobalStore?
The StoreBuilder should already provide a name for the store, if I 
understand the code correctly.
I would avoid using the same name for the source node and the state 
store, because it limits the flexibility in naming. Why do you not use 
Named for the name of the source node?


2.
Did you consider Matthias' proposal to restrict the type of the store 
builder to `StoreBuilder` (or even 
`StoreBuilder`) for the case where 
the processor is built-in?



Best,
Bruno

On 3/13/24 11:05 PM, Walker Carlson wrote:

Thanks for the feedback Bruno, Matthias, and Lucas!

There is a decent amount but I'm going to try and just hit the major points
as I would like to keep this change simple.

I've made corrections for the mistakes pointed out. Thanks for the
suggestions everyone.

The main sticking point seems to be with the method of signalling the
restore behavior. It seems we can all agree with how the API should look
with the default option we are adding. I think keeping the option to load
directly from the topic into the store is a good idea. It is much more
performant and could make a simple metric collector processor much simpler.

I think something that Matthais said about creating a special class of
processors for the global stores helps me think about the issue. I tend to
fall into the category that we should keep global stores open to the
possibility of having child nodes in the future. I don't really see the
downside of having that as an option. It might not be best for a lot of
cases, but something simple could be very useful to put in the PAPI.

I like the idea of having a `GlobalStoreParameters` but only if we decide
to make the processor need to extend an interface like
'GobalStoreProcessor`. If not that seems excessive.

As of right now I don't see a better option than having a boolean flag for
the reprocessOnRestore option. I expanded the description in the docs so I
hope that helps.

I am more than willing to take other ideas on it.

thanks,
Walker

Re: [PR] Update Bruno's picture [kafka-site]

[via GitHub]

cadonna commented on PR #592:
URL: https://github.com/apache/kafka-site/pull/592#issuecomment-2015145311

   Sorry, I forgot to add the both of you as reviewers to the commit.  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Update Bruno's picture [kafka-site]

[via GitHub]

cadonna merged PR #592:
URL: https://github.com/apache/kafka-site/pull/592


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] Update Bruno's picture [kafka-site]

[via GitHub]

cadonna commented on PR #592:
URL: https://github.com/apache/kafka-site/pull/592#issuecomment-2015143868

   Thanks! I looked at the pic in a browser and it looked fine.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

[Claude Warren]

Is there test code, or initial POC code for this KIP somewhere?  I would like 
to help move this forward but need a few pointers to associated resources.  I 
have read KIP-853 and it is beginning to sink in, but code would be nice.

Thanks,
Claude

On 2024/03/21 18:41:04 José Armando García Sancio wrote:
> Hi Jun,
> 
> On Thu, Mar 14, 2024 at 3:38 PM Jun Rao  wrote:
> > 52. Admin client: Could you provide a bit more details on the changes?
> 
> I updated the KIP to include the API changes to the Admin client.
> 
> Thanks,
> -- 
> -José
> 

[PR] Update Bruno's picture [kafka-site]

[via GitHub]

cadonna opened a new pull request, #592:
URL: https://github.com/apache/kafka-site/pull/592

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (KAFKA-16404) Flaky test org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testGetStreamsConfig

[Igor Soarez (Jira)]

Igor Soarez created KAFKA-16404:
---

 Summary: Flaky test 
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testGetStreamsConfig
 Key: KAFKA-16404
 URL: https://issues.apache.org/jira/browse/KAFKA-16404
 Project: Kafka
  Issue Type: Bug
Reporter: Igor Soarez


 
{code:java}
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testGetStreamsConfig()
 failed, log available in 
/home/jenkins/workspace/Kafka_kafka-pr_PR-14903@2/streams/examples/build/reports/testOutput/org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testGetStreamsConfig().test.stdout
Gradle Test Run :streams:examples:test > Gradle Test Executor 87 > 
WordCountDemoTest > testGetStreamsConfig() FAILED
    org.apache.kafka.streams.errors.ProcessorStateException: Error opening 
store KSTREAM-AGGREGATE-STATE-STORE-03 at location 
/tmp/kafka-streams/streams-wordcount/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03
        at 
app//org.apache.kafka.streams.state.internals.RocksDBStore.openRocksDB(RocksDBStore.java:329)
        at 
app//org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:69)
        at 
app//org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:254)
        at 
app//org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:175)
        at 
app//org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:71)
        at 
app//org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:56)
        at 
app//org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:71)
        at 
app//org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$3(MeteredKeyValueStore.java:151)
        at 
app//org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:872)
        at 
app//org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:151)
        at 
app//org.apache.kafka.streams.processor.internals.ProcessorStateManager.registerStateStores(ProcessorStateManager.java:232)
        at 
app//org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:102)
        at 
app//org.apache.kafka.streams.processor.internals.StreamTask.initializeIfNeeded(StreamTask.java:258)
        at 
app//org.apache.kafka.streams.TopologyTestDriver.setupTask(TopologyTestDriver.java:530)
        at 
app//org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:373)
        at 
app//org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:300)
        at 
app//org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:276)
        at 
app//org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.setup(WordCountDemoTest.java:60)
        Caused by:
        org.rocksdb.RocksDBException: While lock file: 
/tmp/kafka-streams/streams-wordcount/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03/LOCK:
 Resource temporarily unavailable
            at app//org.rocksdb.RocksDB.open(Native Method)
            at app//org.rocksdb.RocksDB.open(RocksDB.java:307)
            at 
app//org.apache.kafka.streams.state.internals.RocksDBStore.openRocksDB(RocksDBStore.java:323)
            ... 17 more
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-16403) Flaky test org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testCountListOfWords

[Igor Soarez (Jira)]

Igor Soarez created KAFKA-16403:
---

 Summary: Flaky test 
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testCountListOfWords
 Key: KAFKA-16403
 URL: https://issues.apache.org/jira/browse/KAFKA-16403
 Project: Kafka
  Issue Type: Bug
Reporter: Igor Soarez


{code:java}
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testCountListOfWords()
 failed, log available in 
/home/jenkins/workspace/Kafka_kafka-pr_PR-14903/streams/examples/build/reports/testOutput/org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testCountListOfWords().test.stdout
Gradle Test Run :streams:examples:test > Gradle Test Executor 82 > 
WordCountDemoTest > testCountListOfWords() FAILED
    org.apache.kafka.streams.errors.ProcessorStateException: Error opening 
store KSTREAM-AGGREGATE-STATE-STORE-03 at location 
/tmp/kafka-streams/streams-wordcount/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03
        at 
org.apache.kafka.streams.state.internals.RocksDBStore.openRocksDB(RocksDBStore.java:329)
        at 
org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:69)
        at 
org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:254)
        at 
org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:175)
        at 
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:71)
        at 
org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:56)
        at 
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:71)
        at 
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$3(MeteredKeyValueStore.java:151)
        at 
org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:872)
        at 
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:151)
        at 
org.apache.kafka.streams.processor.internals.ProcessorStateManager.registerStateStores(ProcessorStateManager.java:232)
        at 
org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:102)
        at 
org.apache.kafka.streams.processor.internals.StreamTask.initializeIfNeeded(StreamTask.java:258)
        at 
org.apache.kafka.streams.TopologyTestDriver.setupTask(TopologyTestDriver.java:530)
        at 
org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:373)
        at 
org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:300)
        at 
org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:276)
        at 
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.setup(WordCountDemoTest.java:60)
        Caused by:
        org.rocksdb.RocksDBException: Corruption: IO error: No such file or 
directory: While open a file for random read: 
/tmp/kafka-streams/streams-wordcount/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03/10.ldb:
 No such file or directory in file 
/tmp/kafka-streams/streams-wordcount/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03/MANIFEST-05
            at org.rocksdb.RocksDB.open(Native Method)
            at org.rocksdb.RocksDB.open(RocksDB.java:307)
            at 
org.apache.kafka.streams.state.internals.RocksDBStore.openRocksDB(RocksDBStore.java:323)
            ... 17 more
 {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-16402) Flaky test org.apache.kafka.controller.QuorumControllerTest.testSnapshotSaveAndLoad

[Igor Soarez (Jira)]

Igor Soarez created KAFKA-16402:
---

 Summary: Flaky test 
org.apache.kafka.controller.QuorumControllerTest.testSnapshotSaveAndLoad
 Key: KAFKA-16402
 URL: https://issues.apache.org/jira/browse/KAFKA-16402
 Project: Kafka
  Issue Type: Bug
Reporter: Igor Soarez


{code:java}
org.apache.kafka.controller.QuorumControllerTest.testSnapshotSaveAndLoad() 
failed, log available in 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-14903/metadata/build/reports/testOutput/org.apache.kafka.controller.QuorumControllerTest.testSnapshotSaveAndLoad().test.stdout
Gradle Test Run :metadata:test > Gradle Test Executor 93 > QuorumControllerTest 
> testSnapshotSaveAndLoad() FAILED
    java.lang.IllegalArgumentException: Self-suppression not permitted
        at java.base/java.lang.Throwable.addSuppressed(Throwable.java:1072)
        at 
org.apache.kafka.controller.QuorumControllerTest.testSnapshotSaveAndLoad(QuorumControllerTest.java:645)
        Caused by:
        org.apache.kafka.server.fault.FaultHandlerException: fatalFaultHandler: 
exception while renouncing leadership: Attempt to resign from epoch 1 which is 
larger than the current epoch 0
            at 
app//org.apache.kafka.metalog.LocalLogManager.resign(LocalLogManager.java:808)
            at 
app//org.apache.kafka.controller.QuorumController.renounce(QuorumController.java:1270)
            at 
app//org.apache.kafka.controller.QuorumController.handleEventException(QuorumController.java:547)
            at 
app//org.apache.kafka.controller.QuorumController.access$800(QuorumController.java:179)
            at 
app//org.apache.kafka.controller.QuorumController$ControllerWriteEvent.complete(QuorumController.java:881)
            at 
app//org.apache.kafka.controller.QuorumController$ControllerWriteEvent.handleException(QuorumController.java:871)
            at 
app//org.apache.kafka.queue.KafkaEventQueue$EventContext.completeWithException(KafkaEventQueue.java:149)
            at 
app//org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:138)
            at 
app//org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:211)
            at 
app//org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:182)
            at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
            Caused by:
            java.lang.IllegalArgumentException: Attempt to resign from epoch 1 
which is larger than the current epoch 0
                at 
org.apache.kafka.metalog.LocalLogManager.resign(LocalLogManager.java:808)
                ... 10 more {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-1028: Docker Official Image for Apache Kafka

[Krish Vora]

Hi Hector,

Thanks for reaching out. This KIP builds on top of KIP-975

and
aims to introduce a JVM-based Docker Official Image (DOI
) for Apache
Kafka that will be visible under Docker Official Images
. Once implemented
for Apache Kafka, for each release, there will be one more JVM-based Docker
image available to users.

Currently, we already have an OSS sponsored image, which was introduced via
KIP-975 (apache/kafka ) which
comes under The Apache Software Foundation  in
Docker Hub. The new Docker Image is the Docker Official Image (DOI), which
will be built and maintained by Docker Community.

For example, for a release version like 3.8.0 we will have two JVM based
docker images:-

   - apache/kafka:3.8.0 (OSS sponsored image)
   - kafka:3.8.0 (Docker Official image)


I have added the same in the KIP too for everyone's reference.
Thanks,
Krish.

On Fri, Mar 22, 2024 at 2:50 AM Hector Geraldino (BLOOMBERG/ 919 3RD A) <
hgerald...@bloomberg.net> wrote:

> Hi,
>
> What is the difference between this KIP and KIP-975: Docker Image for
> Apache Kafka?
>
> From: dev@kafka.apache.org At: 03/21/24 07:30:07 UTC-4:00To:
> dev@kafka.apache.org
> Subject: [DISCUSS] KIP-1028: Docker Official Image for Apache Kafka
>
> Hi everyone,
>
> I would like to start the discussion on
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im
> age+for+Apache+Kafka
>  .
>
> This KIP aims to introduce JVM based Docker Official Image (DOI) for Apache
> Kafka.
>
> Regards,
> Krish.
>
>
>

[jira] [Created] (KAFKA-16401) Client requests and consumer-pref test failed

[gendong1 (Jira)]

gendong1 created KAFKA-16401:


 Summary: Client requests and consumer-pref test failed
 Key: KAFKA-16401
 URL: https://issues.apache.org/jira/browse/KAFKA-16401
 Project: Kafka
  Issue Type: Bug
Affects Versions: 2.8.2
Reporter: gendong1


The cluster consists of 3 nodes. When the storeOffsets is delay due to 
fail-slow disk, the corresponding thread is blocked and holds a lock.The retry 
requests consume the IO thread and network threads set in server.proterty. 
However, the retry requests is blocked since they cannot acquire the lock.  The 
coming client requests and cosumer-pref tests failed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)