[
https://issues.apache.org/jira/browse/KAFKA-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738558#comment-17738558
]
Mukesh Mishra commented on KAFKA-15097:
---------------------------------------
There is something wired in log 4146 of consumer offset :
In 00000000000004146.log.swap, i can see below records :
{code:title=4146.log.swap|borderStyle=solid}
Log starting offset: 4146
baseOffset: 7435 lastOffset: 7435 count: 1 baseSequence: -1 lastSequence: -1
producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 23 isTransactional:
false isControl: false deleteHorizonMs: OptionalLong.empty position: 0
CreateTime: 1684734374514 size: 497 magic: 2 compresscodec: none crc:
2946355956 isvalid: true
| offset: 7435 CreateTime: 1684734374514 keySize: 55 valueSize: 372 sequence:
-1 headerKeys: [] key: group_metadata::group=group-//host:8409/SmartServer
payload:
{"protocolType":"consumer","protocol":"range","generationId":47,"assignment":"{consumer-//host:8409/SmartServer-4fc37a78-2757-430f-b974-11ff81bf8fd9=[mdmnxst-topic-0]}"}
baseOffset: 7740 lastOffset: 7740 count: 1 baseSequence: 0 lastSequence: 0
producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 23 isTransactional:
false isControl: false deleteHorizonMs: OptionalLong.empty position: 497
CreateTime: 1684919856106 size: 305 magic: 2 compresscodec: none crc:
1149086487 isvalid: true
| offset: 7740 CreateTime: 1684919856106 keySize: 74 valueSize: 160 sequence: 0
headerKeys: [] key:
offset_commit::group=group-//host:8409/SmartServer,partition=mdmnxst-topic-0
payload: offset=24434,metadata=[Consumer
clientId=consumer-//host:8409/SmartServer,
groupId=group-//host:8409/SmartServer]
{code}
baseOffset (7435, 7740) mentioned in 00000000000004146.log.swap which is
already present in 00000000000006340.log (they should present only in 6340.log)
{code:java}
baseOffset: 7434 lastOffset: 7434 count: 1 baseSequence: -1 lastSequence: -1
producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 23 isTransactional:
false isControl: false deleteHorizonMs: OptionalLong.empty position: 333799
CreateTime: 1684734253921 size: 156 magic: 2 compresscodec: none crc: 867707744
isvalid: true
| offset: 7434 CreateTime: 1684734253921 keySize: 55 valueSize: 32 sequence: -1
headerKeys: [] key: group_metadata::group=group-//host:8409/SmartServer
payload:
{"protocolType":"consumer","protocol":null,"generationId":46,"assignment":"{}"}
baseOffset: 7435 lastOffset: 7435 count: 1 baseSequence: -1 lastSequence: -1
producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 23 isTransactional:
false isControl: false deleteHorizonMs: OptionalLong.empty position: 333955
CreateTime: 1684734374514 size: 497 magic: 2 compresscodec: none crc:
2946355956 isvalid: true
| offset: 7435 CreateTime: 1684734374514 keySize: 55 valueSize: 372 sequence:
-1 headerKeys: [] key: group_metadata::group=group-//host:8409/SmartServer
payload:
{"protocolType":"consumer","protocol":"range","generationId":47,"assignment":"{consumer-//host:8409/SmartServer-4fc37a78-2757-430f-b974-11ff81bf8fd9=[mdmnxst-topic-0]}"}
{code}
It is not normal that same baseoffset present in two logs (4146.log.swap and
6340.log). 7435 and 7740 baseOffset should not be present 4146.log.swap when
there is already 6349.log
> NoSuchFileException in LogCleaner Operation.
> --------------------------------------------
>
> Key: KAFKA-15097
> URL: https://issues.apache.org/jira/browse/KAFKA-15097
> Project: Kafka
> Issue Type: Bug
> Components: log cleaner
> Affects Versions: 3.3.1
> Reporter: Mukesh Mishra
> Priority: Blocker
> Attachments: __consumer_offset_49.png, config.txt, log-cleaner.log,
> server.log
>
>
> Currently we are facing NoSuchFileException in LogCleaner, which is critical
> error as result platform get shutown.
> We are running Kafka in Kraft Mode.
> The cluster is having 3 Node.
> Kafka version - 3.3.1
> We are facing issue systematically, which occurs after retention reached.
> Here's the logs of Node 1 for consumer offset 4146 for which we have faced
> NoSuchFileException on 24 may 2023
> Config :
> [^config.txt]
> ^Logs :^
> ^Server Node 1 [^server.log]^
> Log Cleaner :
> [^log-cleaner.log]
> ^__consumer_offset_49 directory logs :^
> ^!__consumer_offset_49.png!^
> we suspect that swap operation is culprit as we can see swap file for
> consumer offset 4146 for __consumer_offset_49
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
