[
https://issues.apache.org/jira/browse/FLINK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17400893#comment-17400893
]
Dong Lin commented on FLINK-22198:
----------------------------------
I have taken a look at the test code and related logs (not including the Kafka
Server log).
Here are the findings:
- According to the test code and the log, 5 minutes after the test starts, the
logStartOffset=4 and the logEndOffset=4.
- According to the Kafka Source code, the logStartOffset will increase from 0
to 4 only in the following cases:
- The Admin::deleteRecords(...) is invoked by test code
- The log rolling happens because the current_time -
timestamp_of_produced_records >= retention_time
- The log rolling happens because the size of the log > max_log_segment_size.
My hypothesis is that the log rolling happens because the current_time -
timestamp_of_produced_records >= retention_time. This might happen if the clock
on the host that produces messages is several days earlier than the clock on
the host that runs the Kafka Server container. In general this difference in
clock is unlikely, because I find it even less likely to believe other reasons
that could have causes this logStartOffset=4.
In order to validate the hypothesis and move forward in the debugging, we have
updated Flink code to get Kafka Server info level logging and will check the
following:
- What is the timestamp of the producer and Kafka Server in the log?
- Does Kafka Server actually roll the log, and what even has triggered the log
rolling?
> KafkaTableITCase hang.
> ----------------------
>
> Key: FLINK-22198
> URL: https://issues.apache.org/jira/browse/FLINK-22198
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.14.0, 1.12.4
> Reporter: Guowei Ma
> Assignee: Qingsheng Ren
> Priority: Blocker
> Labels: pull-request-available, stale-blocker, test-stability
> Fix For: 1.14.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16287&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5&l=6625
> There is no any artifacts.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
