[
https://issues.apache.org/jira/browse/FLINK-23932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410708#comment-17410708
]
Nicolaus Weidner commented on FLINK-23932:
------------------------------------------
This looks like the same root cause as FLINK-22198 and FLINK-23778. This is
from the logs, looks like log compaction deleted entries before they were read
and then the test waited forever:
{code:java}
16:34:53,147 [docker-java-stream-1151525689] INFO
org.apache.flink.streaming.connectors.kafka.table.KafkaTableTestBase [] -
STDOUT: [2021-08-23 16:34:53,146] INFO [Log
partition=key_partial_value_topic_json-0, dir=/var/lib/kafka/data] Rolled new
log segment at offset 3 in 26 ms. (kafka.log.Log)
16:34:53,148 [docker-java-stream-1151525689] INFO
org.apache.flink.streaming.connectors.kafka.table.KafkaTableTestBase [] -
STDOUT: [2021-08-23 16:34:53,147] INFO [Log
partition=key_partial_value_topic_json-0, dir=/var/lib/kafka/data] Scheduling
segments for deletion List(LogSegment(baseOffset=0, size=389,
lastModifiedTime=1629736493000, largestTime=1583845931123)) (kafka.log.Log)
16:34:53,148 [Source Data Fetcher for Source:
KafkaSource-default_catalog.default_database.kafka -> Calc(select=[k_user_id,
name, CAST(timestamp) AS timestamp, k_event_id, user_id, payload]) -> Sink:
Collect table sink (1/1)#0] INFO
org.apache.kafka.clients.consumer.KafkaConsumer [] - [Consumer
clientId=flink-tests-0, groupId=flink-tests] Subscribed to partition(s):
key_partial_value_topic_json-0
16:34:53,149 [docker-java-stream-1151525689] INFO
org.apache.flink.streaming.connectors.kafka.table.KafkaTableTestBase [] -
STDOUT: [2021-08-23 16:34:53,148] INFO [Log
partition=key_partial_value_topic_json-0, dir=/var/lib/kafka/data] Incrementing
log start offset to 3 (kafka.log.Log)
16:34:53,149 [Source Data Fetcher for Source:
KafkaSource-default_catalog.default_database.kafka -> Calc(select=[k_user_id,
name, CAST(timestamp) AS timestamp, k_event_id, user_id, payload]) -> Sink:
Collect table sink (1/1)#0] INFO
org.apache.kafka.clients.consumer.internals.SubscriptionState [] - [Consumer
clientId=flink-tests-0, groupId=flink-tests] Seeking to EARLIEST offset of
partition key_partial_value_topic_json-0
16:34:53,158 [Source Data Fetcher for Source:
KafkaSource-default_catalog.default_database.kafka -> Calc(select=[k_user_id,
name, CAST(timestamp) AS timestamp, k_event_id, user_id, payload]) -> Sink:
Collect table sink (1/1)#0] INFO
org.apache.kafka.clients.consumer.internals.SubscriptionState [] - [Consumer
clientId=flink-tests-0, groupId=flink-tests] Resetting offset for partition
key_partial_value_topic_json-0 to offset 3.
16:35:21,819 [ Debug Logging Timer] INFO
org.apache.flink.streaming.connectors.kafka.table.KafkaTableTestBase [] -
TopicPartition "key_partial_value_topic_json-0": starting offset: 3, stopping
offset: 3
{code}
> KafkaTableITKafkaTableITCase.testKafkaSourceSinkWithKeyAndPartialValue hangs
> on AzureCase
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-23932
> URL: https://issues.apache.org/jira/browse/FLINK-23932
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Table SQL / Ecosystem
> Affects Versions: 1.14.0
> Reporter: Till Rohrmann
> Priority: Critical
> Labels: test-stability
> Fix For: 1.14.0
>
>
> The test {{KafkaTableITCase.testKafkaSourceSinkWithKeyAndPartialValue}} hangs
> on Azure. Interestingly, the test case seems to spawn 400
> {{kafka-admin-client-thread | adminclient}} threads. I think there is
> something wrong with the test case setup.
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22674&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=15a22db7-8faa-5b34-3920-d33c9f0ca23c&l=22875
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
