[
https://issues.apache.org/jira/browse/FLINK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shengkai Fang updated FLINK-20058:
----------------------------------
Description:
Currenlty, we have already pushed watermark into the kafka table source. If the
consumer doesn't get data from each partition, it will not output a valid
watermark. In the test {{KafkaTableITCase#testPerPartitionWatermark}} the test
sink will drop the record whose rowtime is late than watermark. If the consumer
reads the partition where number of records in the partition is 1 at last, the
sink will receive the valid watermark when all data has been emitted.
Therefore, we'd better to keep the data number in all partitions are more than
1.
Considering, we have already pushed watermark into the source, the issue
FLINK-19878 has been solved.
was:
Currenlty, we have already pushed watermark into the kafka table source. If the
consumer doesn't get data from each partition, it will not output a valid
watermark. In the test {{KafkaTableITCase#testPerPartitionWatermark}} the test
sink will drop the record whose rowtime is late than watermark. If the consumer
reads the partition where number of records in the partition is 1, the sink
will receive the valid watermark when all data has been emitted. Therefore,
we'd better to keep the data number in all partitions are more than 1.
Considering, we have already pushed watermark into the source, the issue
FLINK-19878 has been solved.
> Improve kafka per-partition-watermark tests
> -------------------------------------------
>
> Key: FLINK-20058
> URL: https://issues.apache.org/jira/browse/FLINK-20058
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka, Table SQL / API
> Affects Versions: 1.12.0
> Reporter: Shengkai Fang
> Priority: Major
> Fix For: 1.12.0
>
>
> Currenlty, we have already pushed watermark into the kafka table source. If
> the consumer doesn't get data from each partition, it will not output a valid
> watermark. In the test {{KafkaTableITCase#testPerPartitionWatermark}} the
> test sink will drop the record whose rowtime is late than watermark. If the
> consumer reads the partition where number of records in the partition is 1 at
> last, the sink will receive the valid watermark when all data has been
> emitted. Therefore, we'd better to keep the data number in all partitions are
> more than 1.
>
> Considering, we have already pushed watermark into the source, the issue
> FLINK-19878 has been solved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
