The GitHub Actions job "Flink Connector Kafka CI" on 
flink-connector-kafka.git/main has failed.
Run started by GitHub user bowenli86 (triggered by bowenli86).

Head commit for run:
a7e805a339964b795d1b805527cf3f68117e3b65 / lnbest0707 
<[email protected]>
[FLINK-39238] Support watermark alignment in dynamic Kafka reader (#240)

Flink source has watermark alignment config like 
scan.watermark.alignment.max-drift. The operator would then control:

- checkWatermarkAlignment for whole reader
- checkSplitWatermarkAlignment for the split reader

In dynamic Kafka source, the split reader pause/resume is missed 
(pauseOrResumeSplits not implemented) and the exceptions might raise once such 
config enabled.

This feature is important for workflows like ingesting data from Kafka to Data 
Lake. Missing the alignment could create far more number of small files once 
the ingestion progress diverges and runs our of controls.

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/23222170769

With regards,
GitHub Actions via GitBox

Reply via email to