Hi Experts,
When we implement a sink, usually we implement a batch, according to
the record number or when reaching a time interval, however this may lead to
data of last batch do not write to sink. Because it is triggered by the
incoming record.
I also test the JDBCOutputFormat provided by flink, and found that it
also has the same problem. If the batch size is 50, and 49 items arrive, but
the last one comes in an hour later, then the 49 items will not be written to
sink during the one hour. This may cause data delay or data loss.
So should any pose a solution to this problem?
Thanks a lot.
Best
Henry
