[
https://issues.apache.org/jira/browse/FLINK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lihe ma closed FLINK-22918.
---------------------------
Resolution: Not A Bug
this situation happens when flink job is misconfigured without checkpoint or
savepoint, the pending partition message was discarded, jobmanager cannot
commit previous partitions.
> StreamingFileSink does not commit partition when no message is sent
> --------------------------------------------------------------------
>
> Key: FLINK-22918
> URL: https://issues.apache.org/jira/browse/FLINK-22918
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.11.3
> Reporter: lihe ma
> Priority: Minor
>
> FileSink extract the partitions from messages , then send partition list to
> committer. In dynamic partition case, if one partition contains no data
> between two checkpoints. then the partition wont commit.
> like this:
> date=0101/hour=01/key=a/part1-file (write at 01-01 01:20:00)
> if there is no data with key=a between 01:20 and 02:10 (a time when watermark
> passed) , the committer will not receive this partition and it will never
> commit.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
