[ https://issues.apache.org/jira/browse/FLINK-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhangminglei updated FLINK-9609: -------------------------------- Description: Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users want to do some extra work when a bucket is ready. It would be nice if we can support {{BucketReady}} mechanism for users or we can tell users when a bucket is ready for use. For example, One bucket is created for every 5 minutes, at the end of 5 minutes before creating the next bucket, the user might need to do something as the previous bucket ready, like sending the timestamp of the bucket ready time to a server or do some other stuff. Here, Bucket ready means all the part files name suffix under a bucket neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is ready for user use. was: Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users want to do some extra work when a bucket is ready. It would be nice if we can support {{BucketReady}} mechanism for users or we can tell users when a bucket is ready for user use. For example, One bucket is created for every 5 minutes, at the end of 5 minutes before creating the next bucket, the user might need to do something as the previous bucket ready, like sending the timestamp of the bucket ready time to a server or do some other stuff. Here, Bucket ready means all the part files name suffix under a bucket neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is ready for user use. > Add bucket ready mechanism for BucketingSink when checkpoint complete > --------------------------------------------------------------------- > > Key: FLINK-9609 > URL: https://issues.apache.org/jira/browse/FLINK-9609 > Project: Flink > Issue Type: New Feature > Components: filesystem-connector, Streaming Connectors > Affects Versions: 1.5.0, 1.4.2 > Reporter: zhangminglei > Assignee: zhangminglei > Priority: Major > > Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users > want to do some extra work when a bucket is ready. It would be nice if we can > support {{BucketReady}} mechanism for users or we can tell users when a > bucket is ready for use. For example, One bucket is created for every 5 > minutes, at the end of 5 minutes before creating the next bucket, the user > might need to do something as the previous bucket ready, like sending the > timestamp of the bucket ready time to a server or do some other stuff. > Here, Bucket ready means all the part files name suffix under a bucket > neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is > ready for user use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)