Matthew McMahon created FLINK-17900:
---------------------------------------
Summary: Listener for when Files are committed in StreamingFileSink
Key: FLINK-17900
URL: https://issues.apache.org/jira/browse/FLINK-17900
Project: Flink
Issue Type: Improvement
Affects Versions: 1.10.0
Reporter: Matthew McMahon
I have a scenario where once a file has been committed to S3 using the
StreamingFileSink (I am using Java), that I should notify some downstream
services.
The idea is to produce a message on a Kafka topic for the files as they are
finalized.
I am currently looking into how this might be possible, and considering using
reflection and/or checking the S3 bucket before/after the checkpoint is
committed.
Still trying to find a suitable way.
However I was thinking it would be great if possible to register a listener
that can be fired when StreamingFileSink commits a file.
Does something like this exist, that I'm not aware of (new to flink) or could
it be added?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
