aprilApples commented on issue #5952: URL: https://github.com/apache/seatunnel/issues/5952#issuecomment-2753611112
You can call "snapshotState" in method "prepareCommit" like this:  Then, it will create new file based on the correct checkpointId each transaction. However, this will bring up a new issue : too many small files will be generated in HDFS. You can append these small files to a larger file in the FileSinkAggregatedCommitter, which is what I did. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
