TyrantLucifer commented on issue #1946: URL: https://github.com/apache/incubator-seatunnel/issues/1946#issuecomment-1222182048
> > > > > > > > > > > > > Hi, Brothers. I'm having some difficulties with the submission of the second segment of the sink,I went to consult the HDFS sink Connector,Its precommit is to write data to a temporary file,If the pre-commit is successful, the file name is changed to achieve the actual commit data ,If this fails, the temporary file data is deleted. I don't quite understand what method should be implemented in the pre-commit, what method should be selected for the final commit, My initial idea is that if the sink target side supports the transaction mechanism, it can be implemented using the transaction mechanism. If there is no transaction mechanism, it can only be implemented through a similar HDFS sink connector.The problem is that I can't find a way to rename files in the Minio Client API ,But Minio has a CopyObject API for submitting data a second time;I don't know whether it is correct to implement S3 sink in this way, and whether it is correct to understand the two-phase commit > > > > > > @Hisoka-X Please assign s3 source & sink to me, I can do this work. > > @TyrantLucifer Hi, can you answer this question to help @2013650523 finish this work? If he can do it, I think we can help it, if he can't finish this, he can give you this issue. I think this will be better. Thanks! We have communicated privately and for him the current implementation of s3 sink and source some difficulties, he can first focus on writing iceberg, doris these connectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
