mapleFU commented on issue #40800: URL: https://github.com/apache/arrow/issues/40800#issuecomment-2020904993
S3 uses "Part Upload": 1. Output Stream will change to "part", in S3, a part might be 5MB 2. During closing the "part", writer might call "commit" to commit the part I'm not familiar with Python API, but generally, arrow fs also has a "abort", which could abort the file written in parts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
