tustvold commented on issue #6460: URL: https://github.com/apache/arrow-rs/issues/6460#issuecomment-2376660263
> Changes proposed by me should help mitigate this issue a bit by making shutdown calls relaxed Your proposed change is racy, the first close will wait on the upload to complete, with subsequent calls "completing" instantly. > For the second one, we're using an s3 object store and for some reason, it (s3) doesn't take into account the part number during the multipart upload and return error mentioned above. We managed to solve this issue by calling self.wait_for_capacity(0).await? before calling finish() Perhaps you could write a simple reproducer for this, I'm not saying S3 doesn't do this but I want to be sure we've correctly identified the issue. Your proposed fix will serialize a round-trip which is unfortunate when many stores are not exactly low-latency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
