thinkharderdev commented on issue #6934:
URL: https://github.com/apache/arrow-rs/issues/6934#issuecomment-2569360722
> > Nothing fails but the files are corrupted because the parts are in the
wrong order.
>
> That sounds to me like exactly the same issue, I'm not familiar with how
you are integrating the multipart upload, but my suspicion is whatever is
sinking to it is accidentally deferring the put_part to a spawned future, or
something similar.
>
> Correct
>
> ```
> tokio::spawn(upload.put_part(x));
> ```
>
> Incorrect
>
> ```
> tokio::spawn(async move {
> upload.lock().put_part(x);
> })
> ```
>
> I'd be very suspicious of anything that is using `RWLock` or similar
There is nothing like that. I can't share the code here because it is
internal, but I can assure you there is nothing weird going on. No spawned
tasks, no interior mutability just a plain vanilla usage of `BufWriter` as an
`AsyncWrite`. Conceptually it's just
```
let mut writer = BufWriter::new(store, path);
while something {
writer.write_all(&some_data).await?
}
writer.flush().await?;
writer.shutdown().await?;
```
I didn't want to create an issue until I have something concrete (and am
double-sure it's not a bug on our end even though I am quite confident it is
not) so I will report something when/if I have it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
