Zevrap-81 opened a new issue, #3117:
URL: https://github.com/apache/iceberg-python/issues/3117
### Apache Iceberg version
0.11.0 (latest release)
### Please describe the bug 🐞
Hello,
I am using a setup with pyiceberg and polaris as rest catalog along with
Rustfs for S3 filestorage type. When i am getting this error during table
commits.
```
OSError: When completing multiple part upload for key
'Senec/Stage/Bronze/Ampace_P4/BmsAmpaceModules/data/month_timestamp=2025-09/bucket_mainControllerSerial=7/bucket_moduleSerial=17/00000-0-ab52d0d3-77d2-4d72-b10f-146e1d40e866.parquet'
in bucket 'senec-bucket': AWS Error NO_SUCH_UPLOAD during
CompleteMultipartUpload operation: The specified multipart upload does not
exist. The upload ID may be invalid, or the upload may have been aborted or
completed.
```
And here is the entire traceback:
```
Traceback (most recent call last):
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/scripts/transfer_monthly.py",
line 205, in <module>
transfer_lfp_data_for_module(
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/scripts/transfer_monthly.py",
line 147, in transfer_lfp_data_for_module
iceberg_client.write(
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/novum_iceberg/iceberg_client.py",
line 428, in write table.append(pa_table)
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/table/__init__.py",
line 1427, in append
tx.append(df=df, snapshot_properties=snapshot_properties, branch=branch)
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/table/__init__.py",
line 523, in append
data_files =
list(
^^^^^
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/io/pyarrow.py",
line 2875, in _dataframe_to_data_files
yield from write_file(
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 619, in
result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 317, in
_result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in
__get_result
raise self._exception
File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/io/pyarrow.py",
line 2666, in write_parquet
with fo.create(overwrite=True) as fos:
File "pyarrow/io.pxi", line 136, in pyarrow.lib.NativeFile.__exit__
File "pyarrow/io.pxi", line 206, in pyarrow.lib.NativeFile.close
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
OSError: When completing multiple part upload for key
'Senec/Stage/Bronze/Ampace_P4/BmsAmpaceModules/data/month_timestamp=2025-09/bucket_mainControllerSerial=7/bucket_moduleSerial=17/00000-0-ab52d0d3-77d2-4d72-b10f-146e1d40e866.parquet'
in bucket 'senec-bucket': AWS Error NO_SUCH_UPLOAD during
CompleteMultipartUpload operation: The specified multipart upload does not
exist. The upload ID may be invalid, or the upload may have been aborted or
completed.
```
I have tried to find what is causing the issue but with no luck. I would
appreciate it if someone can point to me what could be the problem and a way to
fix this issue.
I am working on creating a MCE for this.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
