mbutrovich opened a new issue, #4269:
URL: https://github.com/apache/iceberg-python/issues/4269
### Apache Iceberg version
0.12.0 (latest release)
### Please describe the bug 🐞
Appending an empty Arrow table creates a snapshot whose manifest list has no
manifests. After that, `inspect.files()`, `data_files()`, `delete_files()`,
`all_files()`, `all_data_files()`, and `all_delete_files()` all raise
`ArrowInvalid: Must pass at least one table`. I expected each of them to return
an empty table with the files schema.
```python
import tempfile
import pyarrow as pa
from pyiceberg.catalog.sql import SqlCatalog
warehouse = tempfile.mkdtemp()
catalog = SqlCatalog("t", uri=f"sqlite:///{warehouse}/catalog.db",
warehouse=f"file://{warehouse}")
catalog.create_namespace("n")
df = pa.table({"x": pa.array([], pa.int64())})
table = catalog.create_table("n.t", schema=df.schema)
table.append(df)
table.inspect.files() # ArrowInvalid: Must pass at least one table
```
```
Traceback (most recent call last):
File "repro.py", line 12, in <module>
table.inspect.files() # ArrowInvalid: Must pass at least one table
^^^^^^^^^^^^^^^^^^^^^
File "pyiceberg/table/inspect.py", line 900, in files
return self._files(snapshot_id)
^^^^^^^^^^^^^^^^^^^^^^^^
File "pyiceberg/table/inspect.py", line 888, in _files
return pa.concat_tables(results)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/table.pxi", line 6344, in pyarrow.lib.concat_tables
File "pyarrow/error.pxi", line 155, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Must pass at least one table
```
`overwrite()` with an empty table after a non-empty append also leaves a
snapshot with no manifests, and `files()` fails the same way.
`_files` reads one table per manifest and ends with
[`pa.concat_tables(results)`](https://github.com/apache/iceberg-python/blob/e3564a54c10960307147377c578680276865ed18/pyiceberg/table/inspect.py#L888),
so an empty manifest list calls `pa.concat_tables([])`. `_all_files` does the
same at
[L981](https://github.com/apache/iceberg-python/blob/e3564a54c10960307147377c578680276865ed18/pyiceberg/table/inspect.py#L981).
Both already return an empty table one level up: `_files` when the table has
no current snapshot
([L876-L877](https://github.com/apache/iceberg-python/blob/e3564a54c10960307147377c578680276865ed18/pyiceberg/table/inspect.py#L876-L877)),
and `_all_files` when the table has no snapshots
([L968-L970](https://github.com/apache/iceberg-python/blob/e3564a54c10960307147377c578680276865ed18/pyiceberg/table/inspect.py#L968-L970)).
Could they return the same empty table when there are no manifests?
This reproduces on 0.12.0 and on `main` at
e3564a54c10960307147377c578680276865ed18.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
