DG47 opened a new issue, #4163:
URL: https://github.com/apache/iceberg-python/issues/4163
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
`SqlCatalog.namespace_exists` treats a namespace as existing when a child
namespace exists, and checks that with `LIKE '<namespace>.%'`. SQLite evaluates
`LIKE` case-insensitively for ASCII characters, so on SQLite a namespace that
differs only in case from the parent of an existing namespace is reported as
existing. The exact-match lookups elsewhere in the catalog (`==`) are
case-sensitive, so the catalog ends up inconsistent with itself, and the result
differs from PostgreSQL, where `LIKE` is case-sensitive.
Because `create_table`, `register_table`, `rename_table`, `drop_namespace`,
`list_tables`, `list_namespaces` and the namespace property methods all go
through `namespace_exists`, the effects show up across the API:
```python
import tempfile
from pyiceberg.catalog.sql import SqlCatalog
from pyiceberg.schema import Schema
from pyiceberg.types import NestedField, LongType
w = tempfile.mkdtemp()
cat = SqlCatalog("t", uri=f"sqlite:///{w}/c.db", warehouse=f"file://{w}")
cat.create_namespace("sales.eu")
print(cat.namespace_exists("SALES")) # True, expected False
print(cat.create_table("SALES.orders", schema=Schema(NestedField(1, "id",
LongType()))).name())
# ('SALES', 'orders'), expected NoSuchNamespaceError
```
On a fresh catalog with only `sales.eu`, `cat.create_namespace("SALES")`
raises `NamespaceAlreadyExistsError: Namespace SALES already exists`, and
`cat.load_namespace_properties("SALES")` returns `{}` instead of raising
`NoSuchNamespaceError`.
Expected: `namespace_exists("SALES")` returns False when only `sales` /
`sales.eu` exist, matching the case-sensitive exact lookups and the PostgreSQL
behaviour. `create_table("SALES.orders")` raises `NoSuchNamespaceError` and
`create_namespace("SALES")` succeeds.
Without a nested namespace the check is correct (`namespace_exists("SALES")`
is False when only `sales` exists), because only the `LIKE` branch is affected.
Environment: pyiceberg main (0.12.0 dev), SQLAlchemy 2.0.54, SQLite 3.50.4,
Python 3.12, macOS arm64.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
