DG47 opened a new issue, #4111:
URL: https://github.com/apache/iceberg-python/issues/4111
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
`RestCatalog.list_namespaces(parent)` URL-encodes each part of the parent
namespace with `quote()` (via `_encode_namespace_path`) and then hands the
result to `requests` as a query parameter, which encodes it again. Any
namespace part that contains a space, `/`, `%` or a non-ASCII character is
therefore sent double-encoded, and the server looks up the wrong parent
namespace (typically a 404 / `NoSuchNamespaceError`, or an empty listing).
This started when list_namespaces moved from building the query string by
hand to `params=` in #3347.
Minimal reproduction (no server required):
```python
from urllib.parse import parse_qs, urlsplit
import requests_mock
from pyiceberg.catalog.rest import RestCatalog
with requests_mock.Mocker() as m:
m.get("http://srv/v1/config";, json={"defaults": {}, "overrides": {}})
m.get("http://srv/v1/namespaces";, json={"namespaces": []})
cat = RestCatalog("r", uri="http://srv";, token="t")
for ns in [("my namespace",), ("dept/finance",), ("café",), ("100%",)]:
cat.list_namespaces(ns)
url = m.request_history[-1].url
print(ns, url, parse_qs(urlsplit(url).query)["parent"])
```
Output:
```
('my namespace',) http://srv/v1/namespaces?parent=my%2520namespace
['my%20namespace']
('dept/finance',) http://srv/v1/namespaces?parent=dept%252Ffinance
['dept%2Ffinance']
('café',) http://srv/v1/namespaces?parent=caf%25C3%25A9 ['caf%C3%A9']
('100%',) http://srv/v1/namespaces?parent=100%2525 ['100%25']
```
**Expected:** after the server decodes the query string, `parent` is the
namespace parts joined by the namespace separator (`my namespace`,
`dept/finance`, ...). This is what the Java client sends:
`RESTUtil.namespaceToQueryParam` joins the raw levels with the separator and
leaves encoding to the HTTP client, and the server parses the value with
`RESTUtil.namespaceFromQueryParam` without decoding it again.
**Actual:** the server receives `my%20namespace`, `dept%2Ffinance`, and so
on. Plain ASCII names such as `accounting` are unaffected, which is why the
existing tests pass.
Environment: pyiceberg main (b32392d), Python 3.12.13, requests 2.34.2,
macOS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
