brightwon opened a new issue, #3787:
URL: https://github.com/apache/iceberg-python/issues/3787
### Apache Iceberg version
0.11.0 (latest release)
### Please describe the bug 🐞
When connecting to a Kerberos-enabled Hive Metastore, authentication fails
if the service
principal's host does not match the host in the connection URI, and there is
no setting to
correct it.
I connect to the metastore through three HA hosts:
```python
from pyiceberg.catalog import load_catalog
catalog = load_catalog("hive", **{
"type": "hive",
"uri": "thrift://myhms1:9083,thrift://myhms2:9083,thrift://myhms3:9083",
"hive.kerberos-authentication": "true",
"hive.kerberos-service-name": "myservice",
})
catalog.list_namespaces()
```
The last line fails with:
```
Traceback (most recent call last):
File "test.py", line 24, in <module>
ns = catalog.list_namespaces()
File ".../pyiceberg/catalog/hive.py", line 769, in list_namespaces
with self._client as open_client:
File ".../pyiceberg/catalog/hive.py", line 180, in __enter__
self._transport.open()
File ".../thrift/transport/TTransport.py", line 382, in open
initial_response = self.sasl.process()
File ".../puresasl/client.py", line 148, in process
return self._chosen_mech.process(challenge)
File ".../puresasl/mechanisms.py", line 505, in process
kerberos.authGSSClientStep(self.context, '')
kerberos.GSSError
```
Enabling `KRB5_TRACE` shows that the failure happens when the client asks
the KDC for a
service ticket:
```
set-error: -1765328243: Did not find credential for
myservice/[email protected] in cache FILE:...
set-error: -1765328377: Error from KDC: LOOKING_UP_SERVER while looking up
'myservice/[email protected]'
```
My Hive Metastore's service principal is `myservice/[email protected]`.
PyIceberg,
however, requests a ticket using my HMS server hostname (`myhms1`) as the
hostname
component, which is not a valid service principal, so the request fails.
A Kerberos service principal has the form `Service/Hostname@REALM` (see
[3.2 Principal](https://www.kerberos.org/software/tutorial.html#1.3.2) in
the Kerberos
tutorial), and with SASL/GSSAPI
those two components come from the `service` and `host` arguments passed to
the client.
PyIceberg always derives that `host` from the connection URI, so it is
forced to be whichever
host you connect to. `hive.kerberos-service-name` (added in #2141) makes the
`Service` part
configurable, but there is no equivalent for `Hostname`.
The relevant line is
[`pyiceberg/catalog/hive.py#L167`](https://github.com/apache/iceberg-python/blob/pyiceberg-0.11.x/pyiceberg/catalog/hive.py#L167)
(`_HiveClient._init_thrift_transport`):
```python
return TTransport.TSaslClientTransport(socket, host=url_parts.hostname,
service=self._kerberos_service_name)
```
There is no way to avoid this from the client side. The `uri` has to contain
the hostnames I
actually connect to, and the principal's hostname component is not one of
them. Listing multiple URIs
does not help either, because the client is built from the first URI and the
failure happens
later, during authentication.
So I patch that line locally and hardcode the host:
```python
-return TTransport.TSaslClientTransport(socket, host=url_parts.hostname,
service=self._kerberos_service_name)
+return TTransport.TSaslClientTransport(socket, host="hive-host",
service=self._kerberos_service_name)
```
With that one change everything works well. PyIceberg requests
`myservice/[email protected]` and `catalog.list_namespaces()` succeeds.
I would like to propose a new configuration property,
`hive.kerberos-service-host`. It could
be added in the same way as
[#2141](https://github.com/apache/iceberg-python/pull/2141). When
the property is not set, the URI host would be used as the default, so the
current behavior is
preserved for backward compatibility.
I have finished the implementation including unit tests, and verified it
end-to-end against
the same metastore.
If this is welcome, I would like to submit the PR myself.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
