tyoushinya opened a new pull request, #9116:
URL: https://github.com/apache/paimon/pull/9116
…Kerberos-enabled environment
### Purpose
When querying Paimon external tables via Hive in a Kerberos-secured cluster
with proxy user (e.g., Hue, JDBC through HiveServer2), the query fails with:
org.apache.hadoop.hive.ql.parse.SemanticException: Proxy user is not
supported
at
org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer.transform(SimpleFetchOptimizer.java:125)
The root cause is that PaimonStorageHandler#getDataFieldsJsonStr passes null
as the Configuration to HiveSchema.extract:
// Before
```
static String getDataFieldsJsonStr(Properties properties) {
HiveSchema hiveSchema = HiveSchema.extract(null, properties);
...
}
```
With a null Configuration:
Kerberos credentials are invisible: HiveUtils.extractCatalogConfig(null)
returns an empty Options, so Paimon security settings (e.g.,
paimon.security.kerberos.login.keytab,
paimon.security.kerberos.login.principal) configured in Hadoop Configuration
are not loaded.
Filesystem initialization fails silently: FileIO.get() cannot obtain a
properly authenticated filesystem under the proxy user's doAs context. The
schema file fetch silently fails, causing SimpleFetchOptimizer to fall back to
a direct-access code path that rejects proxy users.
The fix passes the actual Configuration instance (which carries Kerberos
credentials and proxy user context) through the call chain:
// After
```
static String getDataFieldsJsonStr(Configuration conf, Properties
properties) {
HiveSchema hiveSchema = HiveSchema.extract(conf, properties);
...
}
```
This ensures HiveSchema.extract can properly initialize the filesystem with
Kerberos authentication under the proxy user's UGI, allowing the schema to be
read correctly and the query to proceed via the normal MR/Tez execution path
instead of falling back to SimpleFetchOptimizer.
### Tests
Added 4 unit tests in HiveTableSchemaTest:
1. testGetDataFieldsJsonStrWithNullConf: Verifies JSON output correctness
when conf is null (backward-compatible, local FS).
2. testGetDataFieldsJsonStrWithConf: Verifies JSON output correctness when
conf is a real Configuration instance (the fixed code path).
3. testGetDataFieldsJsonStrWithEmptyDDLAndPaimonTable: Verifies JSON
roundtrip with empty DDL and an existing Paimon table schema.
4. testGetDataFieldsJsonStrRoundtrip: Verifies that serializing and
deserializing the JSON output preserves field names, types, and comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
