maomaodev opened a new pull request, #7718:
URL: https://github.com/apache/kyuubi/pull/7718
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
2. If the PR is related to an issue in
https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title,
e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new feature, clarify the use cases.
2. If you fix a bug, you can clarify why it is a bug, and what versions
are affected.
-->
The thrift `GetSchemas`/`GetTables` operations assumed `spark_catalog` is
always the builtin V1 session catalog and listed databases/tables via
`spark.sessionState.catalog`. When a custom session catalog is configured, e.g.
```
spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog
```
these operations bypass the DSv2 catalog APIs
(`SupportsNamespaces`/`TableCatalog`) and the V1 session catalog path cannot
even parse the V2 table metadata the custom catalog persists in HMS - it may
return wrong results, or throw and fail the whole request. #7673 is a
real-world example: `GetTables` fails with `ClassNotFoundException:
org.apache.iceberg.mr.hive.HiveIcebergSerDe` when the V1 `HiveExternalCatalog`
tries to load tables registered by Iceberg with `storage_handler =
org.apache.iceberg.mr.hive.HiveIcebergStorageHandler`.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some
test cases that check the changes thoroughly including negative and positive
cases if possible.
If it was tested in a way different from regular unit tests, please clarify
how you tested step by step, ideally copy and paste-able, so that other
reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why
it was difficult to add.
-->
1. Added test coverage: `DeltaMetadataTests#get tables and views` (v1 view
path), `IcebergMetadataTests#get tables and views` (v2 view path), and
`IcebergMetadataTests#get tables` (both `spark_catalog` and `hadoop_prod`). A
testcontainers-based `WithSimpleHMSContainer` is wired into
`SparkIcebergOperationSuite` and `IcebergOperationSuite` so these run against a
real Hive metastore (`catalogImplementation=hive`).
2. Manually verified against Iceberg 1.10.1 (which removed
`iceberg-hive-runtime`): `beeline ... -e "!tables"` previously threw
`ClassNotFoundException: org.apache.iceberg.mr.hive.HiveIcebergStorageHandler`,
and now returns the table as `TABLE` and v1/v2 views as `VIEW`.
### Was this patch assisted by generative AI tooling?
<!--
If generative AI tooling has assisted in authoring or reviewing this patch,
please include
an 'Assisted-by: ' trailer that identifies the agent and/or model.
For example: 'Assisted-by: Claude Opus 4.7' or 'Assisted-by:
Claude:claude-opus-4-7'.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
Assisted-by: DeepSeek-V4-Pro
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
