owenmonn opened a new issue, #3538: URL: https://github.com/apache/amoro/issues/3538
### What happened? When building Amoro with the Hadoop2 profile, an error occurs when attempting to list tables from a Hive 4 Metastore catalog. This is due to [changes](https://issues.apache.org/jira/browse/HIVE-24608) in the Hive Metastore Client API specifications in the currently used Hive dependency (`2.3.9`) for the Hadoop2 build. I will submit a PR to change the Hive dependency for the Hadoop2 build to `2.3.8`. ### Affects Versions master ### What table formats are you seeing the problem on? _No response_ ### What engines are you seeing the problem on? AMS ### How to reproduce 1. Build Amoro with the Hadoop2 profile. 2. In the AMS web interface, register a Hive 4 catalog (Type: External, Metastore: Hive Metastore). 3. In the AMS web interface, navigate to the Tables tab and select the registered catalog (to list tables). ### Relevant log output ```shell 2025-04-03 04:15:11,896 ERROR [JettyServerThreadPool-56909] [org.apache.amoro.server.dashboard.DashboardServer] [] - An error occurred while processing the url:http://myhost/ams/v1/catalogs/bigdata_dev_iceberg/databases/default/tables java.lang.RuntimeException: Failed to list all tables under namespace default at org.apache.iceberg.hive.HiveCatalog.listTables(HiveCatalog.java:158) ~[iceberg-hive-metastore-1.4.3.jar:?] at org.apache.iceberg.CachingCatalog.listTables(CachingCatalog.java:135) ~[iceberg-core-1.4.3.jar:?] at org.apache.amoro.formats.iceberg.IcebergCatalog.lambda$listTables$5(IcebergCatalog.java:94) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.table.TableMetaStore.call(TableMetaStore.java:256) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.table.TableMetaStore.lambda$doAs$0(TableMetaStore.java:231) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_422] at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_422] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1918) ~[hadoop-common-2.10.2.jar:?] at org.apache.amoro.table.TableMetaStore.doAs(TableMetaStore.java:231) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.formats.iceberg.IcebergCatalog.listTables(IcebergCatalog.java:92) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.CommonUnifiedCatalog.listTables(CommonUnifiedCatalog.java:146) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.server.catalog.ExternalCatalog.lambda$listTables$10(ExternalCatalog.java:122) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.table.TableMetaStore.call(TableMetaStore.java:256) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.table.TableMetaStore.lambda$doAs$0(TableMetaStore.java:231) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_422] at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_422] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1918) ~[hadoop-common-2.10.2.jar:?] at org.apache.amoro.table.TableMetaStore.doAs(TableMetaStore.java:231) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.server.catalog.ExternalCatalog.doAs(ExternalCatalog.java:161) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.server.catalog.ExternalCatalog.listTables(ExternalCatalog.java:119) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating] at org.apache.amoro.server.dashboard.controller.TableController.getTableList(TableController.java:509) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating] at io.javalin.core.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:20) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServlet.addHandler$lambda-0(JavalinServlet.kt:96) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:99) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8] at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137) ~[?:1.8.0_422] at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServletHandler.executeNextTask$lambda-11$lambda-10(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996) ~[?:1.8.0_422] at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8] at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137) ~[?:1.8.0_422] at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServletHandler.executeNextTask$lambda-11$lambda-10(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996) ~[?:1.8.0_422] at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8] at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995) ~[?:1.8.0_422] at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137) ~[?:1.8.0_422] at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8] at io.javalin.http.JavalinServlet.service(JavalinServlet.kt:89) ~[javalin-4.6.8.jar:4.6.8] at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) ~[jakarta.servlet-api-4.0.3.jar:4.0.3] at io.javalin.jetty.JavalinJettyServlet.service(JavalinJettyServlet.kt:58) ~[javalin-4.6.8.jar:4.6.8] at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) ~[jakarta.servlet-api-4.0.3.jar:4.0.3] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at io.javalin.jetty.JettyServer$start$wsAndHttpHandler$1.doHandle(JettyServer.kt:52) ~[javalin-4.6.8.jar:4.6.8] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_422] Caused by: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_objects_by_name' at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[libthrift-0.9.3.jar:0.9.3] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) ~[libthrift-0.9.3.jar:0.9.3] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_objects_by_name(ThriftHiveMetastore.java:1544) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_objects_by_name(ThriftHiveMetastore.java:1530) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableObjectsByName(HiveMetaStoreClient.java:1363) ~[hive-metastore-2.3.9.jar:2.3.9] at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_422] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_422] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:169) ~[hive-metastore-2.3.9.jar:2.3.9] at com.sun.proxy.$Proxy51.getTableObjectsByName(Unknown Source) ~[?:?] at org.apache.iceberg.hive.HiveCatalog.lambda$listTables$2(HiveCatalog.java:133) ~[iceberg-hive-metastore-1.4.3.jar:?] at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:58) ~[iceberg-core-1.4.3.jar:?] at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:51) ~[iceberg-core-1.4.3.jar:?] at org.apache.iceberg.hive.CachedClientPool.run(CachedClientPool.java:122) ~[iceberg-hive-metastore-1.4.3.jar:?] at org.apache.iceberg.hive.HiveCatalog.listTables(HiveCatalog.java:133) ~[iceberg-hive-metastore-1.4.3.jar:?] ... 81 more ``` ### Anything else _No response_ ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's Code of Conduct -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
