carc-prathyush-shankar opened a new issue, #2978:
URL: https://github.com/apache/polaris/issues/2978
### Describe the bug
## Environment
- Polaris Version: 1.2.0
- Deployment method: Kubernetes (EKS)
- Client: StarRocks 3.5
- Storage: AWS S3
## Setup
Polaris is configured to federate to an existing Hive Metastore containing
Iceberg tables with `s3a://` storage paths.
## Problem
When querying tables through the external catalog, metadata operations work
(`SHOW DATABASES FROM polaris`), but data queries fail with
`ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not
found`.
## Example Table Definition
The tables in the federated Hive Metastore were created with `s3a://` paths:
```
CREATE TABLE `map_attribute` (
`sc_attribute_id` bigint(20) DEFAULT NULL,
`sc_frame` varchar(1048576) DEFAULT NULL
)
PROPERTIES ("owner" = "***", "write.spark.accept-any-schema" = "true",
"location" = "s3a://cake-cupcake/warehouse/blah.db/map_attribute");
```
## Stack Trace
```
2025-11-04 23:47:49,994 INFO [org.apa.pol.ser.exc.IcebergExceptionMapper]
[67fe3bec-de05-486b-a6c3-9b39d6ef2c16_0000000000000000149,POLARIS] [,,,]
(executor-thread-63) Handling runtimeException
java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem
not found
2025-11-04 23:47:49,995 ERROR [org.apa.pol.ser.exc.IcebergExceptionMapper]
[67fe3bec-de05-486b-a6c3-9b39d6ef2c16_0000000000000000149,POLARIS] [,,,]
(executor-thread-63) Unhandled exception returning INTERNAL_SERVER_ERROR:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.s3a.S3AFileSystem not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2737)
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3569)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3612)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:172)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3716)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3667)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:366)
at org.apache.iceberg.hadoop.Util.getFs(Util.java:55)
at
org.apache.iceberg.hadoop.HadoopInputFile.fromLocation(HadoopInputFile.java:56)
at
org.apache.iceberg.hadoop.HadoopFileIO.newInputFile(HadoopFileIO.java:87)
at
org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:294)
at
org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$0(BaseMetastoreTableOperations.java:180)
at
org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$1(BaseMetastoreTableOperations.java:199)
at
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
at
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
at
org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:199)
at
org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:176)
at
org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:171)
at
org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:131)
at
org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:88)
at
org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:71)
at
org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:49)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogHandler.loadTable(IcebergCatalogHandler.java:767)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter.lambda$loadTable$8(IcebergCatalogAdapter.java:434)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter.withCatalog(IcebergCatalogAdapter.java:201)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter.loadTable(IcebergCatalogAdapter.java:429)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter_Subclass.loadTable$$superforward(Unknown
Source)
at
org.apache.polaris.service.catalog.iceberg.IcebergRestCatalogEventServiceDelegator_Gj_WCptqTcdHu-fbZfgVkAwPXCI_Delegate_Subclass.loadTable(Unknown
Source)
at
org.apache.polaris.service.catalog.iceberg.IcebergRestCatalogEventServiceDelegator.loadTable(IcebergRestCatalogEventServiceDelegator.java:268)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter_Subclass.loadTable(Unknown
Source)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter_ClientProxy.loadTable(Unknown
Source)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi.loadTable(IcebergRestCatalogApi.java:565)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi_Subclass.loadTable$$superforward(Unknown
Source)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi_Subclass$$function$$13.apply(Unknown
Source)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$8(FaultToleranceInterceptor.java:364)
at io.smallrye.faulttolerance.core.Future.from(Future.java:85)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$9(FaultToleranceInterceptor.java:364)
at
io.smallrye.faulttolerance.core.FaultToleranceContext.call(FaultToleranceContext.java:20)
at io.smallrye.faulttolerance.core.Invocation.apply(Invocation.java:29)
at
io.smallrye.faulttolerance.core.metrics.MetricsCollector.apply(MetricsCollector.java:98)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.syncFlow(FaultToleranceInterceptor.java:367)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.intercept(FaultToleranceInterceptor.java:205)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at
io.quarkus.micrometer.runtime.MicrometerTimedInterceptor.timedMethod(MicrometerTimedInterceptor.java:79)
at
io.quarkus.micrometer.runtime.MicrometerTimedInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at
io.quarkus.security.runtime.interceptor.SecurityHandler.handle(SecurityHandler.java:27)
at
io.quarkus.security.runtime.interceptor.RolesAllowedInterceptor.intercept(RolesAllowedInterceptor.java:29)
at
io.quarkus.security.runtime.interceptor.RolesAllowedInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
at
io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:44)
at
io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_RolesAllowedInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
at
io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi_Subclass.loadTable(Unknown
Source)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi$quarkusrestinvoker$loadTable_56b49d5a1874f2749e5229d18bb00aeb8fe7fdc8.invoke(Unknown
Source)
at
org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
at
io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:183)
at
org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
at
io.quarkus.vertx.core.runtime.VertxCoreRecorder$15.runWith(VertxCoreRecorder.java:645)
at
org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2651)
at
org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2630)
at
org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1622)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1589)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
at
org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.s3a.S3AFileSystem not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2641)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2735)
... 80 more
```
## Question
Is this expected behavior? Should S3-related dependencies be bundled with
Polaris for federated catalog support, or is there additional configuration
needed to enable S3A filesystem support?
### To Reproduce
1. Create external catalog in StarRocks that federates to Hive:
```
CREATE EXTERNAL CATALOG `polaris`
PROPERTIES ("iceberg.catalog.scope" = "PRINCIPAL_ROLE:ALL",
"iceberg.catalog.credential" = "****",
"iceberg.catalog.uri" = "https://****/api/catalog";,
"iceberg.catalog.warehouse" = "****",
"iceberg.catalog.security" = "oauth2",
"type" = "iceberg",
"iceberg.catalog.type" = "rest"
);
```
2. Verify that metadata queries work: `SHOW DATABASES FROM polaris;`
4. Attempt to query a table: `SELECT * FROM polaris.database.table;`
### Actual Behavior
_No response_
### Expected Behavior
_No response_
### Additional context
_No response_
### System information
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
