sathesuraj commented on issue #2207:
URL: https://github.com/apache/polaris/issues/2207#issuecomment-3487499048
@dimas-b - Below is the full stack trace of the error. I'll be happy to
add/update documentation once I am able to create and test tables with non-aws
s3 service without STS.
Also, please confirm my understanding about -
- When stsUnavailable is set to true, does Polaris use the default
credential provider chain to retrieve access keys and secrets from the
environment variable e.g. AWS_SHARED_CREDENTIALS_FILE ?
ERROR [org.apa.pol.ser.exc.IcebergExceptionMapper]
[f524228c-efb7-4f92-bee5-6fc355fc85e5_0000000000000000124,POLARIS] [,,,]
(executor-thread-1) Unhandled exception returning INTERNAL_SERVER_ERROR:
software.amazon.awssdk.core.exception.SdkClientException: Received an
UnknownHostException when attempting to interact with a service. See cause for
the exact endpoint that is failing to resolve. If this is happening on an
endpoint that previously worked, there may be a network connectivity issue or
your DNS cache could be storing endpoints for too long.
at
software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
at
software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor.modifyException(HelpfulUnknownHostExceptionInterceptor.java:59)
at
software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyException(ExecutionInterceptorChain.java:181)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.utils.ExceptionReportingUtils.runModifyException(ExceptionReportingUtils.java:54)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.utils.ExceptionReportingUtils.reportFailureToInterceptors(ExceptionReportingUtils.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:210)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:11883)
at
org.apache.iceberg.aws.s3.S3OutputStream.completeUploads(S3OutputStream.java:443)
at
org.apache.iceberg.aws.s3.S3OutputStream.close(S3OutputStream.java:269)
at
org.apache.iceberg.aws.s3.S3OutputStream.close(S3OutputStream.java:255)
at
java.base/sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:435)
at
java.base/sun.nio.cs.StreamEncoder.lockedClose(StreamEncoder.java:237)
at java.base/sun.nio.cs.StreamEncoder.close(StreamEncoder.java:222)
at
java.base/java.io.OutputStreamWriter.close(OutputStreamWriter.java:266)
at
org.apache.iceberg.TableMetadataParser.internalWrite(TableMetadataParser.java:135)
at
org.apache.iceberg.TableMetadataParser.overwrite(TableMetadataParser.java:119)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalog$BasePolarisTableOperations.writeNewMetadata(IcebergCatalog.java:1688)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalog$BasePolarisTableOperations.writeNewMetadataIfRequired(IcebergCatalog.java:1677)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalog$BasePolarisTableOperations.doCommit(IcebergCatalog.java:1546)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalog$BasePolarisTableOperations.commit(IcebergCatalog.java:1397)
at
org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:201)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogHandler.createTableDirect(IcebergCatalogHandler.java:444)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter.lambda$createTable$6(IcebergCatalogAdapter.java:384)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter.withCatalog(IcebergCatalogAdapter.java:201)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter.createTable(IcebergCatalogAdapter.java:368)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter_Subclass.createTable$$superforward(Unknown
Source)
at
org.apache.polaris.service.catalog.iceberg.IcebergRestCatalogEventServiceDelegator_Gj_WCptqTcdHu-fbZfgVkAwPXCI_Delegate_Subclass.createTable(Unknown
Source)
at
org.apache.polaris.service.catalog.iceberg.IcebergRestCatalogEventServiceDelegator.createTable(IcebergRestCatalogEventServiceDelegator.java:217)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter_Subclass.createTable(Unknown
Source)
at
org.apache.polaris.service.catalog.iceberg.IcebergCatalogAdapter_ClientProxy.createTable(Unknown
Source)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi.createTable(IcebergRestCatalogApi.java:193)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi_Subclass.createTable$$superforward(Unknown
Source)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi_Subclass$$function$$3.apply(Unknown
Source)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$8(FaultToleranceInterceptor.java:364)
at io.smallrye.faulttolerance.core.Future.from(Future.java:85)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$9(FaultToleranceInterceptor.java:364)
at
io.smallrye.faulttolerance.core.FaultToleranceContext.call(FaultToleranceContext.java:20)
at
io.smallrye.faulttolerance.core.Invocation.apply(Invocation.java:29)
at
io.smallrye.faulttolerance.core.metrics.MetricsCollector.apply(MetricsCollector.java:98)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.syncFlow(FaultToleranceInterceptor.java:367)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor.intercept(FaultToleranceInterceptor.java:205)
at
io.smallrye.faulttolerance.FaultToleranceInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at
io.quarkus.micrometer.runtime.MicrometerTimedInterceptor.timedMethod(MicrometerTimedInterceptor.java:79)
at
io.quarkus.micrometer.runtime.MicrometerTimedInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at
io.quarkus.security.runtime.interceptor.SecurityHandler.handle(SecurityHandler.java:27)
at
io.quarkus.security.runtime.interceptor.RolesAllowedInterceptor.intercept(RolesAllowedInterceptor.java:29)
at
io.quarkus.security.runtime.interceptor.RolesAllowedInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
at
io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:44)
at
io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_RolesAllowedInterceptor_Bean.intercept(Unknown
Source)
at
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at
io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
at
io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi_Subclass.createTable(Unknown
Source)
at
org.apache.polaris.service.catalog.api.IcebergRestCatalogApi$quarkusrestinvoker$createTable_01f5a1bd6d7815fd3314a553161c943c8cd03101.invoke(Unknown
Source)
at
org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
at
io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:183)
at
org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
at
io.quarkus.vertx.core.runtime.VertxCoreRecorder$15.runWith(VertxCoreRecorder.java:645)
at
org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2651)
at
org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2630)
at
org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1622)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1589)
at
org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
at
org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable
to execute HTTP request: s3-bucket.s3.na-west.amazonaws.com: Name or service
not known (SDK Attempt Count: 6)
at
software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
at
software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:95)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:73)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:53)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:35)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:82)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:62)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:43)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
... 80 more
Suppressed:
software.amazon.awssdk.core.exception.SdkClientException: Request attempt 1
failure: Unable to execute HTTP request: s3-bucket.s3.na-west.amazonaws.com:
Name or service not known
Suppressed:
software.amazon.awssdk.core.exception.SdkClientException: Request attempt 2
failure: Unable to execute HTTP request: s3-bucket.s3.na-west.amazonaws.com:
Name or service not known
Suppressed:
software.amazon.awssdk.core.exception.SdkClientException: Request attempt 3
failure: Unable to execute HTTP request: s3-bucket.s3.na-west.amazonaws.com:
Name or service not known
Suppressed:
software.amazon.awssdk.core.exception.SdkClientException: Request attempt 4
failure: Unable to execute HTTP request: s3-bucket.s3.na-west.amazonaws.com:
Name or service not known
Suppressed:
software.amazon.awssdk.core.exception.SdkClientException: Request attempt 5
failure: Unable to execute HTTP request: s3-bucket.s3.na-west.amazonaws.com:
Name or service not known
Caused by: java.net.UnknownHostException: s3-bucket.na-west.amazonaws.com:
Name or service not known
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native
Method)
at
java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Inet4AddressImpl.java:43)
at
java.base/java.net.InetAddress$PlatformResolver.lookupByName(InetAddress.java:1211)
at
java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1828)
at
java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:1139)
at
java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1818)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1688)
at
org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at
software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$DelegatingHttpClientConnectionManager.connect(ClientConnectionManagerFactory.java:86)
at
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at
software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
at
software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:261)
at
software.amazon.awssdk.http.apache.ApacheHttpClient.access$600(ApacheHttpClient.java:106)
at
software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:238)
at
software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:235)
at
software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:103)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:88)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:64)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:46)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:74)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:43)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:79)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:41)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.executeRequest(RetryableStage.java:93)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:56)
... 92 more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
