[
https://issues.apache.org/jira/browse/HADOOP-14576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236463#comment-16236463
]
Aaron Fabbri commented on HADOOP-14576:
---------------------------------------
I've seen similar issues intermittently just running S3A integration tests. I
will paste a fresh stack trace next time I hit it. For now, sharing another
stack trace our Hive folks provided a while back.
Caused by: org.apache.hadoop.fs.s3a.AWSServiceIOException: get on
s3a:///1000_unpartitioned_parquet_cdh_ip-10-0-0-158/catalog_sales/.hive-staging_hive_2017-06-21_00-27-07_895_5004400669443203568-24/_tmp.-ext-10000/000530_0:
com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException: Requested
resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID:
L93OFC0JOT2N6FT45BIPRQVP1FVV4KQNSO5AEMVJF66Q9ASUAAJG): Requested resource not
found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID:
L93OFC0JOT2N6FT45BIPRQVP1FVV4KQNSO5AEMVJF66Q9ASUAAJG)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:178)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:101)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:395)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:1775)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerRename(S3AFileSystem.java:776)
at org.apache.hadoop.fs.s3a.S3AFileSystem.rename(S3AFileSystem.java:734)
at
org.apache.hadoop.hive.ql.util.ParallelDirectoryRenamer$1.call(ParallelDirectoryRenamer.java:105)
at
org.apache.hadoop.hive.ql.util.ParallelDirectoryRenamer$1.call(ParallelDirectoryRenamer.java:101)
... 4 more
Caused by: com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException:
Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400;
Error Code: ResourceNotFoundException; Request ID: L93OF...snip...)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
> s3guard DynamoDB resource not found: tables not ACTIVE state after initial
> connection
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-14576
> URL: https://issues.apache.org/jira/browse/HADOOP-14576
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Sean Mackrory
> Priority: Major
>
> We currently only anticipate tables not being in the ACTIVE state when first
> connecting. It is possible for a table to be in the ACTIVE state and move to
> an UPDATING state during partitioning events. Attempts to read or write
> during that time will result in an AmazonServerException getting thrown. We
> should try to handle that better...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
