[
https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804093#comment-15804093
]
Mingliang Liu edited comment on HADOOP-13908 at 1/6/17 9:25 AM:
----------------------------------------------------------------
When I tested the v4 patch when the table was not existent (and auto-creation
is enabled), I still got "requested resource not found" exception, which is not
expected. There is some offline discussion with Chris and Steve, and I'll post
the progress or new patch later.
{code}
$ mvn -Dtest=none -Dit.test='ITestS3A*' -q -Dscale clean verify
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
Tests run: 10, Failures: 0, Errors: 10, Skipped: 0, Time elapsed: 7.569 sec <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
testOverwriteNonEmptyDirectory(org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate)
Time elapsed: 7.008 sec <<< ERROR!
org.apache.hadoop.fs.s3a.AWSServiceIOException: get on s3a://mliu-s3guard/test:
com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException: Requested
resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID: ooxx): Requested resource not found
(Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID: ooxx)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:171)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:94)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:309)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1607)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerMkdirs(S3AFileSystem.java:1551)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:1520)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2218)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:338)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:193)
...
{code}
was (Author: liuml07):
When I tested the v4 patch when the table was not existent (and auto-creation
is enabled), I still got "requested resource not found" exception, which is not
expected. There is some offline discussion on-going, and I'll post the progress
or new patch later.
{code}
$ mvn -Dtest=none -Dit.test='ITestS3A*' -q -Dscale clean verify
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
Tests run: 10, Failures: 0, Errors: 10, Skipped: 0, Time elapsed: 7.569 sec <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
testOverwriteNonEmptyDirectory(org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate)
Time elapsed: 7.008 sec <<< ERROR!
org.apache.hadoop.fs.s3a.AWSServiceIOException: get on s3a://mliu-s3guard/test:
com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException: Requested
resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID: ooxx): Requested resource not found
(Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID: ooxx)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:171)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:94)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:309)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1607)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerMkdirs(S3AFileSystem.java:1551)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:1520)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2218)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:338)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:193)
...
{code}
> S3Guard: Existing tables may not be initialized correctly in
> DynamoDBMetadataStore
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-13908
> URL: https://issues.apache.org/jira/browse/HADOOP-13908
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Attachments: HADOOP-13908-HADOOP-13345.000.patch,
> HADOOP-13908-HADOOP-13345.001.patch, HADOOP-13908-HADOOP-13345.002.patch,
> HADOOP-13908-HADOOP-13345.002.patch, HADOOP-13908-HADOOP-13345.003.patch,
> HADOOP-13908-HADOOP-13345.004.patch
>
>
> This was based on discussion in [HADOOP-13455]. Though we should not create
> table unless the config {{fs.s3a.s3guard.ddb.table.create}} is set true, we
> still have to get the existing table in
> {{DynamoDBMetadataStore#initialize()}} and wait for its becoming active,
> before any table/item operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
