[
https://issues.apache.org/jira/browse/HADOOP-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Mackrory updated HADOOP-13995:
-----------------------------------
Attachment: HADOOP-13995.001.patch
Attaching an initial patch - I'm fine to wait and apply this on top of other
changes if we want. I feel like we may want to get HADOOP-13311 settled soon
too, otherwise this is just going to become the de facto starting point for all
S3 commands anyway - since we're going to add fsck and other commands shortly.
[[email protected]] - I believe that failure is what you get if you haven't
configured the DynamoDB endpoint to use. If you *do* configure the endpoint,
TestDynamoDBMetastore fails (it checks an operation fails with no endpoint set,
but it doesn't UNSET it first) so I fixed that, and added the endpoint config
to the doc. I think the endpoint is something we should just have users
configure with their table: their appears to be no "global" endpoint we could
use as a sensible default. And it appears tables with identical names can exist
simultaneously in different regions. But I did observe one case where the
endpoint I configured wasn't honored and a table in a different region was used
instead. So I need to understand better how tables map to regions / endpoints,
etc.
I've been testing this against the US regions. I did some manual testing of the
CLI tools and also fixed a couple of pain points there too (unclear errors,
etc.) The Import command was not initializing the MetadataStore instance, so I
fixed that - I went with NOT creating the table if it doesn't exist, but I
don't feel strongly about that if others disagree. And I split the S3GuardTool
tests into an integration test against DynamoDB and a unit test against Local.
Tests in both camps were making different assumptions about the same instance.
> s3guard cli: make tests easier to run and address failure
> ---------------------------------------------------------
>
> Key: HADOOP-13995
> URL: https://issues.apache.org/jira/browse/HADOOP-13995
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Sean Mackrory
> Attachments: HADOOP-13995.001.patch
>
>
> Following up on HADOOP-13650, we should:
> - Make it clearer which config parameters need to be set for test to succeed,
> and provide good defaults.
> - Address any remaining test failures.
> - Change TestS3GuardTool to an ITest
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
