[
https://issues.apache.org/jira/browse/IMPALA-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-8344.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Add support for running S3 tests with S3Guard
> ---------------------------------------------
>
> Key: IMPALA-8344
> URL: https://issues.apache.org/jira/browse/IMPALA-8344
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: Impala 3.3.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
> Fix For: Impala 3.3.0
>
>
> Impala s3 tests can encounter failures due to s3's consistency model. S3Guard
> adds consistency to s3 operations to avoid these types of failures:
> [https://blog.cloudera.com/blog/2017/08/introducing-s3guard-s3-consistency-for-apache-hadoop/]
> Adding support to run tests with S3Guard provides a way to limit flakiness,
> and it provides coverage for users that would want to use Impala on S3 with
> S3Guard.
> Support will involve adding the appropriate configuration to core-site.xml.
> In order to maintain the S3Guard index appropriately, file modifications
> should go through HDFS commands rather than boto s3 commands. Finally, to
> reduce costs, Impala may want to have a script to purge S3Guard's dynamodb.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
