[
https://issues.apache.org/jira/browse/IMPALA-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell reassigned IMPALA-9677:
-------------------------------------
Assignee: Joe McDonnell
> FE Analysis tests using fake S3 bucket fail with AnalysisException
> ------------------------------------------------------------------
>
> Key: IMPALA-9677
> URL: https://issues.apache.org/jira/browse/IMPALA-9677
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: broken-build
>
> It looks like Hadoop added extra validation for initializing the
> S3AFilesystem.
> [HADOOP-16711|https://issues.apache.org/jira/browse/HADOOP-16711] now
> defaults to checking for the existence of the bucket and permissions on it.
> The level of validation is controlled by fs.s3a.bucket.probe. We use a bogus
> S3 bucket in our analysis tests with the expectation that all we require is a
> valid URI. Instead, we get this error:
> {noformat}
> Error during analysis:
> org.apache.impala.common.AnalysisException: bucket: doesBucketExistV2 on
> bucket: com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key
> Id you provided does not exist in our records. (Service: Amazon S3; Status
> Code: 403; Error Code: InvalidAccessKeyId; Request ID: 10988D12BA86A604; S3
> Extended Request ID:
> qbfw3pMHGGiPTZyLiAOIAvMlJLpy5HOgkHSN4I1n1pBhX2H3BYSwDXdt+gIi3+xiA+E2w5Itv5o=),
> S3 Extended Request ID:
> qbfw3pMHGGiPTZyLiAOIAvMlJLpy5HOgkHSN4I1n1pBhX2H3BYSwDXdt+gIi3+xiA+E2w5Itv5o=:InvalidAccessKeyId
> sql:
> create table tbl (i int) location
> 's3a://bucket/test-warehouse/new_table'{noformat}
> Developers frequently don't have S3 credentials set up, so this will always
> fail.
> Currently, these tests fails:
> {noformat}
> org.apache.impala.analysis.AnalyzeDDLTest.TestAlterTableAddDropPartition
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateDataSource
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateDb
> org.apache.impala.analysis.AnalyzeDDLTest.TestAlterTableSet
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTableLike
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTable
> org.apache.impala.planner.ExplainTest.testScanNodeFsScheme
> org.apache.impala.planner.PlannerTest.testScanNodeFsScheme{noformat}
> It reproduces consistently on jenkins.impala.io:
> [https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/69/]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
