[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15971778#comment-15971778
]
Steve Moist commented on HADOOP-14305:
--------------------------------------
I'm pretty sure it's SSE-C related.
I disabled the SSE-C tests and it ran fine in parallel mode. So I entirely
suspect its that its using the wrong key, wrong method or no key supplied when
trying to re-access the fork-#/ prefix in the tests (either with SSE-C, and the
other tests failing because it's not cleaned up properly) and s3 is failing due
to the wrong key being provided. If I do a "aws s3api head-object --bucket
my-bucket --key sse-c-encrypted.txt", I get a "An error occurred (400) when
calling the HeadObject operation: Bad Request" when trying to get the head of
an object with no encryption key, or the wrong encryption key. Which is the
same http error in the failed tests. SSE-C was one of those s3 features where
the encryption key must always be provided for all operations or you get a 400
error. So I suspect this is the real culprit of the issue.
In general, I don't really advise people to use SSE-C. In the web ui, you
can't tell if it's encrypted, which encryption key was used, nor can you get
the meta-data about the object without the correct encryption key through the
command line. If a customer wishes to use this, the best solution is to use 1
SSE-C key across the cluster. If they wish to use multiple, they need to keep
track of which key maps to where and should NEVER share buckets. I also
recommend the documentation advising only to use this for advanced users.
> S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus
> ---------------------------------------------------------------------------
>
> Key: HADOOP-14305
> URL: https://issues.apache.org/jira/browse/HADOOP-14305
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-14305-001.patch
>
>
> The S3a encryption tests all run in parallel (they were interfering with each
> other, apparently). This adds ~1 min to the test runs.
> They should run in serial. That they fail when this is attempted due to Bad
> Auth problems must be considered a serious problem, as is indicates issues
> related to working with SSE encryption from hadoop
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
