[
https://issues.apache.org/jira/browse/HADOOP-18708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17819725#comment-17819725
]
ASF GitHub Bot commented on HADOOP-18708:
-----------------------------------------
ahmarsuhail commented on PR #6164:
URL: https://github.com/apache/hadoop/pull/6164#issuecomment-1959863096
Tested in eu-west-1, with `fs.s3a.encryption.algorithm` = CSE-KMS,
`fs.s3a.encryption.enabled` = true, and `fs.s3a.scale.test.enabled` = true.
Seeing a few failures:
```
[ERROR]
ITestS3ACommitterFactory.testEverything:115->testInvalidFileBinding:165
Expected a org.apache.hadoop.fs.s3a.commit.PathCommitException to be thrown,
but got the result: :
FileOutputCommitter{PathOutputCommitter{context=TaskAttemptContextImpl{JobContextImpl{jobId=job_202402220694_0010};
taskId=attempt_202402220694_0010_m_000000_0, status=''};
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter@3f226b07};
outputPath=s3a://ahmarsu-test-aws-s3a/fork-00010/test/testEverything,
workPath=s3a://ahmarsu-test-aws-s3a/fork-00010/test/testEverything/_temporary/1/_temporary/attempt_202402220694_0010_m_000000_0,
algorithmVersion=1, skipCleanup=false, ignoreCleanupFailures=false}
[ERROR] Errors:
[ERROR] ITestS3AContractVectoredRead.testEOFRanges416Handling[Buffer type
: array] » EOF End of file reached before reading fully.
[ERROR] ITestS3AContractVectoredRead.testEOFRanges416Handling[Buffer type
: direct] » EOF End of file reached before reading fully.
[ERROR]
ITestConnectionTimeouts.testGeneratePoolTimeouts:131->lambda$testGeneratePoolTimeouts
```
Suspect the vectoreIO failures are intermittent, and
`ITestConnectionTimeouts.testGeneratePoolTimeouts` is cause the S3 Encryption
client is using the async client, so the error is probably different. But will
investigate and fix accordingly asap.
> AWS SDK V2 - Implement CSE
> --------------------------
>
> Key: HADOOP-18708
> URL: https://issues.apache.org/jira/browse/HADOOP-18708
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Ahmar Suhail
> Assignee: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
>
> S3 Encryption client for SDK V2 is now available, so add client side
> encryption back in.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
