[
https://issues.apache.org/jira/browse/HADOOP-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805172#comment-17805172
]
ASF GitHub Bot commented on HADOOP-19027:
-----------------------------------------
steveloughran commented on code in PR #6425:
URL: https://github.com/apache/hadoop/pull/6425#discussion_r1447486233
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractVectoredRead.java:
##########
@@ -72,9 +78,40 @@ public void testEOFRanges() throws Exception {
FileSystem fs = getFileSystem();
List<FileRange> fileRanges = new ArrayList<>();
fileRanges.add(FileRange.createFileRange(DATASET_LEN, 100));
- verifyExceptionalVectoredRead(fs, fileRanges, EOFException.class);
+ verifyExceptionalVectoredRead(fs, fileRanges,
RangeNotSatisfiableEOFException.class);
}
+ /**
+ * Verify response to a vector read request which is beyond the
+ * real length of the file.
+ * Unlike the {@link #testEOFRanges()} test, the input stream in
+ * this test thinks the file is longer than it is, so the call
+ * fails in the GET request.
+ */
+ @Test
+ public void testEOFRanges416Handling() throws Exception {
+ FileSystem fs = getFileSystem();
+
+ CompletableFuture<FSDataInputStream> builder =
+ fs.openFile(path(VECTORED_READ_FILE_NAME))
+ .mustLong(FS_OPTION_OPENFILE_LENGTH, DATASET_LEN + 1024)
Review Comment:
simulates EOF problems and verify reopen failures...doesn't simulate
connection failure within file length. that'd be trickier
> S3A: S3AInputStream doesn't recover from HTTP/channel exceptions
> ----------------------------------------------------------------
>
> Key: HADOOP-19027
> URL: https://issues.apache.org/jira/browse/HADOOP-19027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> S3AInputStream doesn't seem to recover from Http exceptions raised through
> HttpClient or through OpenSSL.
> * review the recovery code to make sure it is retrying enough, it looks
> suspiciously like it doesn't
> * detect the relevant openssl, shaded httpclient and unshaded httpclient
> exceptions, map to a standard one and treat as comms error in our retry policy
> This is not the same as the load balancer/proxy returning 443/444 which we
> map to AWSNoResponseException. We can't reuse that as it expects to be
> created from an
> {{software.amazon.awssdk.awscore.exception.AwsServiceException}} exception
> with the relevant fields...changing it could potentially be incompatible.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
