Steve Loughran created HADOOP-17415:
---------------------------------------
Summary: Use S3 content-range header to update length of an object
during reads
Key: HADOOP-17415
URL: https://issues.apache.org/jira/browse/HADOOP-17415
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.3.0
Reporter: Steve Loughran
As part of all the openFile work, knowing full length of an object allows for a
HEAD to be skipped. But: code knowing only the splits don't know the final
length of the file.
If the content-range header is used, then as soon as a single GET is initiated
against an object, if the field is returned then we can update the length of
the S3A stream to its real/final length
Also: when any input stream fails with an EOF exception, we can distinguish
stream-interrupted from "no, too far"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
