[
https://issues.apache.org/jira/browse/HADOOP-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17807849#comment-17807849
]
ASF GitHub Bot commented on HADOOP-19043:
-----------------------------------------
steveloughran opened a new pull request, #6460:
URL: https://github.com/apache/hadoop/pull/6460
This is actually trickier than it seems as we will need to go deep into the
implementation of caching.
Specifically: the prefetcher knows the file length and if you open a file
shorter than that, but less than one block, the read is considered a failure
and the whole block is skipped, so read() of the nominally in-range data
returns -1.
This fix has to be considered a PoC and should be combined with the other
big PR for prefetching, #5832 as that is where changes should go.
Here is just test tuning and some differentiation of channel problems from
other EOFs.
### How was this patch tested?
```
mvn verify -Dtest=none -Dit.test=ITestS3AOpenCost -Dprefetch
```
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> S3A: Regression: ITestS3AOpenCost fails on prefetch test runs
> -------------------------------------------------------------
>
> Key: HADOOP-19043
> URL: https://issues.apache.org/jira/browse/HADOOP-19043
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
>
> Getting test failures in the new ITestS3AOpenCost tests when run with
> {{-Dprefetch}}
> Thought I'd tested this, but clearly not
> * class cast failures on asserts (fix: skip)
> * bytes read different in one test: (fix: identify and address)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
