[
https://issues.apache.org/jira/browse/HADOOP-18231?focusedWorklogId=769538&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769538
]
ASF GitHub Bot logged work on HADOOP-18231:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 12/May/22 10:57
Start Date: 12/May/22 10:57
Worklog Time Spent: 10m
Work Description: ahmarsuhail opened a new pull request, #4305:
URL: https://github.com/apache/hadoop/pull/4305
### Description of PR
`ITestS3AInputStreamPerformance` was failing when prefetching is enabled.
This PR disables prefetching when running `ITestS3AInputStreamPerformance`.
It also adds in a new test class `ITestS3PrefetchingInputStream` with tests
specific for prefetching. Once more stats are added to prefetching iostats, we
can add in more tests + assertions. For eg it would be good to assert on if a
file has been read via the cache or not on a backward seek.
### How was this patch tested?
Tested in eu-west-1 by running
`mvn -Dparallel-tests -DtestsThreadCount=16 clean verify`
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
Issue Time Tracking
-------------------
Worklog Id: (was: 769538)
Remaining Estimate: 0h
Time Spent: 10m
> tests in ITestS3AInputStreamPerformance are failing
> ----------------------------------------------------
>
> Key: HADOOP-18231
> URL: https://issues.apache.org/jira/browse/HADOOP-18231
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Ahmar Suhail
> Assignee: Ahmar Suhail
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following tests are failing when prefetching is enabled:
> testRandomIORandomPolicy - expects stream to be opened 4 times (once for
> every random read), but prefetching will only open twice.
> testDecompressionSequential128K - expects stream to be opened once, but
> prefetching will open once for each block the file has. landsat file used in
> the test has size 42MB, prefetching block size = 8MB, expected open count is
> 6.
> testReadWithNormalPolicy - same as above.
> testRandomIONormalPolicy - executes random IO, but with a normal policy.
> S3AInputStream will abort the stream and change the policy, prefetching
> handles random IO by caching blocks so doesn't do any of that.
> testRandomReadOverBuffer - multiple assertions failing here, also depends a
> lot on readAhead values, not very relevant for prefetching
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
