saxenapranav opened a new pull request, #6617: URL: https://github.com/apache/hadoop/pull/6617
JIRA: https://issues.apache.org/jira/browse/HADOOP-19102 The method `optimisedRead` creates a buffer array of size `readBufferSize`. If footerReadBufferSize is greater than readBufferSize, abfs will attempt to read more data than the buffer array can hold, which causes an exception. Change: To avoid this, we will keep footerBufferSize = min(readBufferSizeConfig, footerBufferSizeConfig) Also, as part of this PR, have improved tests within `ITestAbfsInputStreamReadFooter`. There are tests which have multiple combination, and there was file getting created for all the combination. There has to be a combination on different fileSize. The change: We will spin up one thread each for each fileSize. And in each thread, all the combination for that particular fileSize will run. This will help in creating file once for a fileSize and multiple fileSize related assertion can happen in parallel and use hardware capability. Improvement: on a 6 processor VM, on trunk, it tool 8min47sec to run all tests of ITestAbfsInputStreamReadFooter and in the PR branch, it took 7 min. (Each test method run one after another). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
