ahmarsuhail opened a new pull request, #4298: URL: https://github.com/apache/hadoop/pull/4298
### Description of PR This PR adds support for unbuffer. Unbuffer is used by certain applications (eg: Impala) when they want to hold onto an input stream but free the resource it's using. This is useful as when it needs to read from the stream again, it doesn't have to open the stream again, and can save on HEAD calls. For prefetching, unbuffer needs to free up the buffer pool, delete any local files, clear state about blocks in the file etc. Also, when reading after an unbuffer, the input stream should reinitialise all this state. It should also read from the last active position before the read. ### How was this patch tested? Tested in eu-west-1 by running `mvn -Dparallel-tests -DtestsThreadCount=16 clean verify` `ITestS3AInputStreamPerformance` is failing, unrelated to this PR. Created issue: https://issues.apache.org/jira/browse/HADOOP-18231 `ITestS3AUnbuffer` fails. instance of assertion & `isObjectStreamOpen()` fails. Similar to the above issue, there are a few different ways to fix this test. I'm not sure what the best way is. Parameterized tests and different assertions based on if prefetching is enabled/new tests? I've left it failing for now. All unbuffer contract tests are passing now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
