[
https://issues.apache.org/jira/browse/VFS-805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17354480#comment-17354480
]
Claus Stadler edited comment on VFS-805 at 5/31/21, 2:15 PM:
-------------------------------------------------------------
A small update: I see that [MonitoredHttpResponseContentInputStream
|https://github.com/apache/commons-vfs/blob/ca5a27dab0aaef84f9cf5e10debfa5827f2a873f/commons-vfs2/src/main/java/org/apache/commons/vfs2/provider/http4/MonitoredHttpResponseContentInputStream.java#L44]
actually does exactly this kind of closing in an *onClose* method - however,
*onClose* is invoked by *close* only after the input stream has been consumed.
was (Author: aklakan):
A small update: I see that [MonitoredHttpResponseContentInputStream
|https://github.com/apache/commons-vfs/blob/ca5a27dab0aaef84f9cf5e10debfa5827f2a873f/commons-vfs2/src/main/java/org/apache/commons/vfs2/provider/http4/MonitoredHttpResponseContentInputStream.java#L44]
actually does exactly this kind of closing an an *onClose* method - however,
*onClose* is invoked by *close* only after the input stream has been consumed.
> HTTP seek always exhausts response
> ----------------------------------
>
> Key: VFS-805
> URL: https://issues.apache.org/jira/browse/VFS-805
> Project: Commons VFS
> Issue Type: Bug
> Affects Versions: 2.8.0
> Reporter: Claus Stadler
> Priority: Major
>
> Seeking on an HTTP resource always downloads ALL content if a Content-Length
> header is present. The problem is that seeking closes the current input
> stream which eventually ends up in ContentLengthInputStream.close() of the
> (ancient) http client library.
>
> To be clear, the problem is actually not with the seek itself, but with the
> underlying close implementation that always exhausts the HTTP response body.
> See the example below.
>
> My use case is to perform binary search on sorted datasets on the Web (RDF
> data in sorted ntriple syntax) - the binary search works locally and *in
> principle* works on HTTP resources abstracted with VFS2, but the seek
> implementation that downloads *ALL* data (in my case several GBs)
> unfortunately defeats the purpose :(
>
> From org.apache.commons.httpclient.ContentLengthInputStream
> (commons-httpclient-3.1):
> {code:java}
> public void close() throws IOException {
> if (!closed) {
> try {
> ChunkedInputStream.exhaustInputStream(this);
> } finally {
> // close after above so that we don't throw an exception
> trying
> // to read after closed!
> closed = true;
> }
> }
> }
> {code}
> Example:
> {code:java}
> public static void main(String[] args) throws Exception {
> String url = "http://localhost/large-file-2gb.txt";;
> FileSystemManager fsManager = VFS.getManager();
>
> try (FileObject file = fsManager.resolveFile(url)) {
> try (RandomAccessContent r =
> file.getContent().getRandomAccessContent(RandomAccessMode.READ)) {
>
> StopWatch sw1 = StopWatch.createStarted();
> r.seek(20);
> System.out.println("Initial seek: " +
> sw1.getTime(TimeUnit.MILLISECONDS));
> StopWatch sw2 = StopWatch.createStarted();
> byte[] bytes = new byte[100];
> r.readFully(bytes);
> System.out.println("Read: " +
> sw2.getTime(TimeUnit.MILLISECONDS));
>
> StopWatch sw3 = StopWatch.createStarted();
> r.seek(100);
> System.out.println("Subsequent seek: " +
> sw3.getTime(TimeUnit.MILLISECONDS));
> }
> }
> System.out.println("Done");
> }
> {code}
> Output (times in milliseconds):
> {code}
> Initial seek: 0
> Read: 4
> Subsequent seek: 2538
> Done
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
