Re: Request/discussion: BufferedReader reading using async API while providing sync API

Hey! Is this forgotten? On 03/11/2016 14:35, Brunoais wrote: Any information you can give on this? On 29/10/2016 19:10, Brunoais wrote: Here's my try on going async. On my tests on my local windows 10 machine (I don't have access to the linux one without a VM ATM) , now with 1GB file, I

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Any information you can give on this? On 29/10/2016 19:10, Brunoais wrote: Here's my try on going async. On my tests on my local windows 10 machine (I don't have access to the linux one without a VM ATM) , now with 1GB file, I noticed: ~2s of speed improvement from BufferedInputStream to

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Here's my try on going async. On my tests on my local windows 10 machine (I don't have access to the linux one without a VM ATM) , now with 1GB file, I noticed: ~2s of speed improvement from BufferedInputStream to BufferedNonBlockStream when BufferedNonBlockStream is at its most

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I'll try going back to a previous version I worked on which used the java7's AsynchronousFileChannel and work from there. My small research shows it can also work with AsynchronousFileChannel mostly without changes. For now, 1 question: Is Thread.sleep() a possible way of dealing the block

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 27/10/2016 22:45, Vitaly Davidovich wrote: On Thursday, October 27, 2016, Brunoais > wrote: You are right. Even in windows it does not set the flags for async reads. It seems like it is windows itself that does the decision

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Sorry nio-dev ... On 28/10/2016 7:09 AM, David Holmes wrote: You might try discussing on net-dev rather than core-libs-dev, to get additional historical info related to the io and nio file APIs. David On 28/10/2016 5:08 AM, Brunoais wrote: You are right. Even in windows it does not set the

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Thursday, October 27, 2016, Brunoais wrote: > You are right. Even in windows it does not set the flags for async reads. > It seems like it is windows itself that does the decision to buffer the > contents based on its own heuristics. > You mean nonblocking, not async,

Re: Request/discussion: BufferedReader reading using async API while providing sync API

You might try discussing on net-dev rather than core-libs-dev, to get additional historical info related to the io and nio file APIs. David On 28/10/2016 5:08 AM, Brunoais wrote: You are right. Even in windows it does not set the flags for async reads. It seems like it is windows itself that

Re: Request/discussion: BufferedReader reading using async API while providing sync API

You are right. Even in windows it does not set the flags for async reads. It seems like it is windows itself that does the decision to buffer the contents based on its own heuristics. But... Why? Why won't it be? Why is there no API for it? How am I getting 100% HDD use and faster times when

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Here's my current code for benchmarking. Does this seem correct to you? On 27/10/2016 14:55, Peter Levart wrote: On 10/27/2016 03:48 PM, Peter Levart wrote: On 10/27/2016 02:20 PM, Brunoais wrote: Hey. Any idea how to skip tests? When testing for BufferedInputStream, the

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 10/27/2016 03:48 PM, Peter Levart wrote: On 10/27/2016 02:20 PM, Brunoais wrote: Hey. Any idea how to skip tests? When testing for BufferedInputStream, the directBufferSize is not used so testing with different directBufferSize makes no sense. I already tried "return 0" on the

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 10/27/2016 02:20 PM, Brunoais wrote: Hey. Any idea how to skip tests? When testing for BufferedInputStream, the directBufferSize is not used so testing with different directBufferSize makes no sense. I already tried "return 0" on the benchmarked test but jmh fills the output with "

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I don't know about Windows specifically, but generally file systems across major OS's will implement readahead in their IO scheduler when they detect sequential scans. On Linux, you can also strace your test to confirm which syscalls are emitted (you should be seeing plain read()'s there, with

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Thanks for the heads up. I'll try that later. These tests are still useful then. Meanwhile, I'll end up also checking how FileChannel queries the OS on windows. I'm getting 100% HDD reads... Could it be that the OS reads the file ahead on its own?... Anyway, I'll look into it. Thanks for the

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Thu, Oct 27, 2016 at 8:34 AM, Brunoais wrote: > Oh... I see. In that case, it means something is terribly wrong. It can be > my initial tests, though. > > I'm testing on both linux and windows and I'm getting performance gains > from using the FileChannel compared to

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Oh... I see. In that case, it means something is terribly wrong. It can be my initial tests, though. I'm testing on both linux and windows and I'm getting performance gains from using the FileChannel compared to using FileInputStream... The tests also make sense based on my predictions O_O...

Re: Request/discussion: BufferedReader reading using async API while providing sync API

tency. Gruss Bernd -- http://bernd.eckenfels.net _ From: Brunoais <brunoa...@gmail.com <mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Thursday, October 27, 2016, Brunoais wrote: > Did you read the C code? I looked at the Linux code in the JDK. > Have you got any idea how many functions Windows or Linux (nearly all > flavors) have for the read operation towards a file? I do. > > I have already done

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 27/10/2016 10:28, Bernd Eckenfels wrote: : AsyncFileChannel uses Java threads in the background for the completion handling (and last time I checked they are not signaled by the OS). Performance with that was not very good for sequential processing in my tests. It does use completion

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Hello, the NIO2 code for files is pretty disappointing as it does not use file completion ports on Windows or AIO on Linux. You use the file channel, in that case it is blocking. It does not implement SelectableChannel. AsyncFileChannel uses Java threads in the background for the completion

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Did you read the C code? Have you got any idea how many functions Windows or Linux (nearly all flavors) have for the read operation towards a file? I have already done that homework myself. I may not have read JVM's source code but I know well that there's functions on both Windows and Linux

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Wednesday, October 26, 2016, Brunoais wrote: > It is actually based on the premise that: > > 1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS >buffer size to fill in as the same size as ByteBuffer. Why do you say that? AFAICT, it issues a read

Re: Request/discussion: BufferedReader reading using async API while providing sync API

uests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _ From: Brunoais <brunoa...@gmail.com <mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading

Re: Request/discussion: BufferedReader reading using async API while providing sync API

-- http://bernd.eckenfels.net _ From: Brunoais <brunoa...@gmail.com <mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo <pavel.ra...@oracle.com <ma

Re: Request/discussion: BufferedReader reading using async API while providing sync API

;>> to the fact that it NOT wraps to real AsyncFIO on most platforms. >>>> But maybe I just measured it wrong, so I will have a closer look on >>>> your impl. >>>> >>>> Generally I would recommend to make the Benchmark a bit more >>>&

Re: Request/discussion: BufferedReader reading using async API while providing sync API

t might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _ From: Brunoais <brunoa...@gmail.com <mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading using async AP

Re: Request/discussion: BufferedReader reading using async API while providing sync API

equests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _ From: Brunoais <brunoa...@gmail.com <mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading

Re: Request/discussion: BufferedReader reading using async API while providing sync API

It is actually based on the premise that: 1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS buffer size to fill in as the same size as ByteBuffer. 2. The consecutive calls to ReadableByteChannel.read(ByteBuffer) orders the JVM to order the OS to execute memcpy() to copy

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I believe I see where you coming from. Please correct me if I'm wrong. Your implementation is based on the premise that a call to ReadableByteChannel.read() _initiates_ the operation and returns immediately. The OS then continues to fill the buffer while there's a free space in the buffer and

Re: Request/discussion: BufferedReader reading using async API while providing sync API

_ From: Brunoais <brunoa...@gmail.com <mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo <pavel.ra...@oracle.com <mailto:pavel.ra...@ora

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo <pavel.ra...@oracle.com <mailto:pavel.ra...@oracle.com>> Cc: <core-libs-dev@openjdk.java.net <mailto:core-libs-dev@openjdk.java.net>> Attached and sending! On 24/10/201

Re: Request/discussion: BufferedReader reading using async API while providing sync API

ubject: Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo <pavel.ra...@oracle.com> Cc: <core-libs-dev@openjdk.java.net> Attached and sending! On 24/10/2016 13:48, Pavel Rappo wrote: > Could you please send a new email on this

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Thank you for your time. I'll try to explain it. I hope I can clear it up. First of it, I made a meaning mistake between asynchronous and non-blocking. This implementation uses a non-blocking algorithm internally while providing a blocking-like algorithm on the surface. It is single-threaded

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I've skimmed through the code and I'm not sure I can see any asynchronicity (you were pointing at the lack of it in BufferedReader). And the mechanics of this is very puzzling to me, to be honest: void blockingFill(boolean forced) throws IOException { fill(forced); while

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Attached and sending! On 24/10/2016 13:48, Pavel Rappo wrote: Could you please send a new email on this list with the source attached as a text file? On 23 Oct 2016, at 19:14, Brunoais wrote: Here's my poc/prototype: http://pastebin.com/WRpYWDJF I've implemented the

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Could you please send a new email on this list with the source attached as a text file? > On 23 Oct 2016, at 19:14, Brunoais wrote: > > Here's my poc/prototype: > http://pastebin.com/WRpYWDJF > > I've implemented the bare minimum of the class that follows the same contract

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Here's my poc/prototype: http://pastebin.com/WRpYWDJF I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments. I also wrote some javadoc to help guiding through the class. I could have

Re: Request/discussion: BufferedReader reading using async API while providing sync API

OK. Will do. On 21/10/2016 19:21, Pavel Rappo wrote: Just to append to my previous email. BufferedReader wraps any Reader out there. Not specifically FileReader. While you're talking about the case of effective reading from a file. I guess there's one existing possibility to provide exactly

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Just to append to my previous email. BufferedReader wraps any Reader out there. Not specifically FileReader. While you're talking about the case of effective reading from a file. I guess there's one existing possibility to provide exactly what you need (as I understand it) under this method: /**

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Pavel is right. In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread. Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Hi Pavel, I think Brunoais asking for a double buffering scheme in which the implementation of BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer and managing the swaps and async reads transparently. It would not change the API but would change

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Off the top of my head, I would say it's not possible to change the design of an _extensible_ type that has been out there for 20 or so years. All these I/O streams from java.io were designed for simple synchronous use case. It's not that their design is flawed in some way, it's that they doesn't

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it. That's why I'm asking this here. On 13/10/2016 14:45,

Request/discussion: BufferedReader reading using async API while providing sync API

Hi, I'm new here, so if this is belongs to a different mailing list please point me in the right direction. I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load

Request/discussion: BufferedReader reading using async API while providing sync API

Hi, I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows