I believe I see where you coming from. Please correct me if I'm wrong. 

Your implementation is based on the premise that a call to 
ReadableByteChannel.read()
_initiates_ the operation and returns immediately. The OS then continues to fill
the buffer while there's a free space in the buffer and the channel hasn't 
encountered EOF.

Is that right?

> On 25 Oct 2016, at 22:16, Brunoais <brunoa...@gmail.com> wrote:
> 
> Thank you for your time. I'll try to explain it. I hope I can clear it up.
> First of it, I made a meaning mistake between asynchronous and non-blocking. 
> This implementation uses a non-blocking algorithm internally while providing 
> a blocking-like algorithm on the surface. It is single-threaded and not 
> multi-threaded where one thread fetches data and blocks waiting and the other 
> accumulates it and provides to whichever wants it.
> 
> Second of it, I had made a mistake of going after BufferedReader instead of 
> going after BufferedInputStream. If you want me to go after BufferedReader 
> it's ok but I only thought that going after BufferedInputStream would be more 
> generically useful than BufferedReaderwhen I started the poc.
> 
> On to my code: 
> Short answers: 
>       • The sleep(int) exists because I don't know how to wait until more 
> data exists in the buffer which is part of read()'s contract.
>       • The ByteBuffer gives a buffer that is filled by the OS (what I 
> believe Channels do) instead of getting data only         by demand (what I 
> believe Streams do).
> Full answers: 
> The blockingFill(boolean) method is a method for a busy wait for a fill which 
> is used exclusively by the read() method. All other methods use the version 
> that does not sleep (fill(boolean)).
> blockingFill(boolean)'s existance like that is only because the read() method 
> must not return unless either:
> 
>       • The stream ended.
>       • The next byte is ready for reading.
> Additionally, statistically, that while loop will rarely evaluate to true as 
> reads are in chunks so readPos will be behind writePos most of the time.
> I have no idea if an interrupt will ever happen, to be honest. The main 
> reasons why I'm using a sleep is because I didn't want a hog onto the CPU in 
> a full thread usage busy wait and because I didn't find any way of doing a 
> thread sleep in order to wake up later when the buffer managed by native code 
> has more data.
> The Non-blocking part is managed by the buffer the OS keeps filling most if 
> not all the time. That buffer is the field
> 
> ByteBuffer readBuffer 
> That's the gaining part against the plain old Buffered classes.
> 
> 
> Did that make sense to you? Feel free to ask anything else you need.
> 
> On 25/10/2016 20:52, Pavel Rappo wrote:
>> I've skimmed through the code and I'm not sure I can see any asynchronicity
>> (you were pointing at the lack of it in BufferedReader).
>> And the mechanics of this is very puzzling to me, to be honest:
>>     void blockingFill(boolean forced) throws IOException {
>>         fill(forced);
>>         while (readPos == writePos) {
>>             try {
>>                 Thread.sleep(100);
>>             } catch (InterruptedException e) {
>>                 // An interrupt may mean more data is available
>>             }
>>             fill(forced);
>>         }
>>     }
>> I thought you were suggesting that we should utilize the tools which OS 
>> provides
>> more efficiently. Instead we have something that looks very similarly to a
>> "busy loop" and... also who and when is supposed to interrupt Thread.sleep()?
>> Sorry, I'm not following. Could you please explain how this is supposed to 
>> work?
>> 
>>> On 24 Oct 2016, at 15:59, Brunoais <brunoa...@gmail.com>
>>>  wrote:
>>> Attached and sending!
>>> On 24/10/2016 13:48, Pavel Rappo wrote:
>>> 
>>>> Could you please send a new email on this list with the source attached as 
>>>> a
>>>> text file?
>>>> 
>>>>> On 23 Oct 2016, at 19:14, Brunoais <brunoa...@gmail.com>
>>>>>  wrote:
>>>>> Here's my poc/prototype:
>>>>> 
>>>>> http://pastebin.com/WRpYWDJF
>>>>> 
>>>>> I've implemented the bare minimum of the class that follows the same 
>>>>> contract of BufferedReader while signaling all issues I think it may have 
>>>>> or has in comments.
>>>>> I also wrote some javadoc to help guiding through the class.
>>>>> I could have used more fields from BufferedReader but the names were so 
>>>>> minimalistic that were confusing me. I intent to change them before 
>>>>> sending this to openJDK.
>>>>> One of the major problems this has is long overflowing. It is major 
>>>>> because it is hidden, it will be extremely rare and it takes a really 
>>>>> long time to reproduce. There are different ways of dealing with it. From 
>>>>> just documenting to actually making code that works with it.
>>>>> I built a simple test code for it to have some ideas about performance 
>>>>> and correctness.
>>>>> 
>>>>> http://pastebin.com/eh6LFgwT
>>>>> 
>>>>> This doesn't do a through test if it is actually working correctly but I 
>>>>> see no reason for it not working correctly after fixing the 2 bugs that 
>>>>> test found.
>>>>> I'll also leave here some conclusions about speed and resource 
>>>>> consumption I found.
>>>>> I made tests with default buffer sizes, 5000B  15_000B and 500_000B. I 
>>>>> noticed that, with my hardware, with the 1 530 000 000B file, I was 
>>>>> getting around:
>>>>> In all buffers and fake work: 10~15s speed improvement ( from 90% HDD 
>>>>> speed to 100% HDD speed)
>>>>> In all buffers and no fake work: 1~2s speed improvement ( from 90% HDD 
>>>>> speed to 100% HDD speed)
>>>>> Changing the buffer size was giving different reading speeds but both 
>>>>> were quite equal in how much they would change when changing the buffer 
>>>>> size.
>>>>> Finally, I could always confirm that I/O was always the slowest thing 
>>>>> while this code was running.
>>>>> For the ones wondering about the file size; it is both to avoid OS cache 
>>>>> and to make the reading at the main use-case these objects are for (large 
>>>>> streams of bytes).
>>>>> @Pavel, are you open for discussion now ;)? Need anything else?
>>>>> On 21/10/2016 19:21, Pavel Rappo wrote:
>>>>> 
>>>>>> Just to append to my previous email. BufferedReader wraps any Reader out 
>>>>>> there.
>>>>>> Not specifically FileReader. While you're talking about the case of 
>>>>>> effective
>>>>>> reading from a file.
>>>>>> I guess there's one existing possibility to provide exactly what you 
>>>>>> need (as I
>>>>>> understand it) under this method:
>>>>>> /**
>>>>>>  * Opens a file for reading, returning a {@code BufferedReader} to read 
>>>>>> text
>>>>>>  * from the file in an efficient manner...
>>>>>>    ...
>>>>>>  */
>>>>>> java.nio.file.Files#newBufferedReader(java.nio.file.Path)
>>>>>> It can return _anything_ as long as it is a BufferedReader. We can do 
>>>>>> it, but it
>>>>>> needs to be investigated not only for your favorite OS but for other 
>>>>>> OSes as
>>>>>> well. Feel free to prototype this and we can discuss it on the list 
>>>>>> later.
>>>>>> Thanks,
>>>>>> -Pavel
>>>>>> 
>>>>>>> On 21 Oct 2016, at 18:56, Brunoais <brunoa...@gmail.com>
>>>>>>>  wrote:
>>>>>>> Pavel is right.
>>>>>>> In reality, I was expecting such BufferedReader to use only a single 
>>>>>>> buffer and have that Buffer being filled asynchronously, not in a 
>>>>>>> different Thread.
>>>>>>> Additionally, I don't have the intention of having a larger buffer than 
>>>>>>> before unless stated through the API (the constructor).
>>>>>>> In my idea, internally, it is supposed to use 
>>>>>>> java.nio.channels.AsynchronousFileChannel or equivalent.
>>>>>>> It does not prevent having two buffers and I do not intent to change 
>>>>>>> BufferedReader itself. I'd do an BufferedAsyncReader of sorts (any name 
>>>>>>> suggestion is welcome as I'm an awful namer).
>>>>>>> On 21/10/2016 18:38, Roger Riggs wrote:
>>>>>>> 
>>>>>>>> Hi Pavel,
>>>>>>>> I think Brunoais asking for a double buffering scheme in which the 
>>>>>>>> implementation of
>>>>>>>> BufferReader fills (a second buffer) in parallel with the application 
>>>>>>>> reading from the 1st buffer
>>>>>>>> and managing the swaps and async reads transparently.
>>>>>>>> It would not change the API but would change the interactions between 
>>>>>>>> the buffered reader
>>>>>>>> and the underlying stream.  It would also increase memory requirements 
>>>>>>>> and processing
>>>>>>>> by introducing or using a separate thread and the necessary 
>>>>>>>> synchronization.
>>>>>>>> Though I think the formal interface semantics could be maintained, I 
>>>>>>>> have doubts
>>>>>>>> about compatibility and its unintended consequences on existing 
>>>>>>>> subclasses,
>>>>>>>> applications and libraries.
>>>>>>>> $.02, Roger
>>>>>>>> On 10/21/16 1:22 PM, Pavel Rappo wrote:
>>>>>>>> 
>>>>>>>>> Off the top of my head, I would say it's not possible to change the 
>>>>>>>>> design of an
>>>>>>>>> _extensible_ type that has been out there for 20 or so years. All 
>>>>>>>>> these I/O
>>>>>>>>> streams from java.io were designed for simple synchronous use case.
>>>>>>>>> It's not that their design is flawed in some way, it's that they 
>>>>>>>>> doesn't seem to
>>>>>>>>> suit your needs. Have you considered using 
>>>>>>>>> java.nio.channels.AsynchronousFileChannel
>>>>>>>>> in your applications?
>>>>>>>>> -Pavel
>>>>>>>>> 
>>>>>>>>>> On 21 Oct 2016, at 17:08, Brunoais <brunoa...@gmail.com>
>>>>>>>>>>  wrote:
>>>>>>>>>> Any feedback on this? I'm really interested in implementing such 
>>>>>>>>>> BufferedReader/BufferedStreamReader to allow speeding up my 
>>>>>>>>>> applications without having to think in an asynchronous way or 
>>>>>>>>>> multi-threading while programming with it.
>>>>>>>>>> That's why I'm asking this here.
>>>>>>>>>> On 13/10/2016 14:45, Brunoais wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hi,
>>>>>>>>>>> I looked at BufferedReader source code for java 9 long with the 
>>>>>>>>>>> source code of the channels/streams used. I noticed that, like in 
>>>>>>>>>>> java 7, BufferedReader does not use an Async API to load data from 
>>>>>>>>>>> files, instead, the data loading is all done synchronously even 
>>>>>>>>>>> when the OS allows requesting a file to be read and getting a 
>>>>>>>>>>> warning later when the file is effectively read.
>>>>>>>>>>> Why Is BufferedReader not async while providing a sync API?
>>>>>>>>>>> 
>>> <BufferedNonBlockStream.java><Tests.java>
>>> 
> 

Reply via email to