Re: [Haskell-cafe] standard poll/select interface
Hello Bulat, Thursday, February 09, 2006, 10:24:59 PM, you wrote: >>> if you can make select/poll transformer, at least for testing >>> purposes, that will be really great. JM>> Yeah, I will look into this. the basic select/poll call will have to be JM>> pretty low level, but hopefully it will allow interesting higher level JM>> constructs based on your streams or an evolution of them. sorry, John, as i now see, Einar already implemented select/epoll machinery in the alt-network lib. moreover, now he promised to extract this functionality to make universal async i/o layer library. the only thing that i don't know - whether he is ready to develop universal API for these modules, as you initially proposed. as this universal API will be done, i will roll up the Stream transormer that uses it and therefore allows async i/o both with files and sockets on any platform where this API can be implemented -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] standard poll/select interface
Hello Einar,
Friday, February 10, 2006, 2:09:08 AM, you wrote:
>> as i understand this idea, transformer implementing async i/o should
>> intercept vGetBuf/vPutBuf calls for the FDs, start the appropriate
>>
>> type FD = Int
>> vGetBuf_async :: FD -> Ptr a -> Int -> IO Int
>> vPutBuf_async :: FD -> Ptr a -> Int -> IO Int
EK> Please don't fix FD = Int, this is not true on some systems,
EK> and when implementing efficient sockets one usually wants
EK> to hold more complex state.
the heart of the library is class Stream. both "File" and "Socket"
should implement this interface. just now i use plain "FD" to
represent files, but that is temporary solution - really file also
must carry additional information: filename, open mode, open/closed
state. This "File" will be an abstract datatype, what can be based not
on FD in other operating systems.
The same applies to the "Socket". it can be any type what carry enough
information to work with network i/o.
implementation of async i/o should have a form of Stream Transformer,
which intercepts only the vGetBuf/vPutBuf operations and pass other
operations as is:
data AsyncFD = AsyncFD FD ... {-additional fields-}
instance Stream IO AsyncFD where
vIsEOF (AsyncFD h ...) = vIsEOF h
vClose (AsyncFD h ...) = vClose h
vGetBuf (AsyncFD h ...) ptr size = vGetBuf_async h ptr size
as far as i see, the select/epoll don't need to know anything about
file/socket except for its descriptor(FD) ? in this case we can make
the Async transformer universal, compatible with both files and
sockets:
data Async h = Async h ... {-additional fields-}
addAsyncIO h = do
.
return (Async h ...)
instance (Stream IO h) => Stream IO (Async h) where
vIsEOF (Async h ...) = vIsEOF h
vClose (Async h ...) = vClose h
vGetBuf (Async h ...) ptr size = doBlockingOp "read" h $ vGetBuf h ptr size
this transformer can be made universal, supporting select/epoll/...
implementations via additional parameter to the "addAsyncIO", or it
can be a series of transformers, one for each method of async i/o. if
we have developed common API for async i/o as John suggested, then one
universal transformer working via this API can be used
>> this simply means that the Streams library cannot be used with JHC,
>> what is bad news, because it is even more rich than GHC's System.IO.
>> jhc had chance to get modern I/O library. but it lost that chance :)
EK> I think it is more like "all haskell-prime programs". Seriously,
EK> if we design a new IO subsystem it would be quite nice to be
EK> able to use it from standard conforming programs.
EK> Maybe things can be reformulated in a way that will be compatible
EK> with haskell-prime.
or haskell-prime can be reformulated ;) as h' will be defined in
first iteration, i will check my lib and say to comittee what i will
need to omit from my library to be compatible with this standard.
then we can decide :) just at current moment, support for complex
class hierrachies outside of Hugs/GHC is very poor
>> please look. at this moment Sreams library lacks only a few important
>> features, already implemented in GHC's System.IO: sockets, line
>> buffering and async i/o. moreover, i don't have an experience in
>> implementing the async i/o, so foreign help is really necessary
EK> If you want I can look at getting network-alt to implement the
EK> interface.
please look. basically, Sock should be made an instance of Stream with
implementations of vGetBuf/vPutBuf and other operations, as much as possible.
it should be easy, see the FD/Handle instances of Stream for example
and your support of select/poll should go into the tranformer(s). this
will allow to use async i/o not only for your own Sock type, but for
files, for the sockets from the old library and so on
--
Best regards,
Bulatmailto:[EMAIL PROTECTED]
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] standard poll/select interface
On 09.02 22:24, Bulat Ziganshin wrote: > as i understand this idea, transformer implementing async i/o should > intercept vGetBuf/vPutBuf calls for the FDs, start the appropriate > async operation, and then switch to another Haskell threads. the I/O > manager thread should run select() in cycle and when the request is > finished, wake up the appropriate thread. what's all. if you will ever > need, this implementation can then be used to extend GHC's System.IO > internals with the support for new async i/o managers (as i > understand, select() is now supported by GHC, but poll(), kqueue() is > not supported?). the only difference that my lib gives an opportunity > to test this implementation without modifying GHC I/O internals, what > is somewhat simpler. so, interface for async vGetBuf/vPutBuf routines > should be the same as for read/write: > > type FD = Int > vGetBuf_async :: FD -> Ptr a -> Int -> IO Int > vPutBuf_async :: FD -> Ptr a -> Int -> IO Int Please don't fix FD = Int, this is not true on some systems, and when implementing efficient sockets one usually wants to hold more complex state. > JM> Don't take the absence of a feature in jhc to mean I don't like or want > JM> that feature. There are a lot of things I don't have but that I'd > JM> definitly want to see in the language simply because I was only shooting > JM> for H98 to begin with and was more interested in a lot of the back end > JM> stuff. You should figure out the nicest design that uses just the > JM> extensions needed for the design you want. it could help us decide what > JM> goes into haskell-prime to know what is absolutely needed for good > JM> design and what is just nice to have. > > this simply means that the Streams library cannot be used with JHC, > what is bad news, because it is even more rich than GHC's System.IO. > jhc had chance to get modern I/O library. but it lost that chance :) I think it is more like "all haskell-prime programs". Seriously, if we design a new IO subsystem it would be quite nice to be able to use it from standard conforming programs. Maybe things can be reformulated in a way that will be compatible with haskell-prime. > please look. at this moment Sreams library lacks only a few important > features, already implemented in GHC's System.IO: sockets, line > buffering and async i/o. moreover, i don't have an experience in > implementing the async i/o, so foreign help is really necessary If you want I can look at getting network-alt to implement the interface. - Einar Karttunen ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] standard poll/select interface
Hello John, Thursday, February 09, 2006, 3:19:30 AM, you wrote: >> JM> If we had a good standard poll/select interface in System.IO then we >> JM> actually could implement a lot of concurrency as a library with no >> JM> (required) run-time overhead. I'd really like to see such a thing get >> JM> into the standard. Well, mainly it would just be a really useful thing >> JM> to have in general. If others think it is a good idea I can try to come >> JM> up with a suitable API and submit it to the repo. >> >> i have delayed answering to this letter until i announced my Streams >> library. now i can say that such API already exists - in terms of my >> library you need just to write an transformer that intercepts >> vGetBuf/vPutBuf calls and pass them to the select/poll machinery. so >> you can write such transformer just now and every program that uses >> Streams will benefit from its usage. Converting programs that use >> Handles to using Streams should be also an easy task. JM> I was actually asking for something much more modest, which was the JM> routine needed to pass them to the select/poll machinery. but yeah, what JM> you say is one of my expected uses of such a routine. Once a standard IO JM> library settles down, then I can start working on the exact API such a JM> routine would have. but if all will wait while the library settles down, it will never occur :) your work can change design of library, like the my library itself can change the shape of haskell' :) at this moment, i just developed the library which satisfy demands in extending current I/O library by new features, such as Unicode support, high speed, portability to other compilers, binary i/o, i/o for packed strings, and asynchronous i/o using methods other than select(). but i don't implement all these features actually, i just developed infrastructure, in which all these features can be easily added. unlike the System.IO library, you don't need to ask someone to implement new features or make corrections in foreign sources. you just need to develop module what implements this standard Stream interface and then it can be used as easy as transformers from the library itself as i understand this idea, transformer implementing async i/o should intercept vGetBuf/vPutBuf calls for the FDs, start the appropriate async operation, and then switch to another Haskell threads. the I/O manager thread should run select() in cycle and when the request is finished, wake up the appropriate thread. what's all. if you will ever need, this implementation can then be used to extend GHC's System.IO internals with the support for new async i/o managers (as i understand, select() is now supported by GHC, but poll(), kqueue() is not supported?). the only difference that my lib gives an opportunity to test this implementation without modifying GHC I/O internals, what is somewhat simpler. so, interface for async vGetBuf/vPutBuf routines should be the same as for read/write: type FD = Int vGetBuf_async :: FD -> Ptr a -> Int -> IO Int vPutBuf_async :: FD -> Ptr a -> Int -> IO Int i think that implementations for ghc and jhc should be slightly different, though, because of different ways to implement multi-threading. but the I/O manager should be the same - it just receives info about I/O operations to run and returns information about completed ones. ... well, this I/O manager should implement just one operation: performIO :: Request -> IO () type Request = (IOType, FD, Ptr a, Int, Notifier) data IOType = Read | Write | ... type Notifier = Result -> IO () data Result = OK Int | Fail ErrorInfo "performIO" starts new I/O operation. On the completion of this operation, Notifier is called with information about results of execution. so, for the GHC the following should work: vGetBuf_async fd ptr size = do done <- newMVar let notifier = putMVar done () performIO (Read, fd, ptr, size, notifier) takeMVar done for JHC, the body of "vGetBuf_async" may be different if you will find this interface reasonable, at least for the first iteration, i will develop appropriate transformer, so for you remains "only" the implementation of "performIO" >> of course, Streams library is not some standard just now, and moreover >> - it is not compatible with JHC. the greatest problem is what i using >> type classes extensions available in GHC/Hugs what is not in H98 >> standard. so, i'm interested in pushing Haskell' to accept most >> advanced possible extensions in this area and, of course, in actual >> implementing these extensions in the Haskell compilers. alternative >> way to make Streams available to wider range of Haskell compilers is >> to strip support of streams working in monads other that IO. JM> Don't take the absence of a feature in jhc to mean I don't like or want JM> that feature. There are a lot of things I don't have but that I'd JM> definitly want to see in the language simply because I was only shooting JM> for H98 to begin with and was
