Re: Asyncronous IO

2008-10-10 Thread Alexander Burger
Hi Tomas, I attach a simple non-blocking echo server. There are some functions Nice! Conceptually it should not block, I am not sure how to test it though. Any ideas? Positive testing works well: abu:~/pico ./p dbg.l nb-ex.l : (more *Run) (5 (when (accept @) (task @ Sock @

Re: Asyncronous IO

2008-10-10 Thread Alexander Burger
Hi Konrad, But the underlying select function also provides non blocking writes. so that you can dely trying to write to a socket until it is ready for more data. Yep, this is also needed. It would appear that task and *Run have no way of using this part of the select function. Right.

Re: Asyncronous IO

2008-10-10 Thread Tomas Hlavaty
Hi Konrad, train. The task and *Run mechanism (if used with enough care) give us non blocking reads. But the underlying select function also provides non blocking writes. so that you can dely trying to write to a socket until it is ready for more data. I would say that select (in C) and

Re: Asyncronous IO

2008-10-10 Thread Tomas Hlavaty
Hi Alex, http://www.software-lab.de/refP.html#poll says: (poll 'cnt) - cnt | NIL Checks for the availability of data for reading on the file descriptor cnt. See also open, in and close. : (and (poll *Fd) (in @ (read))) # Prevent blocking The comment is a bit misleading as it does

Re: Asyncronous IO

2008-10-10 Thread John Duncan
If the socket is set not to block, then the socket will read as much data as is available and the underlying read call will return the number of bytes read, right? So the poll will still be useful in an nbio world. John On 10 Oct 2008, at 12:36 PM, Alexander Burger wrote: Hi Tomas, The

Re: Asyncronous IO

2008-10-09 Thread Alexander Burger
Hi Tomas, Not necessarily. If we use the select() mechanism provided by '*Run' and 'task', you can make input operations like 'listen', 'accept', 'read', 'rd' etc. non-blocking. select() does not make thinks non-blocking as far as I am aware. It just wakes up the process when

Re: Asyncronous IO

2008-10-09 Thread Alexander Burger
Hi Konrad, your response to Tomas's questions dosn't seem to have made it to the list. This is strange. I did receive it back, and it appeared in the archive. How can that be? Is it possible that it got lost in some spam filter? Most of the documentation I have seen seems to suggest that

Re: Asyncronous IO

2008-10-09 Thread Alexander Burger
On Thu, Oct 09, 2008 at 11:03:06AM +0200, Alexander Burger wrote: I'll try this out soon. It will probably involve only the removal of the blocking(YES, ex, sd2); calls in src/net.c, and additional checks for EAGAIN after some read() calls. Does anybody see a problem with that?

Re: Asyncronous IO

2008-10-09 Thread Alexander Burger
On Thu, Oct 09, 2008 at 08:12:28AM +0200, Alexander Burger wrote: Under Linux, select() may report a socket file descriptor as ready for reading, while nevertheless a subsequent read blocks. This could for example happen when data has arrived but upon examination has wrong checksum

Re: Asyncronous IO

2008-10-09 Thread Alexander Burger
Aarrgh ... Sorry, this was complete nonsense! I get into a busy waiting loop with that. I have to change it back, the old version was correct. Sorry for the confusion, - Alex -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Asyncronous IO

2008-10-09 Thread Alexander Burger
Hi all, Sorry for the confusion, Let me try to explain the situation: In the present system, it does not help at all to put the socket into non-blocking mode. The reason is that select() is conceptionally separated from read(). After select() signals (correctly or incorrectly) that data are

Re: Asyncronous IO

2008-10-09 Thread konrad Zielinski
Hi Alex, that sends me back to the drawing board a little. Yet again I recall seeing the fifo function but didn't realise I might need it : ) A few other things occured to me as I was working on this on the train. The task and *Run mechanism (if used with enough care) give us non blocking reads.

Re: Asyncronous IO

2008-10-09 Thread Tomas Hlavaty
Hi Alex, The drawback of implementing such a fully non-blocking system will be that the present separation of event generation (select) and data processing (read) cannot be held up any longer, and a completely different application flow is required. yes, indeed. I attach a simple

Re: Asyncronous IO

2008-10-08 Thread Alexander Burger
Hi Konrad, sufficently abstract the URL space from the physical disk. Most other mature frameworks allow your to creat a completly virtual url space which has no relation to physical files on disk. The way the forms.l maps a full path to its javascript file and puts it in the output Hmm,

Re: Asyncronous IO

2008-10-08 Thread Henrik Sarvell
Sounds clever, but very complicated maybe to maintain the scheduling? You mentioned multiplayer online games as a possibility, are the big ones with thousands of simultaneous users already operating under these principles? It would be great if you could point me to a resource on the basics behind

Re: Asyncronous IO

2008-10-08 Thread rand
Henrik, take a look at http://docs.python.org/library/asyncore.html There is even a few samples. I have used this extensively, reimplemented it directly in C a time or two. I think it would be relatively simple to implement using 'task' and '*Run'. Rand Henrik Sarvell wrote: Sounds

Re: Asyncronous IO

2008-10-08 Thread Alexander Burger
On Wed, Oct 08, 2008 at 09:55:16PM +1100, konrad Zielinski wrote: I belive the issue is that I have actually installed picoLisp under usr/local but don't use it exclusivly from that directory. As such I Ah, yes, in that case the path gets expanded. I didn't think about that, because I always

Re: Asyncronous IO

2008-10-08 Thread Henrik Sarvell
Konrad, Thanks for the links and clarifications, here is another one: http://en.wikipedia.org/wiki/Twisted_(software) However, for really understanding what this is about this link that you posted is very good: http://www.nightmare.com/medusa/medusa.html From the above Medusa link: Most

Re: Asyncronous IO

2008-10-08 Thread rand
For even more discussions, look at http://www.acme.com/software/ and then mini_httpd, micro_httpd, thttpd. These are really lightweight http servers. He did a lot of work comparing threading, forking, and async IO. Async IO wins big for http serving. Apache just happened to be at the right

Re: Asyncronous IO

2008-10-08 Thread konrad Zielinski
Ahh, I belive the issue is that I have actually installed picoLisp under usr/local but don't use it exclusivly from that directory. As such I get the following: script src=http://localhost:8080//usr/local/picoLisp/lib/form.js; type=text/javascript/script which includes an absolure path to

Re: Asyncronous IO

2008-10-08 Thread konrad Zielinski
Henrik, all of the references I'm aware of are based on Python. the standard Python library includes modules called asyncore and asynchat (sections 17.5 and 17.6 of the Python documentation. The original Asyncronous webserver (as far as I'm aware) is medusa http://www.nightmare.com/medusa/

Re: Asyncronous IO

2008-10-08 Thread rand
Henrik Sarvell wrote: Thanks Randall for that! So where do we start, where do I start, am I needed? I just don't have time now to really look into it. :-( Is the best way to simply review the Twisted source and reimplementing in Pico, keeping the OODB in mind all the time maybe? I would

Re: Asyncronous IO

2008-10-08 Thread Tomas Hlavaty
Hi Konrad and Henrik, Is the best way to simply review the Twisted source and reimplementing in Pico, keeping the OODB in mind all the time maybe? reading from http://en.wikipedia.org/wiki/Twisted_(software) Twisted supports an abstraction over raw threads=E2=80=94using a thread = as a

Re: Asyncronous IO

2008-10-08 Thread Alexander Burger
Hi Tomas, I do not think Konrad could achieve single process/thread http server without a bit of coding in C. PicoLisp I/O operations block I think Not necessarily. If we use the select() mechanism provided by '*Run' and 'task', you can make input operations like 'listen', 'accept', 'read',

Re: Asyncronous IO

2008-10-08 Thread Tomas Hlavaty
Hi Alex, In addition, I sometimes use the 'alarm' function (in combination with 'catch', 'throw' and 'finally') to ensure that no operation takes too long. I don't think alarm is good enough though. It is good for timeouts but not for schedulling. BTW: I found a typo in

Re: Asyncronous IO

2008-10-08 Thread Alexander Burger
Hi Tomas, I don't think alarm is good enough though. It is good for timeouts but not for schedulling. True. I would not use if for scheduling, though, but to terminate operations which might possibly take too long (e.g. 'connect' to another server). BTW: I found a typo in

Re: Asyncronous IO

2008-10-08 Thread Alexander Burger
Hi all, oops, sorry! Forget what I wrote: A .html file will be directly sent to the client, without any forking. And also .l files do not automatically cause the fork, but only if (app) is called during their execution. This was wrong. The 'server' function in lib/http.l will always fork

Re: Asyncronous IO

2008-10-08 Thread konrad Zielinski
Hi All, In the matter of how to begin. Twisted is I think the wrong place. It is a very large and complicated beast which has grown in its own unquie direction. So much so that it is incompatable with a lot of standard python tools and libraries, uses its own logging system etc. In copying

Re: Asyncronous IO

2008-10-08 Thread Tomas Hlavaty
Hi Alex, Not necessarily. If we use the select() mechanism provided by '*Run' and 'task', you can make input operations like 'listen', 'accept', 'read', 'rd' etc. non-blocking. select() does not make thinks non-blocking as far as I am aware. It just wakes up the process when something is

Re: Asyncronous IO

2008-10-08 Thread Tomas Hlavaty
Hi Alex, thanks for the explanation! Tomas -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]

Re: Asyncronous IO

2008-10-08 Thread konrad Zielinski
Hi Alex, your response to Tomas's questions dosn't seem to have made it to the list. Most of the documentation I have seen seems to suggest that Sockets need to be put into non blocking mode explicitly, ie using select() is not sufficent. At a low level attempting to read from a nonblocking

Re: Asyncronous IO

2008-10-07 Thread konrad Zielinski
Hi Henrik the whole idea of an asycronous server is that you only use one process. to handle all requests. this way session data can be stored without using an external symbols. Even further several clients may share state in some way, though the only example I can think of off the top of my