Re: Fork and synchronous execution of child processes.
Hi Henrik, > I have no idea either, very weird. I also tested to simply pass (1 2 3 > 4) and using your updatePresentFeed function, however I put (traceAll) > at the top of updatePresentFeeds, this is what I get: > > task : -1000 (0 Pid NIL Cycles 0 Feeds Feeds (cond ((nand Pid (kill 'traceAll' doesn't help here very much, as it traces only Lisp-level functions which are not called here (except 'task' itself). You need to trace explicitly functions you are interested in, or set breakpoints. Anyway, does the posted code work on itself, if you just 'load' it into a ./dbg environment? That's how I tested it. > Can you think of anything at all that might be interfering with the > task execution, for instance running two servers/tasks listening on > different ports at the same time in this very same parent? This should not matter. You could manually insert a breakpoint into the code to have it stop when the task is executed: Cycles 0 Feeds Feeds (! cond ((nand Pid (kill Pid 0)) (if (pop 'Feeds) This will cause it to stop just before the 'cond'. Then you can single step into the expression(s) to see what happens. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Fork and synchronous execution of child processes.
I have no idea either, very weird. I also tested to simply pass (1 2 3 4) and using your updatePresentFeed function, however I put (traceAll) at the top of updatePresentFeeds, this is what I get: task : -1000 (0 Pid NIL Cycles 0 Feeds Feeds (cond ((nand Pid (kill Pid 0)) (if (pop 'Feeds) (setq Pid (or (fork) (updatePresentFeed @)) Cycles 30) (task -1000))) ((=3D0 (dec 'Cycles)) (and (kill Pid) (off Pid) task =3D (-1000 0 (job '((Pid) (Cycles . 0) (Feeds 1 2 3 4)) (cond ((nand Pid (kill Pid 0)) (if (pop 'Feeds) (setq Pid (or (fork) (updatePresentFeed @)) Cycles 30) (task -1000))) ((=3D0 (dec 'Cycles)) (and (kill Pid) (off Pid)) I suspect that the above will not help much. Can you think of anything at all that might be interfering with the task execution, for instance running two servers/tasks listening on different ports at the same time in this very same parent? On Sat, Sep 4, 2010 at 5:26 PM, Alexander Burger wrot= e: > Hi Henrik, > >> me. How exactly is (fork) working? The reference example: >> >> (unless (fork) (do 5 (println 'OK) (wait 1000)) (bye)) >> >> seems to imply that the current expression, ie. (unless ... ) is being >> rerun in the child (becoming the contents of *Fork) whilst execution >> continues to the (bye) in the parent. The (fork) call will return NIL >> in the child so the (do ... ) expression is being run there. It also >> seems to imply that the (bye) expression waits for the child to finish >> before shutting down? > > This description is not correct in some points. > > The PicoLisp 'fork' maps to the underlying fork() system call. When > fork() is called, an new process is created, and both processes (the > parent and the child) continue to execute from that point. At that > moment, both processes are identical, the only difference is that in the > parent process fork() returns the child's process ID, while in the child > 0 is returned. > > The PicoLisp (fork) function returns a number (the child's PID) in the > parent process, and NIL in the child process. So in the above example > > =A0 (unless (fork) > > is executed in the current process (which will be the "parent" > thereafter), and (fork) returns the PID of the child. This is non-NIL, > so the body of the 'unless' call is skipped by the parent, and it will > continue execution with the next expression (not shown here). > > In the child process, (fork) returns NIL, and the body of the 'unless' > call is executed. It consists of the two expressions > > =A0 (do 5 (println 'OK) (wait 1000)) > =A0 (bye) > > This is the code the child process sees. It prints five times 'OK' and > then exits (terminating the child process). > > Note that the (bye) is probably important, because otherwise the child > would continue with the next (not shown) expression which was already > executed by the parent 5 seconds before. > > Thus, (unless (fork) (do-something) (bye)) is a typical pattern to have > (do-something) done in a child process. In that regard, the formulation > "(unless ..) is being rerun in the child" is a bit misleading, as the > 'unless' is begins execution only in the parent, but after its condition > (the (fork)) is evaluated, we suddenly have two processes which execute > different branches in the code (the next statement in the parent, and > the body of 'unless' in the child). That's the reason for the name of > that system call, "fork". > > Then the formulation "(becoming the contents of *Fork)" is not correct. > The global variable '*Fork' is not changed in that process, it may be > set under program control _before_ the call to (fork) to contain > expressions which should be executed in all child processes. Perhaps you > meant '*Pid'? This is changed to a new value in the child process. > > Also, "whilst execution continues to the (bye) in the parent" is not > right, as the parent never sees the (bye). It is only there for the > child to terminate after it did its job. > > >> Anyway, having a wait call like that as is the case in my code could >> result in various race conditions, according to Alex he had to make >> (wait) "smart" in order to avoid common cases, I'm sure he can >> elaborate further on the details... In any case it works in my case > > Yes, though I would not say that there are race conditions. > > 'wait' is actually the PicoLisp "Event Handler". It can be called > explicitly, and is called implicitly by other functions (see the > reference of '*Run'). > > This event handler takes care of several things, listening on file > descriptors and watching timeout values. This includes all expressions > in '*Run' (typically installed and de-installed by 'task'), but also > internal events resulting from interprocess communication: The parent > relays 'tell' messages from a child to all other children, accepts > 'boss' commands (via 'hear'), detects when child processes terminate to > clean up their resources, and is needed to synchronize the database > between all child processes. > > So typically 'w
Re: Porting PicoLisp to Plan9
Plan9 has a GCC port on which pico compiles with a few changes. (The
changes are required because the GCC port itself is not complete!).
The binary runs with no problems.
About the lack of shared libraries on Plan9: Plan9 don't need shared
libraries because components on Plan9 can interact using just four
system calls - open, read, write and close. As an example consider a
graphics library that could be shared by various applications:
/* Pseudo C */
handle =3D LoadLibrary ("graphics.so");
CallFn (handle, "drawRect", 0, 0, 200, 200);
Close (handle);
The equivalent plan9 code will be something like this:
handle =3D open ("/sys/graphics/cntl");
cmd =3D "drawRect 0 0 200 200";
write (handle, cmd, strlen (cmd));
close (handle);
The interesting thing is that /sys/graphics/cntl/ could be on a remote
file system. This makes distributed computing easy and shared
libraries redundant.
Thanks,
-- Vijay
On Fri, Sep 3, 2010 at 1:17 PM, Mateusz Jan Przybylski
wrote:
> Hi Tomas,
>
>
> On Thursday 02 September 2010 19:32:40 you wrote:
>> > makes up my daily work environment ;)
>>
>> cool. =A0I looked at it briefly but couldn't find my way around. =A0Do y=
ou
>> really use that acme editor or how it's called? =A0Is it possible to run
>> it as console only with vi (or emacs;-)?
>
>
>
> To clear it up, i use the `Plan 9 from User Space' (i.e., the tools porte=
d to
> POSIX).
> Mostly because of Acme -- for very quick edit/[compile/]run/open-file-wit=
h-
> error cycle, and for the Rc shell -- practical for scripting.
>
> http://swtch.com/plan9port/man/man1/install.html
>
> Acme requires GUI (it could be implemented with exact same semantics in t=
ext
> mode, I guess, but it isn't). However, it uses little bandwidth, and so i=
t's
> usual and natural to use it over network in both native Plan 9 and ported=
X11
> version. ssh -Y -C gets the job done even on slow links.
>
> It took me a few (3?) months to learn to use Acme efficiently.
>
> Acme takes a three-buttoned mouse, preferably with a scroll. It's pretty
> important to have all three physical buttons, since a lot is done via cho=
rding
> two buttons.
>
> Acme understands and uses a certain text processing language, based on se=
d's.
> It's executed via the `Edit THE_PROGRAM'.
> For example, I indent selected portion of text with:
> Edit s,^, =A0 =A0 =A0 ,g
>
> To find current selection address(es), I do
> Edit =3D
> etc.etc.
>
> You ought to have plumber up and running before you start Acme
> http://swtch.com/plan9port/man/man4/plumber.htm
> to get some extra functionality, mostly related to opening files you indi=
cate
> with rightclick.
> You want the plumber started on the same machine the Acme is running (one=
of
> POSIX limitations)
>
> My plumbing rules (the file is $HOME/lib/plumbing) are as follows:
>
> #file / line in PHP format
> type is text
> data matches '(.+) on line ([0-9]+)'
> arg isfile $1
> data set $file
> attr add addr=3D$2
> plumb to edit
>
> include /usr/local/plan9/plumb/initial.plumbing
>
>
>
>>
>> > As a curiosity, there are no shared libraries in P9 -- only what we ca=
ll
>> > `static linking'. It's awesome in the longer run <3
>>
>> What's so awesome about it?
>
> No DLL hell.
> Programs (and thus processes) and libraries are very small.
> Greater percentage of code & data fits in CPU cache. Less indirect & more
> direct data access & code calls/jumps. etc. etc.
> http://harmful.cat-v.org/software/dynamic-linking/
>
> and it avoids the worst bane of DLL so far:
> http://harmful.cat-v.org/software/dynamic-linking/versioned-symbols
>
>
>
> tl;dr:
> Plan 9 from User Space provides tools that may appeal to a long-time POSI=
X
> user ;-)
>
>
> Cheers,
> --
> Mateusz Jan Przybylski
>
>
> ``One can't proceed from the informal to the formal by formal means.''
> --
> UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=3dunsubscribe
>
--
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Fork and synchronous execution of child processes.
Hi Henrik, > me. How exactly is (fork) working? The reference example: > > (unless (fork) (do 5 (println 'OK) (wait 1000)) (bye)) > > seems to imply that the current expression, ie. (unless ... ) is being > rerun in the child (becoming the contents of *Fork) whilst execution > continues to the (bye) in the parent. The (fork) call will return NIL > in the child so the (do ... ) expression is being run there. It also > seems to imply that the (bye) expression waits for the child to finish > before shutting down? This description is not correct in some points. The PicoLisp 'fork' maps to the underlying fork() system call. When fork() is called, an new process is created, and both processes (the parent and the child) continue to execute from that point. At that moment, both processes are identical, the only difference is that in the parent process fork() returns the child's process ID, while in the child 0 is returned. The PicoLisp (fork) function returns a number (the child's PID) in the parent process, and NIL in the child process. So in the above example (unless (fork) is executed in the current process (which will be the "parent" thereafter), and (fork) returns the PID of the child. This is non-NIL, so the body of the 'unless' call is skipped by the parent, and it will continue execution with the next expression (not shown here). In the child process, (fork) returns NIL, and the body of the 'unless' call is executed. It consists of the two expressions (do 5 (println 'OK) (wait 1000)) (bye) This is the code the child process sees. It prints five times 'OK' and then exits (terminating the child process). Note that the (bye) is probably important, because otherwise the child would continue with the next (not shown) expression which was already executed by the parent 5 seconds before. Thus, (unless (fork) (do-something) (bye)) is a typical pattern to have (do-something) done in a child process. In that regard, the formulation "(unless ..) is being rerun in the child" is a bit misleading, as the 'unless' is begins execution only in the parent, but after its condition (the (fork)) is evaluated, we suddenly have two processes which execute different branches in the code (the next statement in the parent, and the body of 'unless' in the child). That's the reason for the name of that system call, "fork". Then the formulation "(becoming the contents of *Fork)" is not correct. The global variable '*Fork' is not changed in that process, it may be set under program control _before_ the call to (fork) to contain expressions which should be executed in all child processes. Perhaps you meant '*Pid'? This is changed to a new value in the child process. Also, "whilst execution continues to the (bye) in the parent" is not right, as the parent never sees the (bye). It is only there for the child to terminate after it did its job. > Anyway, having a wait call like that as is the case in my code could > result in various race conditions, according to Alex he had to make > (wait) "smart" in order to avoid common cases, I'm sure he can > elaborate further on the details... In any case it works in my case Yes, though I would not say that there are race conditions. 'wait' is actually the PicoLisp "Event Handler". It can be called explicitly, and is called implicitly by other functions (see the reference of '*Run'). This event handler takes care of several things, listening on file descriptors and watching timeout values. This includes all expressions in '*Run' (typically installed and de-installed by 'task'), but also internal events resulting from interprocess communication: The parent relays 'tell' messages from a child to all other children, accepts 'boss' commands (via 'hear'), detects when child processes terminate to clean up their resources, and is needed to synchronize the database between all child processes. So typically 'wait' processes a list of events. Now you may well call another 'wait' in a task, and this in fact happens frequently during database synchronization between child processes, or when GUI functions execute. However, 'wait' must ensure that while processing a list of events, a recursively called 'wait' does not re-execute the same event a second time. For that, the inner 'wait' takes care not to look at the events in the outer 'wait', and this may result that outer events will never get executed if the inner 'wait' is run in a loop in some task. Anyway, this may get rather confusing, and isn't the way 'task's are intended. I mean, instead that a task establishes its own timing loop using nested 'wait's, one central capability of tasks _is_ being a timer. That's why I recommended not to write such a loop in the task, but execute the checking for that child process periodically with. > but in order to avoid using (wait) Alex offered me the following > version: > > (de updatePresentFeeds (Feeds) >(task -1000 0 > Pid NIL > Cycles 0 > Feeds Feeds > (
Fork and synchronous execution of child processes.
First some background to give a real world use case: During usage of VizReader.com I soon realized that one in every few hundred feeds would break the script that is responsible for importing their articles. Due to the fact that this could be because of any kind of reason and might not even be the same reason in each case, plus the fact that I don't miss the lack of content at all (for some reason these problems almost always arise with some obscure programming blog by some clever programmer who coded his own blog app) I'm not going to try and fix it. Instead, what if it would be possible to initiate a child process that in turn calls the parent process which is then responsible for spawning subsequent processes which then imports the articles of each feed? The problem is that this can not happen in an async. fashion, we need to start the next import child after the first is finished or 30sec. If we hit the timeout we simply kill and start the next one. Letting things happen in async. fashion would swamp the RAM with hundreds of picolisp instances (trust me I know, I did the mistake at first), apart from the fact that the database is locked during each import so all these processes would have to wait anyway. The shell command responsible for initiating the process looks like this: ./p lib/http.l -'client "localhost" 8080 "import_feeds.l"' -bye And in import_feeds.l there is among other things a call to initiateUpdatePresentFeeds: (de initiateUpdatePresentFeeds () =A0=A0 (let Feeds (collect 'fid '+Feed) =A0=A0=A0=A0=A0 (boss 'updatePresentFeeds (lit Feeds After some discussions with Alex on IRC he pointed me to line 60 of misc/stress.l which gave me enough pointers to come up with the following: (de updatePresentFeeds (Feeds) =A0=A0 (for Feed Feeds =A0=A0=A0=A0=A0 (let Cycles 30 =A0=A0=A0=A0=A0=A0=A0=A0 (if (fork) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (let Pid @ =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (while (and (n0 Cycles) (kill Pi= d 0)) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (dec 'Cycles) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (wait 1000)) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (kill Pid)) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (updatePresentFeed Feed) The above works but this is where things are starting to get fussy for me. How exactly is (fork) working? The reference example: (unless (fork) (do 5 (println 'OK) (wait 1000)) (bye)) seems to imply that the current expression, ie. (unless ... ) is being rerun in the child (becoming the contents of *Fork) whilst execution continues to the (bye) in the parent. The (fork) call will return NIL in the child so the (do ... ) expression is being run there. It also seems to imply that the (bye) expression waits for the child to finish before shutting down? Anyway, having a wait call like that as is the case in my code could result in various race conditions, according to Alex he had to make (wait) "smart" in order to avoid common cases, I'm sure he can elaborate further on the details... In any case it works in my case but in order to avoid using (wait) Alex offered me the following version: (de updatePresentFeeds (Feeds) (task -1000 0 Pid NIL Cycles 0 Feeds Feeds (cond ((nand Pid (kill Pid 0)) (if (pop 'Feeds) (setq Pid (or (fork) (updatePresentFeed @)) Cycles 30 ) (task -1000) ) ) ((=3D0 (dec 'Cycles)) (and (kill Pid) (off Pid)) ) ) ) ) The main thing here is to initiate a task that is being run every second, when all feeds have been imported we terminate through the (task -1000) call. The first condition will be run initially (Pid is NIL) and when there is no child process: (kill Pid 0) is NIL, ie. after the last condition have been run 30 times or when the update present feed function has concluded (it ends with a bye call). The problem: It won't run at all, not even once and I have no idea why, it all looks good to me. Cheers, Henrik Sarvell -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
