First some background to give a real world use case: During usage of
VizReader.com I soon realized that one in every few hundred feeds
would break the script that is responsible for importing their
articles. Due to the fact that this could be because of any kind of
reason and might not even be the same reason in each case, plus the
fact that I don't miss the lack of content at all (for some reason
these problems almost always arise with some obscure programming blog
by some clever programmer who coded his own blog app) I'm not going to
try and fix it.
Instead, what if it would be possible to initiate a child process that
in turn calls the parent process which is then responsible for
spawning subsequent processes which then imports the articles of each
feed? The problem is that this can not happen in an async. fashion, we
need to start the next import child after the first is finished or
30sec. If we hit the timeout we simply kill and start the next one.
Letting things happen in async. fashion would swamp the RAM with
hundreds of picolisp instances (trust me I know, I did the mistake at
first), apart from the fact that the database is locked during each
import so all these processes would have to wait anyway.
The shell command responsible for initiating the process looks like
this: ./p lib/http.l -'client "localhost" 8080 "import_feeds.l"' -bye
And in import_feeds.l there is among other things a call to
initiateUpdatePresentFeeds:
(de initiateUpdatePresentFeeds ()
=A0=A0 (let Feeds (collect 'fid '+Feed)
=A0=A0=A0=A0=A0 (boss 'updatePresentFeeds (lit Feeds))))
After some discussions with Alex on IRC he pointed me to line 60 of
misc/stress.l which gave me enough pointers to come up with the
following:
(de updatePresentFeeds (Feeds)
=A0=A0 (for Feed Feeds
=A0=A0=A0=A0=A0 (let Cycles 30
=A0=A0=A0=A0=A0=A0=A0=A0 (if (fork)
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (let Pid @
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (while (and (n0 Cycles) (kill Pi=
d 0))
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (dec 'Cycles)
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (wait 1000))
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (kill Pid))
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (updatePresentFeed Feed)))))
The above works but this is where things are starting to get fussy for
me. How exactly is (fork) working? The reference example:
(unless (fork) (do 5 (println 'OK) (wait 1000)) (bye))
seems to imply that the current expression, ie. (unless ... ) is being
rerun in the child (becoming the contents of *Fork) whilst execution
continues to the (bye) in the parent. The (fork) call will return NIL
in the child so the (do ... ) expression is being run there. It also
seems to imply that the (bye) expression waits for the child to finish
before shutting down?
Anyway, having a wait call like that as is the case in my code could
result in various race conditions, according to Alex he had to make
(wait) "smart" in order to avoid common cases, I'm sure he can
elaborate further on the details... In any case it works in my case
but in order to avoid using (wait) Alex offered me the following
version:
(de updatePresentFeeds (Feeds)
(task -1000 0
Pid NIL
Cycles 0
Feeds Feeds
(cond
((nand Pid (kill Pid 0))
(if (pop 'Feeds)
(setq
Pid (or (fork) (updatePresentFeed @))
Cycles 30 )
(task -1000) ) )
((=3D0 (dec 'Cycles))
(and (kill Pid) (off Pid)) ) ) ) )
The main thing here is to initiate a task that is being run every
second, when all feeds have been imported we terminate through the
(task -1000) call. The first condition will be run initially (Pid is
NIL) and when there is no child process: (kill Pid 0) is NIL, ie.
after the last condition have been run 30 times or when the update
present feed function has concluded (it ends with a bye call).
The problem: It won't run at all, not even once and I have no idea
why, it all looks good to me.
Cheers,
Henrik Sarvell
--
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
