Unsubscribe
Re: Workers for multicore processing (Question and RFC)
I believe modern Linux systems have much higher than 1024 file descriptor per process limits. E.g. on my system $ uname -srv Linux 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 $ cat /proc/sys/fs/file-max 2013532 And you can adjust this limit to your requirements. 2017-02-25 12:26 GMT+01:00 Joh-Tob Schäg <johtob...@gmail.com>: > There are limits on how many processes can exist at the same time and > having more 'later processes than cores is wasteful and Linux only allows > for 1024 file descriptors per process. > ? (for N 1 (later (cons) (* N N))) > !? (pipe (pr (prog (* N N > Pipe error: Too many open files > ? N > -> 339 > > Is there some "worker" implementation which keeps a queue of task to do > and creates N worker processes which take tasks from the queue when they > are finished? > > Ideas: > Task would be copy of environment + function + values to apply the > function too. There would be some optimization potential in case the > function and environment is static, or reseted each time to a constant > value but it should be possible to fetch a copy of the current environment > to. > Such a pool of workers is best represented with a symbol on which certain > functions are applied which are may stored inside the symbol. This could be > archived with some overhead with the picolisp object system when compared > to a manual implementation since the worker queue should be accessible fast > and putting it in val of the symbol would be the easiest way to achieve > that. > > Further thoughts: > It might be wise to give the possibility to register a callback on the > return value. > This would allow to make things more efficient. > For example: > > (prog1 # Parallel background calculation of square numbers >(mapcan '((N) (later (cons) )) (range 1 100)) >(wait NIL (full @)) ) > > could be: > > (pool "squares.db") > (with-workerpool W '((workers . 2) (callback . '((R) (put *DB (car R) (cadr > R )) >(mapcar '((N) (add_task 'W '((N) (cons N (* N N))) N ) (range 1 100)) )] > > #Normally the with-workerpool should not finish until the queue is empty > > How ever it should also be possible to do something like that: > > > (with-workerpool W '((callback . NIL) (return . nil) (output . "+file") (wait > . NIL)) >(for X 1000 > (add_task 'W '((N) (ifn (= N (apply '* (prime-factors N))) > (print N))) X ] # workers run in background main thread can continue > > This usage may seem overly complex at first and the proposed interface is > horrible. However i think that such an library could be an valuable addition > to picolisp. > > Lets discuss the idea itself, your proposals for the interface and the > implementation. > > -- Petr Gladkikh
Re: Future of PicoLisp?
I admit that I am a passerby here but my impression is that things around PicoLisp feel closed and this is not about the code itself. In particular, there seems to be no public source code repository so source history can only be inferred somehow from releases. I see that some people already try to mirror it (see, e.g. https://github.com/taij33n/picolisp) but since it is not "official" there's no development or associated discussions. Since revision control is essential for any non-trivial project, the lack of it is surprising. Also there's no public bug tracker. Instead I have to subscribe to this list so it is hard to track what happened to issues. That said removing some friction for outsiders would help to keep things running. 2017-02-03 8:47 GMT+01:00 Alexander Burger <a...@software-lab.de>: > Hi all, > > the future of PicoLisp is dark. I'm not sure if it can survive in packaged > distribution. > > Ubuntu doesn't support it any more, probably due to the PIE (position > independent executable) on x86-64. > > And at least on Android they seem to demand switching to Clang. The 32-bit > versions of PicoLisp (pil32 and mini) which are written in C cannot be > compiled > on Clang, because Clang doesn't support dynamically allocated arrays, which > pil32 depends on. As far as I notices, pil64 also has trouble on > Clang/Android. > > :( Alex > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe > -- Petr Gladkikh
Subscribe
-- Petr Gladkikh
