Re: Subprocess API.
On Mon, 25 Sep 2017 19:59:39 +0100 Chris Vinewrote: > ... you could consider launching the new process in C code via the > guile FFI so you can ensure that no non-async-signal-safe code is > called at the wrong time; but presumably you would still have by some > means to prevent the garbage collector from being able to start a > memory reclaiming run in the new process after the fork and before the > exec, and again I do not know how you would do that. You would also > need to block system asyncs before forking (and unblock after the fork > in the original process) but that is trivial to do. On reflection I don't think there is an issue with the garbage collector if you adopted this approach. After forking there is only one thread running in the new process - the thread of execution of the forking thread - and provided that the new process does not attempt to allocate memory after the fork and before the exec, I doubt the garbage collector has a way in which it can be provoked to begin trying to reclaim memory in the new process.
Re: Subprocess API.
On Mon, 25 Sep 2017 19:14:22 +0200 Mathieu Othacehewrote: > Hi Chris, > > > This works exactly as you would expect from its POSIX equivalents > > and has the advantage that you can read from the pipe as the > > sub-process is proceeding rather than just collect at the end. > > Thank you ! Following your suggestion, I ended-up with : > > --8<---cut here---start->8--- > (let* ((err-pipe (pipe)) >(out-pipe (pipe)) >(read-out (car out-pipe)) >(write-out (cdr out-pipe)) >(read-err (car err-pipe)) >(write-err (cdr err-pipe)) >(pid (run-concurrently+ > (apply tail-call-program "...") > (write-out 1) > (write-err 2))) >(ret (status:exit-val (cdr (waitpid pid) > (close-port write-out) > (close-port write-err) > (let ((output (read-string read-out)) > (error (read-string read-err))) > (close-port read-out) > (close-port read-errs > (case ret > ((0) output) > (else (raise ...) > --8<---cut here---end--->8--- > > which seems to work. However, run-concurrently+ uses "primitive-fork" > which is forbiden in a multi-thread context (sadly, mine). > > Do you have any idea on how to overcome this ? Any launching of a new process requires a fork and if (as appears to be your intention) you want to replace the process image with a new one, an exec. As you appear to know, POSIX allows only async-signal-safe functions to be called in a multi-threaded program between the fork and the exec, although glibc does relax this somewhat. Since anything you do in guile between the fork and the exec has the potential to allocate memory, that appears to mean that, as you say, you cannot call primitive-fork in a guile program at a time when it is running more than one thread. If so, I do not know how to circumvent that: you could consider launching the new process in C code via the guile FFI so you can ensure that no non-async-signal-safe code is called at the wrong time; but presumably you would still have by some means to prevent the garbage collector from being able to start a memory reclaiming run in the new process after the fork and before the exec, and again I do not know how you would do that. You would also need to block system asyncs before forking (and unblock after the fork in the original process) but that is trivial to do. As regards your code, if you do not need to distinguish between stdout and stderr, you would do better to have only one pipe and use the write port of the pipe for both of descriptors 1 and 2. That means that you could read the pipe while the new process is proceeding rather than after it has finished (which risks filling up the pipe): just loop reading the read end of the pipe until an eof-object is received, and then call waitpid after that. Chris
Re: Subprocess API.
Hi Chris, > This works exactly as you would expect from its POSIX equivalents and > has the advantage that you can read from the pipe as the sub-process is > proceeding rather than just collect at the end. Thank you ! Following your suggestion, I ended-up with : --8<---cut here---start->8--- (let* ((err-pipe (pipe)) (out-pipe (pipe)) (read-out (car out-pipe)) (write-out (cdr out-pipe)) (read-err (car err-pipe)) (write-err (cdr err-pipe)) (pid (run-concurrently+ (apply tail-call-program "...") (write-out 1) (write-err 2))) (ret (status:exit-val (cdr (waitpid pid) (close-port write-out) (close-port write-err) (let ((output (read-string read-out)) (error (read-string read-err))) (close-port read-out) (close-port read-err) (case ret ((0) output) (else (raise ...) --8<---cut here---end--->8--- which seems to work. However, run-concurrently+ uses "primitive-fork" which is forbiden in a multi-thread context (sadly, mine). Do you have any idea on how to overcome this ? Thanks, Mathieu
Re: Subprocess API.
On Sat, 23 Sep 2017 11:58:34 +0200 Mathieu Othacehewrote: > Hi, > > I recently used "open-pipe*" to launch a process but was unable to > read from stderr. This subject was already discussed on this ml > here : > > https://lists.gnu.org/archive/html/guile-user/2015-04/msg3.html > > Racket seems to have procedures to provide stdout/stdin/stderr ports > for a given subprocess[1]. > > Mark, you said this subject was on your TODO list, is there anything > available or would it be possible to develop a racket like API ? The run-concurrently+ procedure in guile-lib ( http://www.nongnu.org/guile-lib/doc/ref/os.process/ ) should do what you want. Construct a pipe with guile's POSIX 'pipe' procedure and pass the write port to the run-concurrently+ procedure for file descriptor 2 (stderr) and then read from the read port. Close the write port on the reader side before reading so that when the sub-process ends, the read will return with an eof-object. This works exactly as you would expect from its POSIX equivalents and has the advantage that you can read from the pipe as the sub-process is proceeding rather than just collect at the end. Having said that it would be nice to have this in more formalized form in guile itself, say something along the lines of ocaml's Unix.create_process function. Cjros
Subprocess API.
Hi, I recently used "open-pipe*" to launch a process but was unable to read from stderr. This subject was already discussed on this ml here : https://lists.gnu.org/archive/html/guile-user/2015-04/msg3.html Racket seems to have procedures to provide stdout/stdin/stderr ports for a given subprocess[1]. Mark, you said this subject was on your TODO list, is there anything available or would it be possible to develop a racket like API ? Thanks, Mathieu [1]: https://docs.racket-lang.org/reference/subprocess.html