Re: Python child process in while True loop blocks parent

2021-12-09 Thread Barry Scott
> On 8 Dec 2021, at 17:11, Jen Kris wrote: > > I started this post on November 29, and there have been helpful comments > since then from Barry Scott, Cameron Simpson, Peter Holzer and Chris > Angelico. Thanks to all of you. > > I've found a solution that works for my purpose, and I said

Re: Python child process in while True loop blocks parent

2021-12-08 Thread Peter J. Holzer
On 2021-12-08 18:11:48 +0100, Jen Kris via Python-list wrote: > To recap, I'm using a pair of named pipes for IPC between C and > Python.  Python runs as a child process after fork-execv.  The Python > program continues to run concurrently in a while True loop, and > responds to requests from C at

Re: Python child process in while True loop blocks parent

2021-12-08 Thread Cameron Simpson
On 08Dec2021 18:11, Jen Kris wrote: >Python must terminate its write strings with \n, or read will block in >C waiting for something that never comes. There are two aspects to this: - if upstream is rding "lines of text" then you need a newline to terminate the lines - you (probably) should

Re: Python child process in while True loop blocks parent

2021-12-08 Thread Jen Kris via Python-list
I started this post on November 29, and there have been helpful comments since then from Barry Scott, Cameron Simpson, Peter Holzer and Chris Angelico.  Thanks to all of you.  I've found a solution that works for my purpose, and I said earlier that I would post the solution I found. If anyone

Re: Python child process in while True loop blocks parent

2021-12-06 Thread Barry Scott
> On 6 Dec 2021, at 21:05, Jen Kris wrote: > > Here is what I don't understand from what you said. "The child process is > created with a single thread—the one that called fork()." To me that implies > that the thread that called fork() is the same thread as the child process. > I guess

Re: Python child process in while True loop blocks parent

2021-12-06 Thread Jen Kris via Python-list
Here is what I don't understand from what you said.  "The child process is created with a single thread—the one that called fork()."  To me that implies that the thread that called fork() is the same thread as the child process.  I guess you're talking about the distinction between logical

Re: Python child process in while True loop blocks parent

2021-12-06 Thread Barry Scott
> On 6 Dec 2021, at 17:09, Jen Kris via Python-list > wrote: > > I can't find any support for your comment that "Fork creates a new > process and therefore also a new thread." From the Linux man pages > https://www.man7.org/linux/man-pages/man2/fork.2.html, "The child process is > created

Re: Python child process in while True loop blocks parent

2021-12-06 Thread Chris Angelico
On Tue, Dec 7, 2021 at 4:10 AM Jen Kris via Python-list wrote: > > I can't find any support for your comment that "Fork creates a new > process and therefore also a new thread." From the Linux man pages > https://www.man7.org/linux/man-pages/man2/fork.2.html, "The child process is > created

Re: Python child process in while True loop blocks parent

2021-12-06 Thread Jen Kris via Python-list
I can't find any support for your comment that "Fork creates a new process and therefore also a new thread."  From the Linux man pages https://www.man7.org/linux/man-pages/man2/fork.2.html, "The child process is created with a single thread—the one that called fork()."  I have a one-core

Re: Python child process in while True loop blocks parent

2021-12-06 Thread Barry
> On 5 Dec 2021, at 23:51, Jen Kris wrote: > >  > By embedding, I think you may be referring to embedding Python in a C program > with the Python C API. That's not what I'm doing here -- I'm not using the > Python C API. The C program creates two threads (using pthreads), one for >

Re: Python child process in while True loop blocks parent

2021-12-05 Thread Peter J. Holzer
On 2021-12-06 00:51:13 +0100, Jen Kris via Python-list wrote: > The C program creates two threads (using pthreads), one for itself and > one for the child process.  On creation, the second pthread is pointed > to a C program that calls fork-execv to run the Python program.  That > way Python runs

Re: Python child process in while True loop blocks parent

2021-12-05 Thread Jen Kris via Python-list
By embedding, I think you may be referring to embedding Python in a C program with the Python C API.   That's not what I'm doing here -- I'm not using the Python C API.  The C program creates two threads (using pthreads), one for itself and one for the child process.  On creation, the second

Re: Python child process in while True loop blocks parent

2021-12-05 Thread Barry
> On 5 Dec 2021, at 17:54, Jen Kris wrote: > >  > Thanks for your comments. > > I put the Python program on its own pthread, and call a small C program to > fork-execv to call the Python program as a child process. What do you mean by putting python in it’s own pthread? Are you

Re: Python child process in while True loop blocks parent

2021-12-05 Thread Jen Kris via Python-list
Thanks for your comments.  I put the Python program on its own pthread, and call a small C program to fork-execv to call the Python program as a child process.  I revised the Python program to be a multiprocessing loop using the Python multiprocessing module.  That bypasses the GIL and allows

Re: Python child process in while True loop blocks parent

2021-12-04 Thread Barry Scott
> On 1 Dec 2021, at 16:01, Jen Kris wrote: > > Thanks for your comment re blocking. > > I removed pipes from the Python and C programs to see if it blocks without > them, and it does. > It looks now like the problem is not pipes. Ok. > I use fork() and execv() in C to run Python in a

Re: Python child process in while True loop blocks parent

2021-12-01 Thread Jen Kris via Python-list
Thanks for your comment re blocking.  I removed pipes from the Python and C programs to see if it blocks without them, and it does.  It looks now like the problem is not pipes.  I use fork() and execv() in C to run Python in a child process, but the Python process blocks because fork() does

Re: Python child process in while True loop blocks parent

2021-11-30 Thread Barry Scott
> On 29 Nov 2021, at 22:31, Jen Kris wrote: > > Thanks to you and Cameron for your replies. The C side has an epoll_ctl set, > but no event loop to handle it yet. I'm putting that in now with a pipe > write in Python-- as Cameron pointed out that is the likely source of > blocking on C.

Re: Python child process in while True loop blocks parent

2021-11-29 Thread Jen Kris via Python-list
Thanks to you and Cameron for your replies.  The C side has an epoll_ctl set, but no event loop to handle it yet.  I'm putting that in now with a pipe write in Python-- as Cameron pointed out that is the likely source of blocking on C.  The pipes are opened as rdwr in Python because that's

Re: Python child process in while True loop blocks parent

2021-11-29 Thread Barry
> On 29 Nov 2021, at 20:36, Jen Kris via Python-list > wrote: > > I have a C program that forks to create a child process and uses execv to > call a Python program. The Python program communicates with the parent > process (in C) through a FIFO pipe monitored with epoll(). > > The

Re: Python child process in while True loop blocks parent

2021-11-29 Thread Barry
> On 29 Nov 2021, at 20:36, Jen Kris via Python-list > wrote: > > I have a C program that forks to create a child process and uses execv to > call a Python program. The Python program communicates with the parent > process (in C) through a FIFO pipe monitored with epoll(). > > The

Re: Python child process in while True loop blocks parent

2021-11-29 Thread Cameron Simpson
On 29Nov2021 21:34, Jen Kris wrote: >I have a C program that forks to create a child process and uses execv to call >a Python program.  The Python program communicates with the parent process (in >C) through a FIFO pipe monitored with epoll().  > >The Python child process is in a while True