On Monday, March 13, 2023 at 11:32:23 PM UTC-7, Clint Olsen wrote:
> We have an application that involves submitting hundreds to thousands of jobs
> to a shared computing resource, and we're using asyncio to do so because it
> is far less overhead than threading or multiprocessing for the bookkeeping
> required to keep track of all these jobs. It makes extensive use of
> asyncio.create_subprocess_exec(). This was developed mostly in Python 3.9.7.
Good news! I did end up finding the source of the problem. I kept looking for
the use of global data as a potential source of bugs and didn't really find
anything there. I did fix some potential signal handler problems. However I had
this particular piece of code that I needed to follow my processes.
watcher = asyncio.FastChildWatcher()
watcher.attach_loop(asyncio.get_event_loop())
asyncio.set_child_watcher(watcher)
Since async reuses the same event loop on successive calls, doing this again is
bad. I just needed to ensure I only set this once.
Thanks,
-Clint
--
https://mail.python.org/mailman/listinfo/python-list
