Hey, ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, 22 August 2020 12:00, Tristan Van Berkom <[email protected]> wrote:
[...] > > > > I think the most correct solution would be to get rid of our > > multiprocessed scheduler, which is something I have tried to make > > work for more than two months now. I'd advocate for postponing that a > > bit, seeing how well a non-multiprocessed solution can go, and > > normally the problem should go away? > > My branch with my current work is at [0] > > The`spawn` method being default on OSX seems to be a weird API break > for python 3.8, and I think we should definitely immediately force the > multiprocessing method to use `fork` unconditionally; this should > really be a one liner fix. > > To be clear: I fully support Ben's work on moving BuildStream to a > threaded process model, I just think that we should not block on that > for now for any reason. > > "I'd advocate for postponing that a bit" > > I'm not sure I understand which postponing we are talking about > (postponing landing the threading work until it's ready to land sounds > reasonable ?). I think we definitely should not postpone a one line > patch which would make multiprocessing work on OSX with python 3.8 > while we wait for the better fix. > I was indeed proposing to postpone having OSX Runners for that. Since there is a reason why the Python developers decided to not have fork as a default (it brings a load of problems). However, as we are using fork + asyncio, we are already in an unchartered, unsupported territory. So I think you are right, we should force `fork` for now, and move to threading after! > Cheers, > -Tristan Cheers, Ben
