Roshrini commented on issue #10948: get stuck with subprocess in multithread URL: https://github.com/apache/incubator-mxnet/issues/10948#issuecomment-413707798 It looks like the issue here is that Python 2.7 always uses os.fork() in to implement multiprocessing.Process.start(). In Python3, there is a way to use other possible start methods which are 'fork', 'spawn' and 'forkserver'. You can set context as follows: `multiprocessing.set_start_method('spawn')` to avoid the issues over fork-safety. Ran following code snippet multiple times in Python3 successfully. ``` import multiprocessing as multiprocess import subprocess import mxnet as mx def handle_task(): ctx = mx.cpu() for _ in range(500): print('g') mx.nd.ones((2, 3), ctx) print('i') subprocess.check_output(["ls"]) print('d') if __name__ == '__main__': multiprocess.set_start_method('spawn', force=True) child_processes = [multiprocess.Process(target=handle_task, args=()) for _ in range(4)] for i in range(4): child_processes[i].start() for i in range(4): child_processes[i].join() ``` @HorsonLiu Can you try and confirm if this fixes your issue?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
