AndrewZhaoLuo commented on pull request #8363: URL: https://github.com/apache/tvm/pull/8363#issuecomment-871606570
The main lesson here is if we use python we must use multiprocessing to get parallelism. If we use multiprocessing we have to assume sometimes we cannot use a direct `fork()` to get a subprocess. There might be other bugs as if do not have a direct `fork()` arguments to the children process need to be serialized and reserialized. This has two implications: - Pointer equality assumptions are broken -- even if two objects shared a pointer to a resource before they might not after serialization and reserialization. This one is annoying. - Things passed to subprocesses must be pickleable which they might not be. The solution is to make things pickleable like I did here by removing the anonymous function closure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
