merrymercy edited a comment on issue #6650: URL: https://github.com/apache/incubator-tvm/issues/6650#issuecomment-706448409
The issue is not about RPC. It is about the argument passing in `multiprocessing.Pool.map`. Some TVMObjects cannot be correctly serialized, so we cannot pass them as arguments of `multiprocessing.Pool.map` The issue exists in both autotvm and ansor, because the "fork trick" to avoid serialization is used in both of them. There are three parts that are accelerated by multi-processing/multi-threading in autotvm/ansor: feature extraction, build and benchmark. The last one (benchmark) is the least important one because most people only use one machine for tuning. The benchmark is serial in most cases. The first two items (feature extraction and build) are more important because all people have mulit-core CPUs. In autotvm, we have this issue for all three parts, because all of them are paralleled by `multiprocessing.Pool.map`. In ansor, the feature extraction part is already moved to c++ multi-threading. So we only have this issue for build and benchmark. I think the issue can be solved by moving all parts to c++ multi-threading, which is more reliable and portable. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
