marcoabreu commented on issue #11616: Flaky test test_gluon.test_export URL: https://github.com/apache/incubator-mxnet/issues/11616#issuecomment-403579484 On production systems while running a service, we can't necessarily assume that every single request is handled in a sandbox. On Ubuntu we actually run in a separate container for every single run, but this is not the case. The windows use case seems to be pretty close to reality. We don't want to interact in between executors, but that's a side effect because we don't make unique paths. Just load the model zoo in two separate processes and this problem will pop up. I'd recommend to store the files in a specific non unique directory and check if they exist (and match the hash) before downloading the archive from the internet. If it doesn't exist, download the zip file to a temporary dir. Then put a readwritelock (shared read, unique write) onto the non unique target path and extract the archive file into it. Right now, the code assumes that we have exclusive access - especially the code which tries to delete the archive file which might be used from a different process because it's also non unique (on mobile, otherwise I'd post the code).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
