leezu commented on a change in pull request #17372: [BUGFIX] fix model zoo
parallel download
URL: https://github.com/apache/incubator-mxnet/pull/17372#discussion_r370251465
##########
File path: python/mxnet/gluon/model_zoo/model_store.py
##########
@@ -103,16 +105,19 @@ def get_model_file(name,
root=os.path.join(base.data_dir(), 'models')):
util.makedirs(root)
- zip_file_path = os.path.join(root, file_name+'.zip')
repo_url = os.environ.get('MXNET_GLUON_REPO', apache_repo_url)
if repo_url[-1] != '/':
repo_url = repo_url + '/'
- download(_url_format.format(repo_url=repo_url, file_name=file_name),
- path=zip_file_path,
- overwrite=True)
- with zipfile.ZipFile(zip_file_path) as zf:
- zf.extractall(root)
- os.remove(zip_file_path)
+
+ with tempfile.NamedTemporaryFile(dir=root) as zip_file:
+ download(_url_format.format(repo_url=repo_url, file_name=file_name),
+ path=zip_file.name, overwrite=True, inplace=True)
Review comment:
From the doc:
> Whether the name can be used to open the file a second time, while the
named temporary file is still open, varies across platforms (it can be so used
on Unix; it cannot on Windows NT or later). If delete is true (the default),
the file is deleted as soon as it is closed.
Here you are taking the file name of the opened file, passing it to download
and therein opening it again. One workaround is to work with the open file
object itself when using `tempfile.NamedTemporaryFile`.
Reference https://docs.python.org/2/library/tempfile.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
