Reinout van Rees schreef op 15-04-15 om 16:26:
Setuptools seems to run all "egg_info.writers" entry points it can
find in the piece of code where the slowness occurs. So any of the
entry points could be the culprit. And perhaps even an entry point
outside of setuptools, due to the way entry points work.
I'll try to debug further.
Ok, the egg_info.writers entry points are innocent.
A couple of debug "print" statements later I found that
setuptools/commands/egg_info.py's manifest_maker class calls "findall()",
which is the monkeypatched version from setuptools/__init__.py.
This does an "os.walk" through the entire project directory to create a
full and complete list of files. Inside the os.walk() loop, it calls
os.path.isfile() on every item.
This takes a long time! 90%+ of the time is spend in parts/omelette/.
And most of that in parts/omelette/django's localization folders...
Of course it also goes through the var/ folder, which might store all
sorts of files.
Now... this is before it actually tries to include/exclude/graft stuff
per the MANIFEST.in instructions. It just goes through the entire project.
Why is it so slow? Probably because I'm running it on OSX inside a
vmware VM (running ubuntu). Probably the disk access via the
virtualization layer is slow.
I'm not completely satisfied yet, as sometimes it also is slow on the
server. (Also on VMs, but normally the performance hit isn't noticable).
So.... it almost seems as if there's no solution to this?
Or can someone give a hint on os.walk performance relative to VMs?
Reinout
--
Reinout van Rees http://reinout.vanrees.org/
[email protected] http://www.nelen-schuurmans.nl/
"Learning history by destroying artifacts is a time-honored atrocity"
_______________________________________________
Distutils-SIG maillist - [email protected]
https://mail.python.org/mailman/listinfo/distutils-sig
