On Wed, Feb 23, 2011 at 7:09 AM, Volker Braun <vbraun.n...@gmail.com> wrote: > Its true that there are lots of failed attempts to get files, which is > normal for any system where you have a multiple directories which can > contain any given file: > > [vbraun@volker-desktop ~]$ echo quit | strace -f sage |& grep ENOENT | wc > 24322 298143 3532797 > But, really, that just means that once the directory content is read into > the filesystem cache and then then queries are answered from memory. > Stat'ing ~25k files is pretty much instantaneous:
This is true on your computer; unfortunately, it is absolutely not true on all filesystems. For example, networked filesystems have to deal with the possibility that another process elsewhere on the system changed the contents of the directory listing. Robert Bradshaw: > A psychological improvement such as showing the > prompt (with working tab completion) quickly would be a good thing as > well, but not really solve the problem, especially for a cold start. Indeed, if it takes 5-10 minutes for Sage to start (and yes it does on some slow filesystems), then this wouldn't help at all. At lunch yesterday Robert Bradshaw made the interesting suggestion to read the docs for importlib (http://docs.python.org/dev/library/importlib.html) and write a customized import hook, so that every time during Sage startup that a module is imported, the import is done from a single big in-memory zip file instead of done using the filesystem. If this can be made to work, it would be a huge win for slow filesystems. The basic problem is that some filesystems are fast but have huge *latency*. -- William > [vbraun@volker-desktop sage]$ time find | wc > 24238 24238 1510223 > real 0m0.059s > user 0m0.050s > sys 0m0.035s > I'm pretty sure that we won't be able to beat the FS cache with a python > cache. > This might be different if the cache is cold and the system uses mechanical > harddrives, but then it'll still take a long time to read the ~2k files in > the Sage library: > [vbraun@volker-desktop sage]$ echo quit | strace -f sage |& grep '.py"' | > grep -v ENOENT | wc > 2097 12884 261674 > > -- > To post to this group, send an email to sage-devel@googlegroups.com > To unsubscribe from this group, send an email to > sage-devel+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/sage-devel > URL: http://www.sagemath.org > -- William Stein Professor of Mathematics University of Washington http://wstein.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
