Martin v. Löwis [EMAIL PROTECTED] writes on Tue, 24 May 2005 23:58:03 +0200:
... 10.000 failing opens -- a cause for significant IO during startup ? ...
So I would agree that IO makes a significant part of startup, but
I doubt it is directory reading (unless perhaps you have an
absent NFS
Scott David Daniels [EMAIL PROTECTED] writes on Wed, 25 May 2005 07:10:00
-0700:
...
I'll bet this means that the 'zope.zip', 'python24.zip' would drop
you to about 12500 - 1 = 2500 failing opens. That should be
an easy test: sys.path.insert(0, 'zope.zip') or whatever.
If that works
Dieter Maurer wrote:
Steve Holden [EMAIL PROTECTED] writes on Sun, 22 May 2005 16:19:10 -0400:
...
Indeed I have written PEP 302-based code to import from a relational
database, but I still don't believe there's any satisfactory way to
have [such a hooked import mechanism] be a first-class
Martin v. Löwis wrote:
Scott David Daniels wrote:
Is the interpreter unable to call C functions (stat for example)
to determine whether an object exists before it puts it on path.
What do you mean, unable to? It just doesn't.
In fact, the interpreter doesn't necessarily know when it is
Dieter Maurer wrote:
Martin v. Löwis [EMAIL PROTECTED] writes on Sun, 22 May 2005 21:24:41
+0200:
...
The application was Zope importing about 2.500 modules
from 2 zip files zope.zip and python24.zip.
This resulted in about 12.500 opens -- about 4 times more
than would be expected -- about
Robin Becker wrote:
if the importers are tested statically how does a filesystem path ever
manage
to get back into the loop if it was ever found missing? In other words if
things (eg python24.zip) are found not importable/usable in one pass how do
they get reinstated?
I think (but see
Steve Holden [EMAIL PROTECTED] writes on Sun, 22 May 2005 16:19:10 -0400:
...
Indeed I have written PEP 302-based code to import from a relational
database, but I still don't believe there's any satisfactory way to
have [such a hooked import mechanism] be a first-class component of an
.
...
The application was Zope importing about 2.500 modules
from 2 zip files zope.zip and python24.zip.
This resulted in about 12.500 opens -- about 4 times more
than would be expected -- about 10.000 of them failing opens.
I see. Out of curiosity: how much startup time was saved
when sys.path
Dieter Maurer wrote:
The comparison between warm start (few disc io) and cold start
(much disc io) tells you that the import process is highly
io dominated (for cold starts).
Correct. However, I would expect that the contents of existing
directories is cached, and it might be that the absence
Martin v. Löwis wrote:
Dieter Maurer wrote:
Really?
Is the interpreter unable to call C functions (stat for example)
to determine whether an object exists before it puts it on path.
What do you mean, unable to? It just doesn't.
In fact, the interpreter doesn't necessarily know when it is
Martin v. Löwis wrote:
Now I remember what makes this stuff really difficult: PEP 302
introduces path hooks (sys.path_hooks), allowing imports from
other sources than files. So the items on sys.path don't have
to be directory or file names at all, and importing from them
may still
Robin Becker wrote:
ie if we have N importers and F leading failure syspath entries before
the correct one is found do we get order N*F failed stats/opens etc etc?
No. Each path hook is supposed to provide a decision as to whether this
is a useful item on sys.path only once; the importer
? In other words if
things (eg python24.zip) are found not importable/usable in one pass how do
they get reinstated?
--
Robin Becker
--
http://mail.python.org/mailman/listinfo/python-list
Robin Becker wrote:
Dieter Maurer wrote:
[...]
I think this was my intention, but also I think I have some concern over
having two possible locations for the standard library. It seems non pythonic
and liable to cause confusion if some package should manage to install
python24.zip while I
(at the time the interpreter starts).
There is. When the interpreter starts, it doesn't know what object
do or do not exist. So it must put python24.zip on the path
just in case.
Really?
Is the interpreter unable to call C functions (stat for example)
to determine whether an object exists before
Steve Holden [EMAIL PROTECTED] writes on Sun, 22 May 2005 09:14:43 -0400:
...
There are some aspects of Python's initialization that are IMHO a bit
too filesystem-dependent. I mentioned one in
http://sourceforge.net/tracker/index.php?func=detailaid=1116520group_id=5470atid=105470
zope.zip and python24.zip.
This resulted in about 12.500 opens -- about 4 times more
than would be expected -- about 10.000 of them failing opens.
I see. Out of curiosity: how much startup time was saved
when sys.path was explicitly stripped to only contain these
two zip files?
I would expect
Dieter Maurer wrote:
Steve Holden [EMAIL PROTECTED] writes on Sun, 22 May 2005 09:14:43 -0400:
...
There are some aspects of Python's initialization that are IMHO a bit
too filesystem-dependent. I mentioned one in
. People can package everything they want in python24.zip (including
site.py). This can only work if python24.zip is already on the path
(and I believe it will always be sought in the directory where
python24.dll lives).
The question was:
should python start up with **non-existent** objects
Dieter Maurer wrote:
.
The question was:
should python start up with **non-existent** objects on the path.
I think there is no reason why path needs to contain an object
which does not exist (at the time the interpreter starts).
In your use case, python24.zip does exist
object
do or do not exist. So it must put python24.zip on the path
just in case.
In your use case, python24.zip does exist and therefore may
be on the path. When python24.zip does not exist, it does
not contain anything and especially not site.py.
Yes, but the interpreter cannot know
Robin Becker wrote:
Firstly should python start up with non-existent entries on the path?
Yes, this is by design.
Secondly is this entry be the default for some other kind of python
installation?
Yes. People can package everything they want in python24.zip (including
site.py). This can only
22 matches
Mail list logo
