[issue2953] _zip_directory_cache untested and undocumented
Changes by Terry J. Reedy tjre...@udel.edu: -- versions: +Python 3.3 -Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Changes by Brett Cannon br...@python.org: -- nosy: -brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Éric Araujo added the comment: Eric, as you know importlib, can you comment about this? If zipimport now works with the invalidate_caches system, then we do have a documented way of clearing the cache, and I would recommend closing this bug (the existing attribute should stay for compat, but we should not document or test it more IMO). -- assignee: tarek - nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Eric Snow added the comment: I'll take a look but can't say when. My wife's is having a baby in the next couple days so I'm a bit swamped. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Éric Araujo added the comment: No problem (and congratulations :) -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file13625/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file13638/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Éric Araujo mer...@netwok.org added the comment: Could the recently-added functools.lru_cache help here? -- components: -Distutils nosy: +eric.araujo versions: +Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Brett Cannon br...@python.org added the comment: 2009/4/5 Tarek Ziadé rep...@bugs.python.org@psf.upfronthosting.co.za Tarek Ziadé ziade.ta...@gmail.com added the comment: Does distutils use _zip_directory_cache in any way? Not at all. But pkgutil does, Ah, OK. and uses to get a dirlist, Couldn't one just open the zip file with zipfile to get the directory list? Is there any real requirement that the cache must be used? so I guess a new API should be created in zipimport to handle this case as well. (I think it's some setuptools code originallyt, that landed in pkgutil) Probably. -- Added file: http://bugs.python.org/file13638/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___brbrdiv class=gmail_quote2009/4/5 span dir=ltrlt;quot;Tarek Ziadé lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;quot;@a href=http://psf.upfronthosting.co.za;psf.upfronthosting.co.za/agt;/spanbr blockquote class=gmail_quote style=border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;br Tarek Ziadé lt;a href=mailto:ziade.ta...@gmail.com;ziade.ta...@gmail.com/agt; added the comment:br div class=imbr gt; Does distutils use _zip_directory_cache in any way?br br /divNot at all. But pkgutil does,/blockquotedivbrAh, OK.br /divblockquote class=gmail_quote style=border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex; and uses to get a dirlist,/blockquote divbrCouldn#39;t one just open the zip file with zipfile to get the directory list? Is there any real requirement that the cache must be used?br /divblockquote class=gmail_quote style=border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex; br so I guess a new API should be created in zipimport to handle this casebr as well. (I think it#39;s some setuptools code originallyt, that landed inbr pkgutil)/blockquotedivbrProbably. br/div/divbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Tarek Ziadé ziade.ta...@gmail.com added the comment: _zip_directory_cache is part of the zipimport module (C module) you can find in Modules/zipimport.c. The tests are located here: ./Lib/test/test_zipimport.py _zip_directory_cache is a dict used by zipimporter_init to speed up the lookup. Notice that if it is successfully compiled, zipimport is used by pkgutil. So Maciek, did you have a bug with this ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Maciek Fijalkowski fi...@genesilico.pl added the comment: The bug with this is that setuptools uses it (and even relies on what is there and in what order), while it has no single tests and no documentation. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Tarek Ziadé ziade.ta...@gmail.com added the comment: while it has no single tests and no documentation. what it refers to ? zipimport, distutils or setuptools ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Maciek Fijalkowski fi...@genesilico.pl added the comment: it == _zip_directory_cache -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Tarek Ziadé ziade.ta...@gmail.com added the comment: I am not sure what you would like to see added. in zipimport.c: - _zip_directory_cache: a dict, mapping archive paths to zip directory info dicts, as used in zipimporter._files. while there are no direct test, Lib/test/test_zipimport.py covers it becauseit is used when you instanciate a zipimport class. would you like to see a section explaining how it works at http://docs.python.org/library/zipimport.html ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Tarek Ziadé ziade.ta...@gmail.com added the comment: maybe what's missing is a public API for the clear() method since setuptools uses it by calling it directly -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Maciek Fijalkowski fi...@genesilico.pl added the comment: setuptools at least deletes stuff from _zip_directory_cache (via _uncache function in easy_install.py). it also iterates over it and checks for existance of elements. What about documenting details about _zip_directory_cache: * does it need to be a normal python dictionary or anything? * what methods it needs to support? * what api zipimporter exposes? Right now the situation is: if I want my library not to be cached any more, I need to grab object called _zip_directory_cache, and remove things from it. That's a very non-obvious API for caching. Is it documented in some PEP? The way I stomped on it - just passing the tests of zipimporter module actually does not guarantee that you've implemented it correctly. You also need to expose at least _zip_directory_cache otherwise setuptools won't be too happy. Stdlib is not only about testing cpython, it's also about what you define as python these days. I propose to have a clean api on zipimporter how to invalidate it's caches. Maybe a general mechanism for that, I don't know. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Brett Cannon br...@python.org added the comment: Does distutils use _zip_directory_cache in any way? If not then setuptools is in the wrong here for using a private attribute and it is on them to make it work if the attribute does not exist, period. Otherwise someone will need to come up with a very simple API or set or rules to follow for clearing the cache and get setuptools to switch to it (IMO it should be handled by simply deleting all entries for a module in sys.modules, removing the path hook from sys.path_hooks, and then clear out sys.path_importer_cache for all paths associated with the module). Be aware, though, this is all leading down the road of special-casing zip imports when you may very well run into the same issues if people start to develop other importers -- e.g. tarball, dbm, or sqlite3 importers -- and this will all start up again. -- Added file: http://bugs.python.org/file13625/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___Does distutils use _zip_directory_cache in any way? If not then setuptools is in the wrong here for using a private attribute and it is on them to make it work if the attribute does not exist, period.brbrOtherwise someone will need to come up with a very simple API or set or rules to follow for clearing the cache and get setuptools to switch to it (IMO it should be handled by simply deleting all entries for a module in sys.modules, removing the path hook from sys.path_hooks, and then clear out sys.path_importer_cache for all paths associated with the module). Be aware, though, this is all leading down the road of special-casing zip imports when you may very well run into the same issues if people start to develop other importers -- e.g. tarball, dbm, or sqlite3 importers -- and this will all start up again.br ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Tarek Ziadé ziade.ta...@gmail.com added the comment: Does distutils use _zip_directory_cache in any way? Not at all. But pkgutil does, and uses to get a dirlist, so I guess a new API should be created in zipimport to handle this case as well. (I think it's some setuptools code originallyt, that landed in pkgutil) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- priority: - normal versions: +Python 2.7, Python 3.1 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Brett Cannon br...@python.org added the comment: Actually importlib does not test this; that is an old thing from an old sandbox version of importlib that I no longer use. I am assigning to Tarek as the better solution is for distutils to stop using this private attribute. -- assignee: brett.cannon - tarek stage: - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Georg Brandl ge...@python.org added the comment: That's fortunate, because I can just reassign this to him :) -- assignee: georg.brandl - brett.cannon nosy: +brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Georg Brandl ge...@python.org added the comment: I can't find a reference to _zip_directory_cache in distutils. Do you mean setuptools? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Maciek Fijalkowski fi...@genesilico.pl added the comment: Seems I meant setuptools indeed. Note that brett's importlib contains some tests for _zip_directory_cache. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
Changes by Daniel Diniz aja...@gmail.com: -- nosy: +tarek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2953] _zip_directory_cache untested and undocumented
New submission from Maciek Fijalkowski [EMAIL PROTECTED]: _zip_directory_cache has no single test nor piece of documentation. The only place where it's mentioned is a call to method .clear() on it in tests. Besides, it can be whatever. Despite _ prefix, which I would consider to be private, distutils crash obscurely if it's not there or does not contain sequences as keys. -- assignee: georg.brandl components: Distutils, Documentation, Extension Modules messages: 67267 nosy: fijal, georg.brandl severity: normal status: open title: _zip_directory_cache untested and undocumented type: behavior versions: Python 2.6 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2953 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
