On Sat, 2 Jan 2016, 20:42 Brett Cannon <br...@python.org> wrote: > I just wanted to quickly say that Guido's observation as to how a VFS is > overkill is right. Imagine implementing a loader using sqlite and you > quickly realize that doing a dull VFS is more > "dull" -> "full"
than necessary to implement what import needs to function. > > On Sat, 2 Jan 2016, 19:42 Guido van Rossum <gu...@python.org> wrote: > >> On Sat, Jan 2, 2016 at 3:26 PM, <mike.romb...@comcast.net> wrote: >> >>> >>> -- >>> >>>>> "Brett" == Brett Cannon <br...@python.org> writes: >>> >>> > I opened >>> > https://bugs.python.org/issue25711 to specifically try to >>> > fix this issue once and for all and along the way modernize >>> > zipimport by rewriting it from scratch to be more >>> > maintainable >>> >>> Every time I read about impementing a custom loader: >>> >>> https://docs.python.org/3/library/importlib.html >>> >>> I've wondered why python does not have some sort of virtual >>> filesystem layer to deal with locating modules/packages/support >>> files. Virtual file systems seem like a good way to store data on a >>> wide range of storage devices. >>> >> >> Yeah, but most devices already implement a *real* filesystem, so the only >> time the VFS would come in handy would be for zipfiles, where we already >> have a solution. >> >> >>> A VFSLoader object would interface with importlib and deal with: >>> >>> - implementing a finder and loader >>> >>> - Determine the actual type of file to load (.py, .pyc, .pyo, >>> __pycache__, etc). >>> >>> - Do all of it's work by calling virtual functions such as: >>> * listdir(path) >>> * read(path) >>> * stat(path) # for things like mtime, size, etc >>> * write(path, data) # not all VFS implement this >>> >> >> Emulating a decent filesystem API requires you to implement functionality >> that would never be used by an import loader (write() is an example -- many >> of the stat() fields are another example). So it would just be overkill. >> >> >>> Then things like a ziploader would just inherit from VFSLoader >>> implement the straight forward methods and everything should "just >>> work" :). I see no reason why every type of loader (real filesystem, >>> http, ssh, sql database, etc) would not do this as well. >> >> >> All those examples except "real filesystem" are of very limited practical >> value. >> >> >>> Leave all >>> the details such as implicit namespace packages, presence of >>> __init__.py[oc] files, .pth files, etc in one single >>> location and put the details on how to interact with the actual >>> storage device in leaf classes which don't know or care about the high >>> level details. They would not even know they are loading python >>> modules. It is just blobs of data to them. >>> >> >> Actually the import loader API is much more suitable and less work to >> implement than a VFS. >> >> >>> I may try my hand at creating a prototype of this for just the >>> zipimporter and see how it goes. >>> >> >> That would nevertheless be an interesting exercise -- I hope you do it >> and report back. >> >> >>> > At this point the best option might be, Mike, if you do a >>> > code review for https://bugs.python.org/issue17633, even if >>> > it is simply a LGTM. I will then personally make sure the >>> > approved patch gets checked in for Python 3.6 in case the >>> > rewrite of zipimport misses the release. >>> >>> Cool. I'll see what I can do. I was having a bit of trouble with >>> the register/login part of the bug tracker. Which is why I came >>> here. I'll battle with it one more time and see if I can get it to >>> log me in. >>> >> >> If you really can't manage to comment in the tracker (which sounds >> unlikely -- many people have succeeded :-) you can post your LGTM on the >> specific patch here. >> >> >>> The patch should be fairly simple. In a nutshell it just does a: >>> >>> path.replace('.', '/') + '/' in two locations. One where it checks >>> for the path being a directory entry in the zip file and the second to >>> return an implicit namespace path (instead of not found) if it is a >>> match. I'll check the patch on the tracker and see if it still works >>> with 3.5.1. If not I'll attach mine. >> >> >> Well, Brett would like to see your feedback on the specific patch. Does >> it work for you? >> >> -- >> --Guido van Rossum (python.org/~guido) >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/brett%40python.org >> >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
