[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, Jan 26, 2021 at 10:56:57AM +0100, Victor Stinner wrote: > On Tue, Jan 26, 2021 at 12:44 AM Steven D'Aprano wrote: [...] > > Your first instinct that it is too long is correct. Just call it > > "stdlib" or "stdlib_names". The fact that it is a frozen set of module > > names will be obvious from just looking at it, and there is no need for > > the name to explain everything about it. We have: > > The sys module already has a sys.modules attribute, and so > sys.module_names sounds like "give me the name of all imported > modules": sys.module.keys(). Then its a good thing I didn't propose calling it "module_names" :-) -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/K7FP7PSFULJYL2GPZDSOGKDPKZZ5GYKP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, Jan 26, 2021 at 11:19:13PM +0100, Victor Stinner wrote: > On Tue, Jan 26, 2021 at 10:04 PM Steve Dower wrote: > > > > On 1/26/2021 8:32 PM, Steve Holden wrote: > > > If the length of the name is any kind of issue, since the stdlib > > > only contains modules (and packages), why not just sys.stdlib_names? > > > > And since the modules can vary between platforms and builds, why > > wouldn't this be sysconfig.stdlib_names rather than sys.stdlib_names? > > The list is the same on all platforms on purpose ;-) Example: > > >>> 'winsound' in sys.stdlib_module_names > True Right. This is (I think) Steve's point: the list is inaccurate, because the existence of 'winsound' in the stdlib_module_names doesn't mean that the module 'winsound' exists. -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BHKOH3N2FGMTUVXT4YCJMPCOI7SWOTEU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, Jan 26, 2021 at 12:08:03PM -0800, Brett Cannon wrote: > On Tue, Jan 26, 2021 at 1:26 AM Antoine Pitrou wrote: [...] > > Disagreed. This is niche enough that it warrants a long but explicit > > name, rather than some ambiguous shortcut. > > > > I agree w/ Antoine that a more descriptive name for such a niche (but > useful!) attribute makes sense. This descriptive name is *literally incorrect*. By design, it doesn't list modules. It only lists sub-packages and not sub-modules, to keep the number of entries more managable. (Personally, I don't think an extra hundred or two names makes that much difference. Its going to be a big list one way or the other.) So by the current rules, many stdlib modules are not included and the name is inaccurate. If you're not going to list all the dotted modules of a package, why distinguish sub-modules from sub-packages? It is confusing and ackward to have only some dotted modules listed based on the **implementation**. (We need a good term for "things you can import that create a module *object* regardless of whether they are a *module file* or a *package*. I'm calling them a dotted module for lack of a better name.) By the current rules, many stdlib modules are not listed, and you can't see why unless you know their implementation: * urllib - listed * urllib.parse - not listed * collections - listed * collections.abc - not listed * email - listed * email.parser - not listed * email.mime - listed # Surprise! So we have this weird situation where an implementation detail of the dotted module (whether it is a file `package/module.py` or `package/module/__init__.py`) determines whether it shows up or not. And because the file system structure of a module is not part of its API, that implementation detail could change without warning. I think that either of: 1. list *all* package dotted modules regardless of whether they are implemented as a sub-module or sub-package; 2. list *no* package dotted modules, only the top-level package; would be better than this inconsistent hybrid of only listing some dotted modules. (Excluding the test modules is fine.) -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GNOAP4TVTHIUKE2GUGZWV6HNVE37KU4Q/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Mon, Jan 25, 2021 at 10:23 PM Random832 wrote: > On Mon, Jan 25, 2021, at 18:44, Chris Jerdonek wrote: > > But to issue a warning when a standard module is being overridden like > > I was suggesting, wouldn’t you also need to know whether the name of > > the module being imported is a standard name, which is what > > says.module_names provides? > > I don't think the warning would be only useful for stdlib modules... has > any thought been given to warning when a module being imported from the > current directory / script directory is the same as an installed package? > Related to this, I wonder if another application of sys.stdlib_module_names could be for installers: When installing a new package, a warning could be issued if the package is attempting to install a package with a name already in sys.stdlib_module_names. I don't know off-hand what happens if one were to try to do that today.. --Chris > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XTLEMKOZ4REXKJY2OI5RNJFBAAJABGD7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, Jan 26, 2021 at 10:04 PM Steve Dower wrote: > > On 1/26/2021 8:32 PM, Steve Holden wrote: > > If the length of the name is any kind of issue, since the stdlib > > only contains modules (and packages), why not just sys.stdlib_names? > > And since the modules can vary between platforms and builds, why > wouldn't this be sysconfig.stdlib_names rather than sys.stdlib_names? The list is the same on all platforms on purpose ;-) Example: >>> 'winsound' in sys.stdlib_module_names True >>> 'ossaudiodev' in sys.stdlib_module_names True For example, grouping stdlib imports using sys.stdlib_module_names gives the same output on any platform, even if there were missing dependencies when you built Python. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3EEZW2MYNFIE4ZE75RLOLYQC5ZUKSN2D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On 1/26/2021 8:32 PM, Steve Holden wrote: If the length of the name is any kind of issue, since the stdlib only contains modules (and packages), why not just sys.stdlib_names? And since the modules can vary between platforms and builds, why wouldn't this be sysconfig.stdlib_names rather than sys.stdlib_names? "Modules that were built into the stdlib" sounds more like sysconfig, and having an accurate list seems better than one that specifies (e.g.) distutils, ensurepip, resource or termios when those are absent. Cheers, Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7JBPSATSJMONLAGEU5PKTJHZ72MFRXBK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
If the length of the name is any kind of issue, since the stdlib only contains modules (and packages), why not just sys.stdlib_names? On Mon, Jan 25, 2021 at 5:18 PM Victor Stinner wrote: > Hi Bernat, > > "stdlib_module_names" was my first idea but it looks too long, so I > chose "module_names". But someone on Twitter and now you asked me why > not "stdlib_module_names", so I wrote a PR to rename module_names to > sys.stdlib_module_names: > https://github.com/python/cpython/pull/24332 > > At least "stdlib_module_names" better summarizes its definition: "A > frozenset of strings containing the names of standard library > modules". > > Victor > > > On Mon, Jan 25, 2021 at 5:39 PM Bernat Gabor > wrote: > > > > Hello, > > > > In general, I love the idea and implementation. I'm not in love with the > name though, it makes it sound like it contains all module names > imported/available. We have sys.module already containing all module > imported. So without a deeper knowledge sys.modules_names is very close to > sys.module.keys() or all available modules. Can we name it instead > sys.stdlib_modules_names to clarify that this is standard library only > subset and not all available modules for the interpreter? > > > > Thanks, > > > > On Mon, Jan 25, 2021 at 4:33 PM Victor Stinner > wrote: > >> > >> Hi Ivan, > >> > >> On Mon, Jan 25, 2021 at 4:53 PM Ivan Pozdeev via Python-Dev > >> wrote: > >> > Just _names_? There's a recurring error case when a 3rd-party module > overrides a standard one if it happens to have the same name. If you > >> > filter such a module out, you're shooting yourself in the foot... > >> > >> Overriding stdlib modules has been discussed in the issue. > >> > >> For example, it was proposed to add an attribute to all stdlib modules > >> (__stdlib__=True or __author__ = 'PSF'), and then check if the > >> attribute exists or not. The problem is that importing a module to > >> check for its attribute cause side effect or fail, and so cannot be > >> used for some use cases. For example, it would be a surprising to open > >> a web browser window when running isort on a Python code containing > >> "import antigravity". Another problem is that third party can also add > >> the attribute to pretend that their code is part of the stdlib. > >> > >> In a previous version of my PR, I added a note about sys.path and > >> overriding stdlib modules, but I have been asked to remove it. Feel > >> free to propose a PR to add such note if you consider that it's > >> related to sys.module_names. > >> > >> Please read the discussion at https://bugs.python.org/issue42955 and > >> https://github.com/python/cpython/pull/24238 > >> > >> Victor > >> ___ > >> Python-Dev mailing list -- python-dev@python.org > >> To unsubscribe send an email to python-dev-le...@python.org > >> https://mail.python.org/mailman3/lists/python-dev.python.org/ > >> Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/7HMWTGBECAVLINLO3MAEN74YVDHOMZKM/ > >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > Night gathers, and now my watch begins. It shall not end until my death. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/WJMYK2JKZPTXMID7WRMP4KMJ656WEMI5/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2JXT2M2DU2ZLPLPJF5LX3UVPRGWXKVPW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, Jan 26, 2021 at 1:26 AM Antoine Pitrou wrote: > On Tue, 26 Jan 2021 10:36:10 +1100 > Steven D'Aprano wrote: > > On Mon, Jan 25, 2021 at 06:17:09PM +0100, Victor Stinner wrote: > > > Hi Bernat, > > > > > > "stdlib_module_names" was my first idea but it looks too long, so I > > > chose "module_names". But someone on Twitter and now you asked me why > > > not "stdlib_module_names", so I wrote a PR to rename module_names to > > > sys.stdlib_module_names: > > > https://github.com/python/cpython/pull/24332 > > > > > > At least "stdlib_module_names" better summarizes its definition: "A > > > frozenset of strings containing the names of standard library > > > modules". > > > > Your first instinct that it is too long is correct. > > Disagreed. This is niche enough that it warrants a long but explicit > name, rather than some ambiguous shortcut. > I agree w/ Antoine that a more descriptive name for such a niche (but useful!) attribute makes sense. -Brett > > > Just call it > > "stdlib" or "stdlib_names". > > If you call it "stdlib", then you should make it a namedtuple that will > expose various information, such as "sys.stdlib.module_names". > > > Regards > > Antoine. > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/CJYWXVBIMDHRJCT4HZMOLJ7XMSVNZF6I/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PI6IOOBUJ4DLW67VIJQILBXYXSI27X37/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, Jan 26, 2021 at 12:44 AM Steven D'Aprano wrote: > > On Mon, Jan 25, 2021 at 06:17:09PM +0100, Victor Stinner wrote: > > Hi Bernat, > > > > "stdlib_module_names" was my first idea but it looks too long, so I > > chose "module_names". But someone on Twitter and now you asked me why > > not "stdlib_module_names", so I wrote a PR to rename module_names to > > sys.stdlib_module_names: > > https://github.com/python/cpython/pull/24332 > > > > At least "stdlib_module_names" better summarizes its definition: "A > > frozenset of strings containing the names of standard library > > modules". > > Your first instinct that it is too long is correct. Just call it > "stdlib" or "stdlib_names". The fact that it is a frozen set of module > names will be obvious from just looking at it, and there is no need for > the name to explain everything about it. We have: The sys module already has a sys.modules attribute, and so sys.module_names sounds like "give me the name of all imported modules": sys.module.keys(). It's confusing. Just after I announced the creation of the attribute, at least 3 people told me that they were confused by the name. Also, my PR was approved quickly 3 times which confirms that the rename was a good idea ;-) In general, I agree that short names are great ;-) For example, I like short obj.name() rather than obj.getname() when it doesn't make sense to set the name. Naming is a hard problem :-D Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DKRUV2HDWIN4XWDNXT55BBHNQOUZTOU6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Mon, 25 Jan 2021 23:05:07 +0100 Victor Stinner wrote: > > This is a different use case which requires a different solution. > sys.module_names solve some specific use cases (that I listed in my > first email). > > In Python 3.9, you can already check if a module __file__ is in the > sysconfig.get_paths()['stdlib'] directory. You don't need to modify > Python for that. Is this reliable? What if the stdlib is zipped or frozen in some way? > If you also would like to check if an *extension* module comes from > the stdlib, you need to get the "lib-dynload" directory. So you're saying the need is already fulfilled, even though it only has a cryptic (*) and partial solution? (*) who would think about `sysconfig.get_paths()['stdlib']` on their own? Regards Antoine. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2XBORMKGYQX2EYWXEEURSYRWJW3F3VKF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules
On Tue, 26 Jan 2021 10:36:10 +1100 Steven D'Aprano wrote: > On Mon, Jan 25, 2021 at 06:17:09PM +0100, Victor Stinner wrote: > > Hi Bernat, > > > > "stdlib_module_names" was my first idea but it looks too long, so I > > chose "module_names". But someone on Twitter and now you asked me why > > not "stdlib_module_names", so I wrote a PR to rename module_names to > > sys.stdlib_module_names: > > https://github.com/python/cpython/pull/24332 > > > > At least "stdlib_module_names" better summarizes its definition: "A > > frozenset of strings containing the names of standard library > > modules". > > Your first instinct that it is too long is correct. Disagreed. This is niche enough that it warrants a long but explicit name, rather than some ambiguous shortcut. > Just call it > "stdlib" or "stdlib_names". If you call it "stdlib", then you should make it a namedtuple that will expose various information, such as "sys.stdlib.module_names". Regards Antoine. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CJYWXVBIMDHRJCT4HZMOLJ7XMSVNZF6I/ Code of Conduct: http://python.org/psf/codeofconduct/
