Re: [Python-Dev] PEP 451 update
On Thu, Oct 31, 2013 at 6:10 PM, Nick Coghlan wrote: > > On 1 Nov 2013 01:37, "PJ Eby" wrote: > . ;-) > > > > I also suspect, that if properly spelled out, those use cases are > > going to boil down to: > > > > 1. Throwing errors if you have an existing module object you can't > > load into, and > > 2. Passing in a previous spec object, if available > > > > In other words, loaders should not really have any responsibility for > > or concept of "reloading" -- they always load into a module object > > (that they may or may not have created), and they may get given a spec > > from a previous load. They should deal only in "module reuse" and > > "spec reuse". While a typical reload() might involve both reuses, > > there are cases where one sort of reuse could occur independently, and > > not all loaders care about both (or even either) condition. > > > > At any rate, it means a loader author doesn't have to figure out how > > to handle "reloading", all they have to figure out is whether they can > > load into a particular module object, and whether they can do > > something useful with a spec that was previously used to load a module > > with the same name -- a spec that may or may not refer to a similar > > previous loader. These are rather more well-defined endeavors than > > trying to determine in the abstract whether one "supports reload". > > ;-) > > It may not have been clear from the email exchange, but that's basically > where we ended up :) > > The change Eric is currently going to make to the PEP is to add an > optional "previous" parameter to the various find_spec APIs. > > At the time when find_spec runs, sys.modules hasn't been touched yet, so > the old module state is still available when reloading. Passing the spec in > lets the loader decide whether or not it can actually load that module > given the information about the previous load operation. > Do you mean loader or finder? You say "find_spec" which is for finders, but then you say "loader". > However, perhaps it makes more sense to pass the entire existing module, > rather than just the spec? I'd like to minimise the need for new-style > loader authors to retrieve things directly from the sys module, and you're > right that "can I reload into this particular module?" is a slightly > different question from "can I reload a module previously loaded using this > particular spec?". > > A spec-based API could still be used to answer the first question by > looking up sys.modules, but so could the name based API. Passing in the > module reference, on the other hand, should let loaders answer both > questions without needing to touch the sys module. > > I also now think this pass-the-module approach when finding the spec > approach could also clean up some of the messiness that is __main__ > initialisation, where we repeatedly load things into the same module object. > See you mention "finding" but before now everything has been "loader". > Let's say we be completely explicit and call the new parameter something > like "load_target". If we do that then I would make the *same* change to > runpy.run_path and runpy.run_module: let you pass in an existing module > object under that name to say "hey, run in this namespace, rather than a > fresh one". (Those APIs currently only support pre-populating a *fresh* > namespace, rather than allowing you to directly specify an existing one) > > Most loaders won't care, but those that do will have all the info they > need to throw an exception saying "I could provide a spec for this, but > it's not compatible with that load target". > Exactly what APIs -- new or changed -- are you proposing? Method signatures only please to avoid ambiguity. =) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-10-25 - 2013-11-01) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open4217 (+18) closed 27029 (+64) total 31246 (+82) Open issues with patches: 1955 Issues opened (49) == #19394: distutils.core.Extension: empty strings in library_dirs and in http://bugs.python.org/issue19394 opened by robotron #19397: test_pydoc fails with -S http://bugs.python.org/issue19397 opened by pitrou #19398: test_trace fails with -S http://bugs.python.org/issue19398 opened by pitrou #19402: AbstractBasicAuthHandler http://bugs.python.org/issue19402 opened by Perry.Lorier #19403: Make contextlib.redirect_stdout reentrant http://bugs.python.org/issue19403 opened by ncoghlan #19404: Simplify per-instance control of help() output http://bugs.python.org/issue19404 opened by ncoghlan #19406: PEP 453: add the ensurepip module http://bugs.python.org/issue19406 opened by ncoghlan #19407: PEP 453: update the "Installing Python Modules" documentation http://bugs.python.org/issue19407 opened by ncoghlan #19411: binascii.hexlify docs say it returns a string (it returns byte http://bugs.python.org/issue19411 opened by Devin Jeanpierre #19412: Add docs for test.support.requires_docstrings decorator http://bugs.python.org/issue19412 opened by ncoghlan #19414: iter(ordered_dict) yields keys not in dict in some circumstanc http://bugs.python.org/issue19414 opened by Nikratio #19415: test_gdb fails when using --without-doc-strings on Fedora 19 http://bugs.python.org/issue19415 opened by ncoghlan #19417: bdb test coverage http://bugs.python.org/issue19417 opened by Colin.Williams #19419: Use abc.ABC in the collections ABC http://bugs.python.org/issue19419 opened by rhettinger #19420: Leak in _hashopenssl.c http://bugs.python.org/issue19420 opened by skrah #19422: Neither DTLS nor error for SSLSocket.sendto() of UDP socket http://bugs.python.org/issue19422 opened by christian.heimes #19428: marshal: error cases are not documented http://bugs.python.org/issue19428 opened by haypo #19429: OSError constructor does not handle errors correctly http://bugs.python.org/issue19429 opened by haypo #19431: Document PyFrame_FastToLocals() and PyFrame_FastToLocalsWithEr http://bugs.python.org/issue19431 opened by haypo #19433: Define PY_UINT64_T on Windows 32bit http://bugs.python.org/issue19433 opened by christian.heimes #19434: Wrong documentation of MIMENonMultipart class http://bugs.python.org/issue19434 opened by vajrasky #19437: More failures found by pyfailmalloc http://bugs.python.org/issue19437 opened by haypo #19438: Where is NoneType in Python 3? http://bugs.python.org/issue19438 opened by mpb #19439: Build _testembed on Windows http://bugs.python.org/issue19439 opened by zach.ware #19440: Clean up test_capi http://bugs.python.org/issue19440 opened by zach.ware #19441: itertools.tee improve documentation http://bugs.python.org/issue19441 opened by Alan.Cristhian #19444: mmap.mmap() allocates a file descriptor that isn't CLOEXEC http://bugs.python.org/issue19444 opened by rfm #19447: py_compile.compile raises if a file has bad encoding http://bugs.python.org/issue19447 opened by bkabrda #19448: SSL: add OID / NID lookup http://bugs.python.org/issue19448 opened by christian.heimes #19449: csv.DictWriter can't handle extra non-string fields http://bugs.python.org/issue19449 opened by tomas_grahn #19450: Bug in sqlite in Windows binaries http://bugs.python.org/issue19450 opened by schlamar #19451: urlparse accepts invalid hostnames http://bugs.python.org/issue19451 opened by daenney #19453: pydoc.py doesn't detect IronPython, help(foo) can hang http://bugs.python.org/issue19453 opened by owlmonkey #19454: devguide: Document what a "platform support" is http://bugs.python.org/issue19454 opened by techtonik #19456: ntpath doesn't join paths correctly when a drive is present http://bugs.python.org/issue19456 opened by gvanrossum #19459: Python does not support the GEORGIAN-PS charset http://bugs.python.org/issue19459 opened by Caolán.McNamara #19460: Add test for MIMENonMultipart http://bugs.python.org/issue19460 opened by vajrasky #19461: RawConfigParser modifies empty strings unconditionally http://bugs.python.org/issue19461 opened by Nacsa.Kristóf #19462: Add remove_argument() method to argparse.ArgumentParser http://bugs.python.org/issue19462 opened by ustinov #19463: assertGdbRepr depends on hash randomization / endianess http://bugs.python.org/issue19463 opened by christian.heimes #19464: Remove warnings from Windows buildbot "clean" script http://bugs.python.org/issue19464 opened by zach.ware #19465: selectors: provide a helper to choose a selector using constra http://bugs.python.org/issue19465 opened by haypo #19466: Clear state of threads earlier in Python shutdown http://bugs.python
Re: [Python-Dev] PEP 451 update
On Thu, Oct 31, 2013 at 4:10 PM, Nick Coghlan wrote: > However, perhaps it makes more sense to pass the entire existing module, > rather than just the spec? Sounds good to me. :) -eric ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Contributing to PSF
Hi, My name is Anup. I am pursuing my Enginering degree at Amrita in India. I want to start contributing to PSF by fixing bugs and if possible by helping to develop the python software. So, I request some one to help me for that. Thanks in advance, Regards, A. Anup, 2nd Year CSE, Amrita University, http://anup07.wordpress.com/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Nov 1, 2013 at 7:35 AM, Brett Cannon wrote: > On Thu, Oct 31, 2013 at 6:10 PM, Nick Coghlan wrote: >> At the time when find_spec runs, sys.modules hasn't been touched yet, so >> the old module state is still available when reloading. Passing the spec in >> lets the loader decide whether or not it can actually load that module given >> the information about the previous load operation. > > Do you mean loader or finder? You say "find_spec" which is for finders, but > then you say "loader". > >> >> However, perhaps it makes more sense to pass the entire existing module, >> rather than just the spec? I'd like to minimise the need for new-style >> loader authors to retrieve things directly from the sys module, and you're >> right that "can I reload into this particular module?" is a slightly >> different question from "can I reload a module previously loaded using this >> particular spec?". >> >> A spec-based API could still be used to answer the first question by >> looking up sys.modules, but so could the name based API. Passing in the >> module reference, on the other hand, should let loaders answer both >> questions without needing to touch the sys module. >> >> I also now think this pass-the-module approach when finding the spec >> approach could also clean up some of the messiness that is __main__ >> initialisation, where we repeatedly load things into the same module object. > > > See you mention "finding" but before now everything has been "loader". > >> >> Let's say we be completely explicit and call the new parameter something >> like "load_target". If we do that then I would make the *same* change to >> runpy.run_path and runpy.run_module: let you pass in an existing module >> object under that name to say "hey, run in this namespace, rather than a >> fresh one". (Those APIs currently only support pre-populating a *fresh* >> namespace, rather than allowing you to directly specify an existing one) >> >> Most loaders won't care, but those that do will have all the info they >> need to throw an exception saying "I could provide a spec for this, but it's >> not compatible with that load target". > > Exactly what APIs -- new or changed -- are you proposing? Method signatures > only please to avoid ambiguity. =) I'm about to update the PEP but I'll clarify here. The signature of finder.find_spec() will grow an extra "existing" parameter (default to None) that is an existing module. During reload the module from sys.modules will be passed in. Having this be part of find_spec() is consistent with the job of the finder to identify a loader that should be used to load the module (reload or not). As to the inconsistency in what Nick said, it still fits in with the way I see it. I'm sure he'll correct me if he had something else in mind. :-) finder.find_spec() will be making a decision on what loader (if any) to return. As part of that, when it finds a loader it can further decide on whether or not the loader supports loading into an existing module (if one was provided). As part of that it may or may not consult with the loader. Either way it stands as proxy for the loader in the latter decision. In lieu of an explicit loader.supports_reload() method, a loader will rely on its associated finder to be a proxy for reload-related decisions, likely communicated via spec.loader_state. Perhaps Nick meant something else, but my understanding is that he was referring to this -eric ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Contributing to PSF
On Fri, 01 Nov 2013 22:50:28 +0530, Anup Kumar wrote: > My name is Anup. I am pursuing my Enginering degree at Amrita in India. I > want to start contributing to PSF by fixing bugs and if possible by helping > to develop the python software. So, I request some one to help me for that. Welcome, Anup. Good first steps are signing up for the core-mentorship mailing list, and reading the devguide (http://docs.python.org/devguide). --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Nov 1, 2013 at 1:52 PM, Eric Snow wrote: > On Fri, Nov 1, 2013 at 7:35 AM, Brett Cannon wrote: > > On Thu, Oct 31, 2013 at 6:10 PM, Nick Coghlan > wrote: > >> At the time when find_spec runs, sys.modules hasn't been touched yet, so > >> the old module state is still available when reloading. Passing the > spec in > >> lets the loader decide whether or not it can actually load that module > given > >> the information about the previous load operation. > > > > Do you mean loader or finder? You say "find_spec" which is for finders, > but > > then you say "loader". > > > >> > >> However, perhaps it makes more sense to pass the entire existing module, > >> rather than just the spec? I'd like to minimise the need for new-style > >> loader authors to retrieve things directly from the sys module, and > you're > >> right that "can I reload into this particular module?" is a slightly > >> different question from "can I reload a module previously loaded using > this > >> particular spec?". > >> > >> A spec-based API could still be used to answer the first question by > >> looking up sys.modules, but so could the name based API. Passing in the > >> module reference, on the other hand, should let loaders answer both > >> questions without needing to touch the sys module. > >> > >> I also now think this pass-the-module approach when finding the spec > >> approach could also clean up some of the messiness that is __main__ > >> initialisation, where we repeatedly load things into the same module > object. > > > > > > See you mention "finding" but before now everything has been "loader". > > > >> > >> Let's say we be completely explicit and call the new parameter something > >> like "load_target". If we do that then I would make the *same* change to > >> runpy.run_path and runpy.run_module: let you pass in an existing module > >> object under that name to say "hey, run in this namespace, rather than a > >> fresh one". (Those APIs currently only support pre-populating a *fresh* > >> namespace, rather than allowing you to directly specify an existing one) > >> > >> Most loaders won't care, but those that do will have all the info they > >> need to throw an exception saying "I could provide a spec for this, but > it's > >> not compatible with that load target". > > > > Exactly what APIs -- new or changed -- are you proposing? Method > signatures > > only please to avoid ambiguity. =) > > I'm about to update the PEP but I'll clarify here. The signature of > finder.find_spec() will grow an extra "existing" parameter (default to > None) that is an existing module. During reload the module from > sys.modules will be passed in. Having this be part of find_spec() is > consistent with the job of the finder to identify a loader that should > be used to load the module (reload or not). > > As to the inconsistency in what Nick said, it still fits in with the > way I see it. I'm sure he'll correct me if he had something else in > mind. :-) finder.find_spec() will be making a decision on what loader > (if any) to return. As part of that, when it finds a loader it can > further decide on whether or not the loader supports loading into an > existing module (if one was provided). As part of that it may or may > not consult with the loader. Either way it stands as proxy for the > loader in the latter decision. > > In lieu of an explicit loader.supports_reload() method, a loader will > rely on its associated finder to be a proxy for reload-related > decisions, likely communicated via spec.loader_state. Perhaps Nick > meant something else, but my understanding is that he was referring to > this > Thanks for the clarification. It all SGTM. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Nov 1, 2013 at 11:59 AM, Brett Cannon wrote: > Thanks for the clarification. It all SGTM. I've updated the PEP and the reference implementation. If I've missed anything please let me know. There is only one thing I thought of that I'd like to get an opinion on: In the Finders section, the PEP specifies returning None (or using a different loader) when the found loader does not support loading into an existing module (e.g during reload). An alternative to returning None would be for the finder to raise ImportError with a message like "the loader does not support reloading the module". This may actually be a better approach since "could not find a loader" and "the found loader won't work" are different situations that a single return value (None) may not sufficiently represent. Other than that, I'm not aware of any blockers for the PEP. Once I settle that last question, I'll ask for pronouncement. (Anyone may feel free to pronounce before then. ) -eric ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
