Re: [Python-Dev] Python Language Summit at PyCon: Agenda
Stefan Krah, 02.03.2013 21:01: > Stefan Behnel wrote: I'm not so happy with the argument clinic, but that's certainly also because I'm biased. I've written the argument unpacking code for Cython some years ago, so it's not surprising that I'm quite happy with that and fail to see the need for a totally new DSL *and* a totally new implementation, especially with its mapping to the slowish ParseTuple*() C-API functions. I've also not seen a good argument why the existing Py3 function signatures can't do what the proposed DSL tries to achieve. They'd at least make it clear that the intention is to make things more Python-like, and would at the same time provide the documentation. >>> >>> That's why Stefan Krah is writing a competing PEP - a number of us >>> already agree with you, and think the case needs to be made for >>> choosing something completely different like Argument Clinic >> >> I'll happily provide my feedback to that approach. It might also have a >> positive impact on the usage of Py3 argument annotations, which I think >> merit some more visibility and "useful use cases". > > > BTW, I think so far no one has stepped forward to implement the custom > argument handlers. I've looked at Cython output and, as you say, most of > it is there already. > > Is it possible to write a minimal version of the code generator that just > produces the argument handling code? It should be possible, although it does have a lot of dependencies on Cython's type system, so a part of that would have to be extracted as well or at least emulated. Conversion functions are based on it, for example. However, I think it would actually be easiest to just let Cython generate the module interface completely. I.e. you'd remove all code that currently deals with Python function signatures from the C module, only leaving the bare C API, and then generate a Cython interface module like this: cdef extern from *: object original_c_xyzfunc(object x, int y, double z) def xyzfunc(x, int y=0, *, double z=1.0): "docstring goes here" return original_c_xyzfunc(x,y,z) Finally, #include the generated C file at the end of the original module. There'd be a bit of a hassle with the module init function, I guess. Maybe renaming it in the Cython C code (even just a #define) and calling it from the original module would work. Or do it the other way round and add a hook function somewhere that does the manually written parts of the module setup. Sounds simple enough in both cases, although I'm sure there's lots of little details. Extension types and their methods are certainly part of those details ... Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Planning on removing cache invalidation for file finders
On Sat, 2 Mar 2013 18:38:15 -0500 Brett Cannon wrote: > > > > You can deprecate the heuristic if you want (and can figure out how), > > but a definite -1 on removing it without at least the usual > > deprecation period for backwards incompatible changes. > > > > That part is easy: ImportWarning still exists so simply continuing to check > the directory and noticing when a difference exists that affects subsequent > imports and then raising the warning will handle that. Won't that raise spurious ImportWarnings for people who don't actually care about that? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Mar 2, 2013, at 11:41 AM, Nick Coghlan wrote: > On Fri, Mar 1, 2013 at 9:39 AM, Doug Hellmann wrote: >> >> On Feb 27, 2013, at 11:51 AM, Michael Foord wrote: >> >>> Hello all, >>> >>> PyCon, and the Python Language Summit, is nearly upon us. We have a good >>> number of people confirmed to attend. If you are intending to come to the >>> language summit but haven't let me know please do so. >>> >>> The agenda of topics for discussion so far includes the following: >>> >>> * A report on pypy status - Maciej and Armin >>> * Jython and IronPython status reports - Dino / Frank >>> * Packaging (Doug Hellmann and Monty Taylor at least) >> >> Since the time I suggested we add packaging to the agenda, Nick has set up a >> separate summit meeting for Friday evening. I don't know if it makes sense >> to leave this on the agenda for Wednesday or not. >> >> Nick, what do you think? > > I think it's definitely worth my taking some time to explain my goals > for the Friday night session, and some broader things in terms of > where I'd like to see packaging going, but a lot of the key packaging > people aren't involved in Python language development *per se*, and > hence won't be at the language summit. OK, a summary seems like a good idea. > > There's also one controversial point that *does* need to be raised at > the summit: I would like to make distutils-sig the true authority for > packaging standards, so we can stop cross-posting PEPs intended to > apply to packaging in *current* versions of Python to python-dev. The > split discussions suck, and most of the people that need to be > convinced in order for packaging standards to be supported in current > versions of Python aren't on python-dev, since it's a tooling issue > rather than a language design issue. Standard lib support is necessary > in the long run to provide a good "batteries included" experience, but > it's *not* the way to create the batteries in the first place. Until > these standards have been endorsed by the authors of *existing* > packaging tools, proposing them for stdlib addition is premature, but > has been perceived as necessary in the past due to the confused power > structure. +1 -- anything to reduce the confusion about where to get involved :) > > This means that those core developers that want a say in the future > direction of packaging and distribution of Python software would need > to be actively involved in the ongoing discussions on distutils-sig, > rather than relying on being given an explicit invitation to weigh in > at the last minute through a thread (or threads) on python-dev. The > requirement that BDFL-delegates for packaging and distribution related > PEPs also be experienced core developers will remain, however, as > "suitable for future stdlib inclusion" is an important overarching > requirement for packaging and distribution standards. Such delegates > will just be expected to participate actively in distutils-sig *as > well as* python-dev. > > Proposals for *actual* standard library updates (to bring it into line > with updated packaging standards) would still be subject to python-dev > discussion and authority (and would *not* have their Discussions-To > header set). Such discussions aren't particularly relevant to most of > the packaging tool developers, since the standard library version > isn't updated frequently enough to be useful to them, and also isn't > available on older Python releases, so python-dev is a more > appropriate venue from both perspectives. > > At the moment, python-dev, catalog-sig and distutils-sig create an > awkward trinity where decision making authority and the appropriate > venues for discussion are grossly unclear. I consider this to be one > of the key reasons that working on packaging issues has quite a high > incidence of developer burnout - it's hard to figure out who needs to > be convinced of what, so it's easy for the frustration levels to reach > the "this just isn't worth the hassle" stage (especially when trying > to bring python-dev members up to speed on discussions that may have > taken months on distutils-sig, and when many of the details are > awkward compromises forced by the need to support *existing* tools and > development processes on older versions of Python). Under my proposal, > the breakdown would be slightly clearer: > > distutils-sig: overall authority for packaging and distribution > related standards, *including* the interfaces between index servers > (such as PyPI) and automated tools. If a PEP has "Discussions-To" set > to distutils-sig, announcements of new PEPs, new versions of those > PEPs, *and* their acceptance or rejection should be announced there, > and *not* on python-dev. The "Resolution" header will thus point to a > distutils-sig post rather than a python-dev one. distutils-sig will > focus on solutions that work for *current* versions of Python, while > keeping in mind the need for future stdlib support. > > python-d
Re: [Python-Dev] Planning on removing cache invalidation for file finders
On Sat, Mar 2, 2013 at 8:16 PM, Antoine Pitrou wrote: > On Sat, 2 Mar 2013 11:16:28 -0500 > Brett Cannon wrote: > > > In addition, it may be appropriate for importlib to offer a > > > "write_module" method that accepts (module name, target path, > > > contents). This would: > > > > > > 1. Allow in-process caches to be invalidated implicitly and > > > selectively when new modules are created > > > > I don't think that's necessary. If people don't want to blindly clear all > > caches for a file they can write the file, search the keys in > > sys.path_importer_cache for the longest prefix for the newly created > file, > > and then call the invalidate_cache() method on that explicit finder. > > That's too complicated for non-import experts IMHO. > Which is why they can just call importlib.import_module(). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Planning on removing cache invalidation for file finders
On Sun, Mar 3, 2013 at 6:29 AM, Antoine Pitrou wrote: > On Sat, 2 Mar 2013 18:38:15 -0500 > Brett Cannon wrote: > > > > > > You can deprecate the heuristic if you want (and can figure out how), > > > but a definite -1 on removing it without at least the usual > > > deprecation period for backwards incompatible changes. > > > > > > > That part is easy: ImportWarning still exists so simply continuing to > check > > the directory and noticing when a difference exists that affects > subsequent > > imports and then raising the warning will handle that. > > Won't that raise spurious ImportWarnings for people who don't actually > care about that? > It shouldn't. If the implementation I have in my head works (set of original files, another set of what mtime says is there to know what would not have been found w/o the cache invalidation), then it will only come up when someone would break in Python 3.5 if the cache invalidation is removed. Plus warnings are off by default as it is. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Planning on removing cache invalidation for file finders
On Sun, Mar 3, 2013 at 12:08 PM, Brett Cannon wrote: > > > > On Sat, Mar 2, 2013 at 8:16 PM, Antoine Pitrou wrote: > >> On Sat, 2 Mar 2013 11:16:28 -0500 >> Brett Cannon wrote: >> > > In addition, it may be appropriate for importlib to offer a >> > > "write_module" method that accepts (module name, target path, >> > > contents). This would: >> > > >> > > 1. Allow in-process caches to be invalidated implicitly and >> > > selectively when new modules are created >> > >> > I don't think that's necessary. If people don't want to blindly clear >> all >> > caches for a file they can write the file, search the keys in >> > sys.path_importer_cache for the longest prefix for the newly created >> file, >> > and then call the invalidate_cache() method on that explicit finder. >> >> That's too complicated for non-import experts IMHO. >> > > Which is why they can just call importlib.import_module(). > That obviously should have said importlib.invalidate_caches(). =) But how about this as a compromise over introducing write_module(): invalidate_caches() can take a path for something to specifically invalidate. The path can then be passed to the invalidate_caches() on sys.meta_path. In the case of PathFinder it would take that path, try to find the directory in sys.path_importer_cache, and then invalidate the most specific finder for that path (if there is one that has any directory prefix match). Lots of little details to specify (e.g. absolute path forced anywhere in case a relative path is passed in by sys.path is all absolute paths? How do we know something is a file if it has not been written yet?), but this would prevent importlib from subsuming file writing specifically for source files and minimize performance overhead of invalidating all caches for a single file. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On 28 February 2013 05:51, Michael Foord wrote: > Hello all, > > PyCon, and the Python Language Summit, is nearly upon us. We have a good > number of people confirmed to attend. If you are intending to come to the > language summit but haven't let me know please do so. > > The agenda of topics for discussion so far includes the following: > > * A report on pypy status - Maciej and Armin > * Jython and IronPython status reports - Dino / Frank > * Packaging (Doug Hellmann and Monty Taylor at least) > * Cleaning up interpreter initialisation (both in hopes of finding areas > to rationalise and hence speed things up, as well as making things > more embedding friendly). Nick Coghlan > * Adding new async capabilities to the standard library (Guido) > * cffi and the standard library - Maciej > * flufl.enum and the standard library - Barry Warsaw > * The argument clinic - Larry Hastings > > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. I'd like to talk about overhauling - not tweaking, overhauling - the standard library testing facilities. -Rob -- Robert Collins Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins wrote: > I'd like to talk about overhauling - not tweaking, overhauling - the > standard library testing facilities. That seems like too big a topic and too vague a description to discuss usefully. Perhaps you have a specific proposal? Or at least just a use case that's poorly covered? TBH, your choice of words is ambiguous -- are you interested in overhauling the facilities for testing *of* the standard library (i.e. the 'test' package), or the testing facilities *provided by* the standard library (i.e. the unittest module)? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On 4 March 2013 18:54, Guido van Rossum wrote: > On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins > wrote: >> I'd like to talk about overhauling - not tweaking, overhauling - the >> standard library testing facilities. > > That seems like too big a topic and too vague a description to discuss > usefully. Perhaps you have a specific proposal? Or at least just a use > case that's poorly covered? I have both - I have a draft implementation for a new test result API (and forwards and backwards compat code etc), and use cases that drive it. I started a thread here - http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html , with blog posts https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/ https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/ https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/ https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/ They are focused on subunit, but much of subunit's friction has been due to issues encountered from the stdlibrary TestResult API - in particular three things: - the single-active-test model that the current API (or at least implementation) has. - the expectation that all test outcomes will originate from the same interpreter (or something with a live traceback object) - the inability to supply details about errors other than the exception All of which start to bite rather deep when working on massively parallel test environments. It is of course possible for subunit and related tools to run their own implementation, but it seems ideal to me to have a common API which regular unittest, nose, py.test and others can all agree on and use : better reuse for pretty printers, GUI displays and the like depend on some common API. > TBH, your choice of words is ambiguous -- are you interested in > overhauling the facilities for testing *of* the standard library (i.e. > the 'test' package), or the testing facilities *provided by* the > standard library (i.e. the unittest module)? Sorry! Testing facilities provided by the standard library. They should naturally facilitate testing of the standard library too. -Rob > -- > --Guido van Rossum (python.org/~guido) -- Robert Collins Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Mon, Mar 4, 2013 at 4:26 PM, Robert Collins wrote: > On 4 March 2013 18:54, Guido van Rossum wrote: >> On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins >> wrote: >>> I'd like to talk about overhauling - not tweaking, overhauling - the >>> standard library testing facilities. >> >> That seems like too big a topic and too vague a description to discuss >> usefully. Perhaps you have a specific proposal? Or at least just a use >> case that's poorly covered? > > I have both - I have a draft implementation for a new test result API > (and forwards and backwards compat code etc), and use cases that drive > it. I started a thread here - > http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html > , with blog posts > https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/ > https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/ > https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/ > https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/ > > They are focused on subunit, but much of subunit's friction has been > due to issues encountered from the stdlibrary TestResult API - in > particular three things: > - the single-active-test model that the current API (or at least > implementation) has. > - the expectation that all test outcomes will originate from the same > interpreter (or something with a live traceback object) > - the inability to supply details about errors other than the exception > > All of which start to bite rather deep when working on massively > parallel test environments. Your feedback on http://bugs.python.org/issue16997 would be greatly appreciated. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On 4 March 2013 19:40, Nick Coghlan wrote: > Your feedback on http://bugs.python.org/issue16997 would be greatly > appreciated. Done directly to Antoine on IRC the other day in a conversation with him and Michael about the compatability impact of subtests. Happy to do a full code review if that would be useful. -Rob -- Robert Collins Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Mon, Mar 4, 2013 at 4:46 PM, Robert Collins wrote: > On 4 March 2013 19:40, Nick Coghlan wrote: > >> Your feedback on http://bugs.python.org/issue16997 would be greatly >> appreciated. > > Done directly to Antoine on IRC the other day in a conversation with > him and Michael about the compatability impact of subtests. Happy to > do a full code review if that would be useful. The extra set of eyes couldn't hurt (and if you can spot a better way to tie @expectedfailure into the rest of the test running machinery, that would be great. Making sure that decorator doesn't break is the ugliest part of the whole patch) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
