Re: [Python-Dev] Restricted API versioning
On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes wrote: > Am 24.06.2012 01:11, schrieb Larry Hastings: >> On 06/23/2012 03:08 PM, "Martin v. Löwis" wrote: >>> On 23.06.2012 23:41, Antoine Pitrou wrote: Perhaps something more user-friendly than the hexversion? >>> Please propose something. I think the hexversion *is* user-friendly, >> >> +1 to the idea, and specifically to using hexversion here. > > +1 for the general idea and for using Py_LIMITED_API. I still like my > idea of a simple macro based on Include/patchlevel.h, for example: > > #define Py_API_VERSION(major, minor, micro) \ > (((major) << 24) | ((minor) << 16) | ((micro) << 8)) > > #if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0) > #endif +1 to all 3 of those from me (the general idea, using hexversion, and providing a convenience macro to skip having to spell out hexversion manually). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
As long as we are _before_ feature freeze, I would have thought it fine. Also, I'm sure we could allow ourselves some flexibility, after all, it is we that make these schedules, not the other way round. But no matter. Condition variable wakeup has been sluggish for many years, I'm sure our users can wait a few more. Meanwhile, the patch is there for those who want it. Cheers, K Frá: python-dev-bounces+kristjan=ccpgames@python.org [python-dev-bounces+kristjan=ccpgames@python.org] fyrir hönd Antoine Pitrou [solip...@pitrou.net] Sent: 23. júní 2012 12:37 To: python-dev@python.org Efni: Re: [Python-Dev] 3.3 release plans On Sat, 23 Jun 2012 12:32:37 + Kristján Valur Jónsson wrote: > Au contraire, it is actually a very major improvement, the result of pretty > extensive profiling, see > http://blog.ccpgames.com/kristjan/2012/05/25/optimizing-python-condition-variables-with-telemetry/ It might be. But to evaluate it, we have to digest a long technical blog post, then carefully review the patch (and perhaps even re-do some measurements). You understand that it's not reasonable to do so just one day before feature freeze. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
Am 24.06.2012 01:44, schrieb Chris Angelico: > This strikes me as in opposition to the Python-level policy of duck > typing. Would it be more appropriate to, instead of asking if it's > Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or > would that result in an unnecessary proliferation of flag macros? The version number is a sufficient rule. Flags aren't necessary as we can never remove or alter the signature of a API function. We can only add new features. Otherwise we'd break the API and binary interface (ABI) for C extensions. C compilers, linkers, dynamic library loaders and calling conventions are limited and don't support fancy stuff like OOP. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On 06/23/2012 04:44 PM, Chris Angelico wrote: On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes wrote: +1 for the general idea and for using Py_LIMITED_API. I still like my idea of a simple macro based on Include/patchlevel.h, for example: #define Py_API_VERSION(major, minor, micro) \ (((major)<< 24) | ((minor)<< 16) | ((micro)<< 8)) #if Py_LIMITED_API+0>= Py_API_VERSION(3, 3, 0) #endif This strikes me as in opposition to the Python-level policy of duck typing. Would it be more appropriate to, instead of asking if it's Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or would that result in an unnecessary proliferation of flag macros? python != c Or, if you prefer python is not c C lacks niceties like constructors, destructors, and default arguments. I think C APIs need to be much more precise than Python APIs; mix-n-match C APIs would be an invitation to heartburn and migranes. //arry/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes wrote: > +1 for the general idea and for using Py_LIMITED_API. I still like my > idea of a simple macro based on Include/patchlevel.h, for example: > > #define Py_API_VERSION(major, minor, micro) \ > (((major) << 24) | ((minor) << 16) | ((micro) << 8)) > > #if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0) > #endif This strikes me as in opposition to the Python-level policy of duck typing. Would it be more appropriate to, instead of asking if it's Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or would that result in an unnecessary proliferation of flag macros? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
Am 24.06.2012 01:11, schrieb Larry Hastings: > On 06/23/2012 03:08 PM, "Martin v. Löwis" wrote: >> On 23.06.2012 23:41, Antoine Pitrou wrote: >>> Perhaps something more user-friendly than the hexversion? >> Please propose something. I think the hexversion *is* user-friendly, > > +1 to the idea, and specifically to using hexversion here. +1 for the general idea and for using Py_LIMITED_API. I still like my idea of a simple macro based on Include/patchlevel.h, for example: #define Py_API_VERSION(major, minor, micro) \ (((major) << 24) | ((minor) << 16) | ((micro) << 8)) #if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0) #endif > (Though what will we do after Python 255.0?) Luckily it's gonna take another 1500 years, or so. Our progenies could rename Python to Circus ... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On 06/23/2012 03:08 PM, "Martin v. Löwis" wrote: On 23.06.2012 23:41, Antoine Pitrou wrote: Perhaps something more user-friendly than the hexversion? Please propose something. I think the hexversion *is* user-friendly, +1 to the idea, and specifically to using hexversion here. (Though what will we do after Python 255.0?) //arry/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On 23.06.2012 23:41, Antoine Pitrou wrote: > On Sat, 23 Jun 2012 23:31:07 +0200 > "Martin v. Löwis" wrote: >> I've been thinking about extensions to the stable ABI. On the one hand, >> introducing new API can cause extension modules not to run on older >> Python versions. On the other hand, the new API may well be stable in >> itself, i.e. remain available for all coming 3.x versions. >> >> As a compromise, I propose that such API can be added, but extension >> authors must explicitly opt into using it. To define their desired >> target Python versions, they need to set Py_LIMITED_API to the >> hexversion of the first Python release they want to support. > > Perhaps something more user-friendly than the hexversion? Please propose something. I think the hexversion *is* user-friendly, since it allows easy comparisons (Py_LIMITED_API+0 >= 0x0303). Users that run into missing symbols will, after inspection of the header file, easily know what to do. We could require a second macro, but users will already have to define Py_LIMITED_API, so not making them define a second macro is also more friendly. Plus, with the hexversion, we can add stuff to a bugfix release, such a annoying omissions (e.g. the omission of the _SizeT functions, which I missed since I didn't compile the headers with PY_SSIZE_T_CLEAN when generating the function list). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
Am 23.06.2012 23:41, schrieb Antoine Pitrou: > Perhaps something more user-friendly than the hexversion? IMHO 0x0303 for 3.0.0 is user-friendly enough. A macro like PY_VERSION(3, 0, 0) could be added, too. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On Sat, 23 Jun 2012 23:31:07 +0200 "Martin v. Löwis" wrote: > I've been thinking about extensions to the stable ABI. On the one hand, > introducing new API can cause extension modules not to run on older > Python versions. On the other hand, the new API may well be stable in > itself, i.e. remain available for all coming 3.x versions. > > As a compromise, I propose that such API can be added, but extension > authors must explicitly opt into using it. To define their desired > target Python versions, they need to set Py_LIMITED_API to the > hexversion of the first Python release they want to support. Perhaps something more user-friendly than the hexversion? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On Sat, Jun 23, 2012 at 2:31 PM, "Martin v. Löwis" wrote: > I've been thinking about extensions to the stable ABI. On the one hand, > introducing new API can cause extension modules not to run on older > Python versions. On the other hand, the new API may well be stable in > itself, i.e. remain available for all coming 3.x versions. > > As a compromise, I propose that such API can be added, but extension > authors must explicitly opt into using it. To define their desired > target Python versions, they need to set Py_LIMITED_API to the > hexversion of the first Python release they want to support. > > Objections? > +1 This sounds reasonable to me. Many other libraries have used this approach in the past. > The first use case of this are some glitches in the heap type API > that Robin Schreiber detected in his GSoC project. E.g. specifying > a heap type whose base also is a heap type was not really possible: > the type spec would have to contain a pointer to the base, but that > is not constant. In addition, if we use multiple interpreters, the > base type should be a different object depending on the current > interpreter - something that PyType_FromSpec couldn't support at all. > So there is a new API function PyType_FromSpecWithBases which covers > this case, and this API will only be available in 3.3+. > > Regards, > Martin > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On Sat, Jun 23, 2012 at 5:31 PM, "Martin v. Löwis" wrote: > I've been thinking about extensions to the stable ABI. On the one hand, > introducing new API can cause extension modules not to run on older > Python versions. On the other hand, the new API may well be stable in > itself, i.e. remain available for all coming 3.x versions. > > As a compromise, I propose that such API can be added, but extension > authors must explicitly opt into using it. To define their desired > target Python versions, they need to set Py_LIMITED_API to the > hexversion of the first Python release they want to support. > > Objections? > Nope, it sounds like a good idea to allow for the ABI to slowly grow. -Brett > > The first use case of this are some glitches in the heap type API > that Robin Schreiber detected in his GSoC project. E.g. specifying > a heap type whose base also is a heap type was not really possible: > the type spec would have to contain a pointer to the base, but that > is not constant. In addition, if we use multiple interpreters, the > base type should be a different object depending on the current > interpreter - something that PyType_FromSpec couldn't support at all. > So there is a new API function PyType_FromSpecWithBases which covers > this case, and this API will only be available in 3.3+. > > Regards, > Martin > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Restricted API versioning
I've been thinking about extensions to the stable ABI. On the one hand, introducing new API can cause extension modules not to run on older Python versions. On the other hand, the new API may well be stable in itself, i.e. remain available for all coming 3.x versions. As a compromise, I propose that such API can be added, but extension authors must explicitly opt into using it. To define their desired target Python versions, they need to set Py_LIMITED_API to the hexversion of the first Python release they want to support. Objections? The first use case of this are some glitches in the heap type API that Robin Schreiber detected in his GSoC project. E.g. specifying a heap type whose base also is a heap type was not really possible: the type spec would have to contain a pointer to the base, but that is not constant. In addition, if we use multiple interpreters, the base type should be a different object depending on the current interpreter - something that PyType_FromSpec couldn't support at all. So there is a new API function PyType_FromSpecWithBases which covers this case, and this API will only be available in 3.3+. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
Original-Nachricht > Datum: Sat, 23 Jun 2012 21:55:55 +0200 > Von: Christian Heimes > An: python-dev@python.org > Betreff: Re: [Python-Dev] 3.3 release plans > Am 23.06.2012 12:54, schrieb g.brandl-nos...@gmx.net: > > Hi all, > > > > now that the final PEP scheduled for 3.3 is final, we're entering > > the next round of the 3.3 cycle. > > > > I've decided to make Tuesday 26th the big release day. That means: > > > > - Saturday: last feature-level changes that should be done before beta, > > e.g. removal of packaging > > - Sunday: final feature freeze, bug fixing > > - Monday: focus on stability of the buildbots, even unstable ones > > - Tuesday: forking of the 3.3.0b1 release clone, tagging, start > > of binary building > > I'd like to get the C implementation of the timing safe compare_digest > into 3.3. http://bugs.python.org/issue15061 > > The patch went to several incarnations and I implemented input from > Antoine, Serhiy and others. The function finally ended up as private > function in the operator module because the _hashlib module isn't > available without openssl and a new module for a single function is > kinda overkill. Fine with me. You have time until tomorrow to push it. Georg -- NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone! Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
Am 23.06.2012 12:54, schrieb g.brandl-nos...@gmx.net: > Hi all, > > now that the final PEP scheduled for 3.3 is final, we're entering > the next round of the 3.3 cycle. > > I've decided to make Tuesday 26th the big release day. That means: > > - Saturday: last feature-level changes that should be done before beta, > e.g. removal of packaging > - Sunday: final feature freeze, bug fixing > - Monday: focus on stability of the buildbots, even unstable ones > - Tuesday: forking of the 3.3.0b1 release clone, tagging, start > of binary building I'd like to get the C implementation of the timing safe compare_digest into 3.3. http://bugs.python.org/issue15061 The patch went to several incarnations and I implemented input from Antoine, Serhiy and others. The function finally ended up as private function in the operator module because the _hashlib module isn't available without openssl and a new module for a single function is kinda overkill. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: #4489: Add a shutil.rmtree that isn't suspectible to symlink attacks
>> It is used automatically on platforms supporting the necessary os.openat() >> and >> os.unlinkat() functions. Main code by Martin von Löwis. > > Unfortunately, this isn't actually having any effect at the moment > since the os module APIs changed for the beta release. > > The "hasattr(os, 'unlinkat')" and "hasattr(os, 'openat')" checks need > to become "os.unlink in os.supports_dir_fd" and "os.open in > os.supports_dir_fd", and the affected calls need to be updated to pass > "dir_fd" as an argument to the normal versions of the functions. > > At least we know the graceful fallback to the old behaviour is indeed > graceful, though :) Yeah I've been told on IRC already. I'll commit a fix in a few minutes if my regression tests on OS X and Linux work fine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: #4489: Add a shutil.rmtree that isn't suspectible to symlink attacks
On Sun, Jun 24, 2012 at 2:18 AM, hynek.schlawack wrote: > http://hg.python.org/cpython/rev/c910af2e3c98 > changeset: 77635:c910af2e3c98 > user: Hynek Schlawack > date: Sat Jun 23 17:58:42 2012 +0200 > summary: > #4489: Add a shutil.rmtree that isn't suspectible to symlink attacks > > It is used automatically on platforms supporting the necessary os.openat() and > os.unlinkat() functions. Main code by Martin von Löwis. Unfortunately, this isn't actually having any effect at the moment since the os module APIs changed for the beta release. The "hasattr(os, 'unlinkat')" and "hasattr(os, 'openat')" checks need to become "os.unlink in os.supports_dir_fd" and "os.open in os.supports_dir_fd", and the affected calls need to be updated to pass "dir_fd" as an argument to the normal versions of the functions. At least we know the graceful fallback to the old behaviour is indeed graceful, though :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 3.3 release plans
Hi all, I've checked in the (hopefully final) update of PEP 398: all PEP scale changes are now final or deferred to 3.4. I also adjusted the release day to be the 26th of June, which leaves us with the following rough plan: - Saturday: last large changes, such as removal of packaging - Sunday: final feature freeze for 3.3; resolve last blockers from bugs.python.org - Monday: ensure build stability for stable buildbots, and as many unstable buildbots as possible - Tuesday: release clone forked off the main repo; tagging and start of binary building cheers, Georg -- NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone! Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty directory is a namespace?
On Sat, 23 Jun 2012 17:55:24 +0200 mar...@v.loewis.de wrote: > > That's true. I would have hoped for it to be recognized only when > > there's at least one module or package inside, but it doesn't sound > > easy to check for (especially in the recursive namespace packages case > > - is that possible?). > > Yes - a directory becomes a namespace package by not having an __init__.py, > so the "namespace package" case will likely become the default, and people > will start removing the empty __init__.pys when they don't need to support > 3.2- anymore. Have you tested the performance of namespace packages compared to normal packages? > In the long run, I expect that we will see namespace packages such as > org.openstack, com.canonical, com.ibm, etc. Then, "com" is a namespace > package, com.canonical is a namespace package, and com.canonical.launchpad > might still be a namespace package with multiple portions. I hope we are spared such naming schemes. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Antoine Pitrou pitrou.net> writes: > packaging already improves a lot over distutils. I don't see where I don't dispute that. > there is a credibility problem, except for people who think "distutils > is sh*t". I don't think you have to take such an extreme position in order to suggest that there might be problems with its basic design. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty directory is a namespace?
That's true. I would have hoped for it to be recognized only when there's at least one module or package inside, but it doesn't sound easy to check for (especially in the recursive namespace packages case - is that possible?). Yes - a directory becomes a namespace package by not having an __init__.py, so the "namespace package" case will likely become the default, and people will start removing the empty __init__.pys when they don't need to support 3.2- anymore. If you wonder whether a nested namespace package may have multiple portions: that can also happen, i.e. if you have z3c.recipe.ldap, z3c.recipe.template, z3c.recipe.sphinxdoc. They may all get installed as separate zip files, each contributing a portion to z3c.recipe. In the long run, I expect that we will see namespace packages such as org.openstack, com.canonical, com.ibm, etc. Then, "com" is a namespace package, com.canonical is a namespace package, and com.canonical.launchpad might still be a namespace package with multiple portions. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty directory is a namespace?
Should even an empty directory be a valid namespace package? Yes, that's what the PEP says, by BDFL pronouncement. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty directory is a namespace?
On Sat, 23 Jun 2012 08:38:02 -0700 Guido van Rossum wrote: > Yes. Otherwise, where to draw the line? What if it contains a single > dot file? What if it contains no Python files? What if it contains > only empty subdirectories? That's true. I would have hoped for it to be recognized only when there's at least one module or package inside, but it doesn't sound easy to check for (especially in the recursive namespace packages case - is that possible?). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty directory is a namespace?
Yes. Otherwise, where to draw the line? What if it contains a single dot file? What if it contains no Python files? What if it contains only empty subdirectories? On Sat, Jun 23, 2012 at 8:25 AM, Antoine Pitrou wrote: > > Hello, > > I've just noticed the following: > > $ mkdir foo > $ ./python > Python 3.3.0a4+ (default:837d51ba1aa2+1794308c1ea7+, Jun 23 2012, > 14:43:41) [GCC 4.5.2] on linux > Type "help", "copyright", "credits" or "license" for more information. import foo foo > > > > Should even an empty directory be a valid namespace package? > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Empty directory is a namespace?
Hello, I've just noticed the following: $ mkdir foo $ ./python Python 3.3.0a4+ (default:837d51ba1aa2+1794308c1ea7+, Jun 23 2012, 14:43:41) [GCC 4.5.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import foo >>> foo Should even an empty directory be a valid namespace package? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: #15114: the strict mode of HTMLParser and the HTMLParseError exception are
On Sat, Jun 23, 2012 at 3:29 PM, Antoine Pitrou wrote: > On Sat, 23 Jun 2012 15:28:00 +0200 > ezio.melotti wrote: >> >> + .. deprecated-removed:: 3.3 3.5 >> + The *strict* argument and the strict mode have been deprecated. >> + The parser is now able to accept and parse invalid markup too. >> + > > What if people want to accept only valid markup? > The problem with the "strict" mode is that is not really strict. Originally the parser was trying to work around some common errors (e.g. missing quotes around attribute values), but was giving up when other markup errors were encountered. When the non-strict mode was introduced, the old behavior was called "strict" and left unchanged for backward compatibility, even thought it wasn't strict enough to be used for validation and it was happy to parse some broken markup (but not other). At the same time the non-strict mode was able to accept some markup errors but not others, and sometimes parsing valid markup yielded different results in strict and non-strict modes. Then HTML5 was announced, with specific algorithms to parse both valid and invalid markup, so I improved the non-strict mode to 1) be able to parse everything; 2) try to be as close as the HTML5 standard as possible (I don't claim HTML5 conformance though). Now parsing a valid HTML page should give the same result in strict and non-strict mode, so the strict mode is now only useful if you want HTMLParseErrors for an arbitrary subset of markup errors. As someone already suggested, I should write a blog post explaining all this, but I'm still working on ironing out the last things in the code, so the blog post has yet to reach the top of my todo list. Best Regards, Ezio Melotti > Regards > > Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, 23 Jun 2012 14:14:46 + (UTC) Vinay Sajip wrote: > > Some > projects can be worked on in comparative isolation; other things, like > packaging, need inputs from a wider range of people to gain the necessary > credibility. packaging already improves a lot over distutils. I don't see where there is a credibility problem, except for people who think "distutils is sh*t". Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Antoine Pitrou pitrou.net> writes: > But what makes you think that redesigning everything would make those > Windows features magically available? Nothing at all. > This isn't about "representing" "constitutencies". python-dev is not a > bureaucracy, it needs people doing actual work. People could have Well, for example, PEP 397 was proposed by Mark Hammond to satisfy a particular constituency (people using multiple Python versions on Windows). Interested parties added their input. Then it got implemented and integrated. You can see from some of the Numpy/Scipy developer comments that some in that constituency feel that their needs aren't/weren't being addressed. > proposed patches for these features and they didn't do it (AFAIK). Sure they did - for example, I implemented Windows executable script handling as part of my work on testing venv operation with pysetup, and linked to it on the relevant ticket. It wasn't exactly rejected, though perhaps it wasn't reviewed because of lack of time, other priorities etc. and fell between the cracks. But I'm not making any complaint about this; there were definitely bigger issues to work on. I only bring it up in response to the "don't just talk about it; code doesn't write itself, you know" I read in your comment. > Like WSGI2 and other similar things, this is the kind of discussion > that will peter out in a few weeks and fall into oblivion. Quite possibly, but that'll be the chicken-and-egg thing I mentioned. Some projects can be worked on in comparative isolation; other things, like packaging, need inputs from a wider range of people to gain the necessary credibility. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 06/23/2012 03:20 PM, Vinay Sajip wrote: Dag Sverre Seljebotn astro.uio.no> writes: Of course you can always do anything, as numpy.distutils is a living proof of. Question is if it is good design. Can I be confident that the hooks are well-designed for my purposes? Only you can look at the design to determine that. But the point is I can't! I don't trust myself to do that. I've many times wasted days and weeks on starting to use tools that looked quite nice to me, but suddenly I get bit by needing to do something turns out to be almost impossible to do cleanly due to design constraints. That's why it's so import to me to rely on experts that have *more* experience than I do (such as David). (That's of course also why it's so important to copy designs from what works elsewhere. And have a really deep knowledge of those designs and their rationale.) On 06/23/2012 02:27 PM, Vinay Sajip wrote: > This deep understanding is not essential in the packaging/distutil2 team, > AFAICT. They just need to make sure that the hook APIs are sufficiently > flexible, that the hooks invoked at the appropriate time, and that they are > adequately documented with appropriate examples. All I'm doing is expressing my doubts that "making the hook API sufficiently flexible" and "invoked at the appropriate time" (and I'll add "has the right form") can be achieved at all without having subject expertise covering all the relevant usecases. Of course I can't mathematically prove this, it's just my experience as a software developer. Dag ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: #15114: the strict mode of HTMLParser and the HTMLParseError exception are
On Sat, 23 Jun 2012 15:28:00 +0200 ezio.melotti wrote: > > + .. deprecated-removed:: 3.3 3.5 > + The *strict* argument and the strict mode have been deprecated. > + The parser is now able to accept and parse invalid markup too. > + What if people want to accept only valid markup? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, 23 Jun 2012 13:14:42 + (UTC) Vinay Sajip wrote: > Kudos to Tarek, Éric and others for taking this particular world on their > shoulders and re-energizing the discussion and development work to date, but > it > seems the net needs to be spread even wider to ensure that all constituencies > are represented (for example, features needed only on Windows, such as binary > distributions and executable scripts, have lagged a little bit behind). But what makes you think that redesigning everything would make those Windows features magically available? This isn't about "representing" "constitutencies". python-dev is not a bureaucracy, it needs people doing actual work. People could have proposed patches for these features and they didn't do it (AFAIK). Like WSGI2 and other similar things, this is the kind of discussion that will peter out in a few weeks and fall into oblivion. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Dag Sverre Seljebotn astro.uio.no> writes: > Of course you can always do anything, as numpy.distutils is a living > proof of. Question is if it is good design. Can I be confident that the > hooks are well-designed for my purposes? Only you can look at the design to determine that. > And of course, propagating compilation options/configuration, and > auto-detecting configuration options, is one of the most important parts > of a complex build. Thus this seem to contradict what you say above. I'm not talking about those needs being invalid; just that distutils' way of fulfilling those needs is too fragile - else, why do you need things like Bento? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Antoine Pitrou pitrou.net> writes: > Remember that distutils2 was at first distutils. It was only decided to > be forked as a "new" package when some people complained. > This explains a lot of the methodology. Also, forking distutils helped > maintain a strong level of compatibility. Right, but distutils was hard to extend for a reason, even though designed with extensibility in mind; hence the rise of setuptools. I understand the pragmatic nature of the design decisions in packaging, but in this case a little too much purity was sacrificed for practicality. Compatibility at a command-line level should be possible to achieve even with a quite different internal design. > Apparently people now think it's time to redesign it all. That's fine, > but it won't work without a huge amount of man-hours. It's not like you > can write a couple of PEPs and call it done. Surely. But more than implementation man-hours, it requires that people are willing to devote some time and expertise in firming up the requirements, use cases etc. to go into the PEPs. It's classic chicken-and-egg; no-one wants to invest that time until they know a project's going somewhere and will have widespread backing, but the project won't go anywhere quickly unless they step up and invest the time up front. Kudos to Tarek, Éric and others for taking this particular world on their shoulders and re-energizing the discussion and development work to date, but it seems the net needs to be spread even wider to ensure that all constituencies are represented (for example, features needed only on Windows, such as binary distributions and executable scripts, have lagged a little bit behind). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 06/23/2012 02:27 PM, Vinay Sajip wrote: Dag Sverre Seljebotn astro.uio.no> writes: As for me, I believe I've been rather blunt and direct in my criticism in this thread: It's been said by Tarek that distutils2 authors that they don't know anything about compilers. Therefore it's almost unconceivable to me that much good can come from distutils2 *for my needs*. Even if packaging and building isn't the same, the two issues do tangle at a fundamental level, *and* most existing solutions already out there (RPM, MSI..) distribute compiled software and therefore one needs a solid understanding of build processes to also understand these tools fully and draw on their experiences and avoid reinventing the wheel. But packaging/distutils2 contains functionality for hooks, which can be used to implement custom builds using tools that packaging/distutils2 doesn't need to know or care about (a hook will do that). One can imagine that a set of commonly used templates would become available over time, so that some problems wouldn't need to have solutions re-invented. Of course you can always do anything, as numpy.distutils is a living proof of. Question is if it is good design. Can I be confident that the hooks are well-designed for my purposes? I think Bento's hook concept was redesigned 2 or 3 times to make sure it fit well... Somebody with a deep understanding of 3-4 existing build systems and long experience in cross-platform builds and cross-architecture builds would need to be on board for me to take it seriously (even the packaging parts). As per Tarek's comments, I'm therefore pessimistic about the distutils2 efforts. This deep understanding is not essential in the packaging/distutil2 team, AFAICT. They just need to make sure that the hook APIs are sufficiently flexible, that the hooks invoked at the appropriate time, and that they are adequately documented with appropriate examples. For me, the bigger problem with the present distutils2/packaging implementation is that it propagates the command-class style of design which IMO caused so much pain in extending distutils. Perhaps some of the dafter limitations have been removed, and no doubt the rationale was to get to something usable more quickly, but it seems a bit like papering over cracks. The basic low-level building blocks like versioning, metadata and markers should be fine, but I'm not convinced that the command-class paradigm is appropriate in this case. The whole intricate "initialize_options"/"finalize_options"/"set_undefined_options" /"get_finalized_command"/"reinitialize_command" dance just makes me say, And of course, propagating compilation options/configuration, and auto-detecting configuration options, is one of the most important parts of a complex build. Thus this seem to contradict what you say above. (Sorry all, I felt like I should answer to a direct challenge. This will be my last post in this thread; I've subscribed to distutils-sig.) Dag ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
On Sat, 23 Jun 2012 12:32:37 + Kristján Valur Jónsson wrote: > Au contraire, it is actually a very major improvement, the result of pretty > extensive profiling, see > http://blog.ccpgames.com/kristjan/2012/05/25/optimizing-python-condition-variables-with-telemetry/ It might be. But to evaluate it, we have to digest a long technical blog post, then carefully review the patch (and perhaps even re-do some measurements). You understand that it's not reasonable to do so just one day before feature freeze. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, 23 Jun 2012 12:27:52 + (UTC) Vinay Sajip wrote: > > For me, the bigger problem with the present distutils2/packaging > implementation > is that it propagates the command-class style of design which IMO caused so > much pain in extending distutils. Perhaps some of the dafter limitations have > been removed, and no doubt the rationale was to get to something usable more > quickly, but it seems a bit like papering over cracks. Remember that distutils2 was at first distutils. It was only decided to be forked as a "new" package when some people complained. This explains a lot of the methodology. Also, forking distutils helped maintain a strong level of compatibility. Apparently people now think it's time to redesign it all. That's fine, but it won't work without a huge amount of man-hours. It's not like you can write a couple of PEPs and call it done. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
Au contraire, it is actually a very major improvement, the result of pretty extensive profiling, see http://blog.ccpgames.com/kristjan/2012/05/25/optimizing-python-condition-variables-with-telemetry/ The proposed patch reduces signaling latency in busy applications as demonstrated by the example program from 10s of milliseconds to about one, on my 64 bit windows box. This matters very much for applications using threading.Condition to dispatch work to threads. This includes those using queue.Queue(). K Frá: python-dev-bounces+kristjan=ccpgames@python.org [python-dev-bounces+kristjan=ccpgames@python.org] fyrir hönd Antoine Pitrou [solip...@pitrou.net] Sent: 23. júní 2012 11:54 To: python-dev@python.org Efni: Re: [Python-Dev] 3.3 release plans On Sat, 23 Jun 2012 13:12:19 +0200 Antoine Pitrou wrote: > On Sat, 23 Jun 2012 11:00:34 + > Kristján Valur Jónsson wrote: > > I realize it is late, but any chance to get > > http://bugs.python.org/issue15139 in today? > > -1. Let me elaborate: the patch hasn't been reviewed, and it's a very minor improvement (assuming it's an improvement at all) in a rather delicate area. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Dag Sverre Seljebotn astro.uio.no> writes: > As for me, I believe I've been rather blunt and direct in my criticism > in this thread: It's been said by Tarek that distutils2 authors that > they don't know anything about compilers. Therefore it's almost > unconceivable to me that much good can come from distutils2 *for my > needs*. Even if packaging and building isn't the same, the two issues do > tangle at a fundamental level, *and* most existing solutions already out > there (RPM, MSI..) distribute compiled software and therefore one needs > a solid understanding of build processes to also understand these tools > fully and draw on their experiences and avoid reinventing the wheel. But packaging/distutils2 contains functionality for hooks, which can be used to implement custom builds using tools that packaging/distutils2 doesn't need to know or care about (a hook will do that). One can imagine that a set of commonly used templates would become available over time, so that some problems wouldn't need to have solutions re-invented. > Somebody with a deep understanding of 3-4 existing build systems and > long experience in cross-platform builds and cross-architecture builds > would need to be on board for me to take it seriously (even the > packaging parts). As per Tarek's comments, I'm therefore pessimistic > about the distutils2 efforts. This deep understanding is not essential in the packaging/distutil2 team, AFAICT. They just need to make sure that the hook APIs are sufficiently flexible, that the hooks invoked at the appropriate time, and that they are adequately documented with appropriate examples. For me, the bigger problem with the present distutils2/packaging implementation is that it propagates the command-class style of design which IMO caused so much pain in extending distutils. Perhaps some of the dafter limitations have been removed, and no doubt the rationale was to get to something usable more quickly, but it seems a bit like papering over cracks. The basic low-level building blocks like versioning, metadata and markers should be fine, but I'm not convinced that the command-class paradigm is appropriate in this case. The whole intricate "initialize_options"/"finalize_options"/"set_undefined_options" /"get_finalized_command"/"reinitialize_command" dance just makes me say, "Seriously?". Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signed packages
I'm surprised gpg hasn't been mentioned here. I think these are all solved problems, most free software that is signed signs it with the gpg key of the author. In that case all that is needed is that the cheeseshop allows the uploading of the signature. For the record, the cheeseshop has been supporting pgp signatures for about ten years now. Several projects have been using that for quite a while in their releases. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
On Sat, 23 Jun 2012 13:12:19 +0200 Antoine Pitrou wrote: > On Sat, 23 Jun 2012 11:00:34 + > Kristján Valur Jónsson wrote: > > I realize it is late, but any chance to get > > http://bugs.python.org/issue15139 in today? > > -1. Let me elaborate: the patch hasn't been reviewed, and it's a very minor improvement (assuming it's an improvement at all) in a rather delicate area. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, Jun 23, 2012 at 9:53 PM, David Cournapeau wrote: > Nick, I am unfamiliar with python-ideas rules: should we continue > discussion in distutils-sig entirely, or are there some specific > topics that are more appropriate for python-ideas ? No, I think I just need to join distutils-sig. python-ideas is more for ideas that don't have an appropriate SIG list. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, Jun 23, 2012 at 1:25 PM, Nick Coghlan wrote: > If you think that, you haven't read the whole thread. This is true, I kinda gave up early yesterday. It's good that it became better. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, Jun 23, 2012 at 12:25 PM, Nick Coghlan wrote: > On Sat, Jun 23, 2012 at 8:37 PM, Lennart Regebro wrote: >> In the end, I think this discussion is very similar to all previous >> packaging/building/installing discussions: There is a lot of emotions, >> and a lot of willingness to declare that "X sucks" but very little >> concrete explanation of *why* X sucks and why it can't be fixed. > > If you think that, you haven't read the whole thread. Thanks to this > discussion, I now have a *much* clearer idea of what's broken, and a > few ideas on what can be done to fix it. > > However, distutils-sig and python-ideas will be the place to post about those. Nick, I am unfamiliar with python-ideas rules: should we continue discussion in distutils-sig entirely, or are there some specific topics that are more appropriate for python-ideas ? David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signed packages
Oh sorry, having read the thread this spawned from I see you're taking about MS Windows singed binaries. Something I know next to nothing about, so ignore my babbling. On 23 June 2012 11:52, Floris Bruynooghe wrote: > On 22 June 2012 17:56, Donald Stufft wrote: >> On Friday, June 22, 2012 at 12:54 PM, Alexandre Zani wrote: >> >> Key distribution is the real issue though. If there isn't a key >> distribution infrastructure in place, we might as well not bother with >> signatures. PyPI could issue x509 certs to packagers. You wouldn't be >> able to verify that the name given is accurate, but you would be able >> to verify that all packages with the same listed author are actually >> by that author. >> >> I've been sketching out ideas for key distribution, but it's very much >> a chicken and egg problem, very few people sign their packages (because >> nothing uses it currently), and nobody is motivated to work on >> infrastructure >> or tooling because no one signs their packages. > > > I'm surprised gpg hasn't been mentioned here. I think these are all > solved problems, most free software that is signed signs it with the > gpg key of the author. In that case all that is needed is that the > cheeseshop allows the uploading of the signature. As for key > distribution, the keyservers take care of that just fine and we'd > probably see more and better attended signing parties at python > conferences. > > Regards, > Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Sat, Jun 23, 2012 at 8:37 PM, Lennart Regebro wrote: > In the end, I think this discussion is very similar to all previous > packaging/building/installing discussions: There is a lot of emotions, > and a lot of willingness to declare that "X sucks" but very little > concrete explanation of *why* X sucks and why it can't be fixed. If you think that, you haven't read the whole thread. Thanks to this discussion, I now have a *much* clearer idea of what's broken, and a few ideas on what can be done to fix it. However, distutils-sig and python-ideas will be the place to post about those. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
On Sat, 23 Jun 2012 11:00:34 + Kristján Valur Jónsson wrote: > I realize it is late, but any chance to get http://bugs.python.org/issue15139 > in today? -1. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 06/23/2012 12:37 PM, Lennart Regebro wrote: Why do I get the feeling that most people who hate distutils and want to replace it, has transferred those feelings to distutils2/packaging, mainly because of the name? In the end, I think this discussion is very similar to all previous packaging/building/installing discussions: There is a lot of emotions, and a lot of willingness to declare that "X sucks" but very little concrete explanation of *why* X sucks and why it can't be fixed. I think David has been pretty concrete in a lot of his criticism too (though he has refrained from repeating himself too much in this thread). Some of the criticism is spelled out here: http://bento.readthedocs.org/en/latest/faq.html This blog post is even more concrete (but perhaps outdated?): http://cournape.wordpress.com/2010/10/13/271/ As for me, I believe I've been rather blunt and direct in my criticism in this thread: It's been said by Tarek that distutils2 authors that they don't know anything about compilers. Therefore it's almost unconceivable to me that much good can come from distutils2 *for my needs*. Even if packaging and building isn't the same, the two issues do tangle at a fundamental level, *and* most existing solutions already out there (RPM, MSI..) distribute compiled software and therefore one needs a solid understanding of build processes to also understand these tools fully and draw on their experiences and avoid reinventing the wheel. Somebody with a deep understanding of 3-4 existing build systems and long experience in cross-platform builds and cross-architecture builds would need to be on board for me to take it seriously (even the packaging parts). As per Tarek's comments, I'm therefore pessimistic about the distutils2 efforts. (You can always tell me that I shouldn't criticise unless I'm willing to join and do something about it. That's fair. I'm just saying that my unwillingness to cheer for distutils2 is NOT based on the name only!) Dag ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.3 release plans
I realize it is late, but any chance to get http://bugs.python.org/issue15139 in today? Frá: python-dev-bounces+kristjan=ccpgames@python.org [python-dev-bounces+kristjan=ccpgames@python.org] fyrir hönd g.brandl-nos...@gmx.net [g.brandl-nos...@gmx.net] Sent: 23. júní 2012 10:54 To: python-dev@python.org Efni: [Python-Dev] 3.3 release plans Hi all, now that the final PEP scheduled for 3.3 is final, we're entering the next round of the 3.3 cycle. I've decided to make Tuesday 26th the big release day. That means: - Saturday: last feature-level changes that should be done before beta, e.g. removal of packaging - Sunday: final feature freeze, bug fixing - Monday: focus on stability of the buildbots, even unstable ones - Tuesday: forking of the 3.3.0b1 release clone, tagging, start of binary building cheers, Georg -- NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone! Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 3.3 release plans
Hi all, now that the final PEP scheduled for 3.3 is final, we're entering the next round of the 3.3 cycle. I've decided to make Tuesday 26th the big release day. That means: - Saturday: last feature-level changes that should be done before beta, e.g. removal of packaging - Sunday: final feature freeze, bug fixing - Monday: focus on stability of the buildbots, even unstable ones - Tuesday: forking of the 3.3.0b1 release clone, tagging, start of binary building cheers, Georg -- NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone! Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signed packages
On 22 June 2012 17:56, Donald Stufft wrote: > On Friday, June 22, 2012 at 12:54 PM, Alexandre Zani wrote: > > Key distribution is the real issue though. If there isn't a key > distribution infrastructure in place, we might as well not bother with > signatures. PyPI could issue x509 certs to packagers. You wouldn't be > able to verify that the name given is accurate, but you would be able > to verify that all packages with the same listed author are actually > by that author. > > I've been sketching out ideas for key distribution, but it's very much > a chicken and egg problem, very few people sign their packages (because > nothing uses it currently), and nobody is motivated to work on > infrastructure > or tooling because no one signs their packages. I'm surprised gpg hasn't been mentioned here. I think these are all solved problems, most free software that is signed signs it with the gpg key of the author. In that case all that is needed is that the cheeseshop allows the uploading of the signature. As for key distribution, the keyservers take care of that just fine and we'd probably see more and better attended signing parties at python conferences. Regards, Floris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Why do I get the feeling that most people who hate distutils and want to replace it, has transferred those feelings to distutils2/packaging, mainly because of the name? In the end, I think this discussion is very similar to all previous packaging/building/installing discussions: There is a lot of emotions, and a lot of willingness to declare that "X sucks" but very little concrete explanation of *why* X sucks and why it can't be fixed. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] AUTO: Jon K Peck is out of the office (returning 06/30/2012)
I am out of the office until 06/30/2012. I will be out of the office the week of June 25. I expect to have some email access but will likely be delayed in responding. Note: This is an automated response to your message "Python-Dev Digest, Vol 107, Issue 104" sent on 06/23/2012 2:02:24. This is the only notification you will receive while this person is away.___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] On a new version of pickle [PEP 3154]: self-referential frozensets
Hello,
I'm one of this year's Google Summer of Code students working
on improving pickle by creating a new version. My name is Stefan and
my mentor is Alexandre Vassalotti.
If you're interested, you can monitor the progress in the dedicated
blog at [2] and the bitbucket repository at [3].
One of the goals for picklev4 is to add native opcodes for pickling
of sets and frozensets. Currently these 4 opcodes were added:
* EMPTY_SET, EMPTY_FROZENSET: push an empty set/frozenset in the stack
* UPDATE_SET: update the set in the stack with the top stack slice
stack before: ... pyset mark stackslice
stack after : ... pyset
effect: pyset.update(stackslice) # inplace union
* UNION_FROZENSET: like UPDATE_SET, but create a new frozenset
stack before: ... pyfrozenset mark stackslice
stack after : ... pyfrozenset.union(stackslice)
While this design allows pickling of self-referential sets, self-referential
frozensets are still problematic. For instance, trying to pickle `fs':
a=A(); fs=frozenset([a]); a.fs = fs
(when unpickling, the object a has to be initialized before it is added to
the frozenset)
The only way I can think of to make this work is to postpone
the initialization of all the objects inside the frozenset until after
UNION_FROZENSET.
I believe this is doable, but there might be memory penalties if the
approach
is to simply store all the initialization opcodes in memory until
pickling the frozenset is finished.
Currently, pickle.dumps(fs,4) generates:
EMPTY_FROZENSET
BINPUT 0
MARK
BINGLOBAL_COMMON '0 A' # same as GLOBAL '__main__ A' in v3
EMPTY_TUPLE
NEWOBJ
EMPTY_DICT
SHORT_BINUNICODE 'fs'
BINGET 0 # retrieves the frozenset which is empty at this
point, and it
# will never be filled because it's immutable
SETITEM
BUILD # a.__setstate__({'fs' : frozenset()})
UNION_FROZENSET
By postponing the initialization of a, it should instead generate:
EMPTY_FROZENSET
BINPUT 0
MARK
BINGLOBAL_COMMON '0 A' # same as GLOBAL '__main__ A' in v3
EMPTY_TUPLE
NEWOBJ # create the object but don't initialize its state yet
BINPUT 1
UNION_FROZENSET
BINGET 1
EMPTY_DICT
SHORT_BINUNICODE 'fs'
BINGET 0
SETITEM
BUILD
POP
While self-referential frozensets are uncommon, a far more problematic
situation is with the self-referential objects created with REDUCE. While
pickle uses the idea of creating empty collections and then filling them,
reduce tipically creates already-filled objects. For instance:
cnt = collections.Counter(); cnt[a]=3; a.cnt=cnt; cnt.__reduce__()
(, ({<__main__.A object at 0x0286E8F8>: 3},))
where the A object contains a reference to the counter. Unpickling an
object pickled with this reduce function is not possible, because the reduce
function, which "explains" how to create the object, is asking for the
object
to exist before being created.
The fix here would be to pass Counter's dictionary in the state argument,
as opposed to the "constructor parameters" one, as follows:
(, (), {<__main__.A object at 0x0286E8F8>: 3})
When unpickling this, an empty Counter will be created first, and then
__setstate__ will be called to fill it, at which point self-references
are allowed.
I assume this modification has to be done in the implementations of the data
structures rather than in pickle itself. Pickle could try to fix this by
detecting
when reduce returns a class type as the first tuple arg and move the
dict ctor parameter to the state, but this may not always be intended.
It's also a bit strange that __getstate__ is never used anywhere in
pickle directly.
I'm looking forward to hearing your suggestions and opinions in this matter.
Regards,
Stefan
[1] http://www.python.org/dev/peps/pep-3154/
[2] http://pypickle4.wordpress.com/
[3] http://bitbucket.org/mstefanro/pickle4
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Checking if unsigned int less then zero.
> Playing with cpython source, I found some strange strings in
> socketmodule.c:
>
> ---
> if (flowinfo < 0 || flowinfo > 0xf) {
> PyErr_SetString(
> PyExc_OverflowError,
> "getsockaddrarg: flowinfo must be 0-1048575.");
> return 0;
> }
> ---
>
> ---
> if (flowinfo < 0 || flowinfo > 0xf) {
> PyErr_SetString(PyExc_OverflowError,
> "getsockaddrarg: flowinfo must be 0-1048575.");
> return NULL;
> }
> ---
>
> The flowinfo variable declared few strings above as unsgined int. Is
> there any practical sense in this check? Seems like gcc just removes
> this check. I think any compiler will generate code that checks as
> unsigned, for example in x86 its JAE/JGE. May be this code is for "bad"
> compilers or exotic arch?
Removed.
Thanks,
cf
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Paul Moore writes: > I suppose if you're saying that "pip install lxml" should download and > install for me Visual Studio, libxml2 sources and any dependencies, > and run all the builds, then you're right. But I assume you're not. Indeed, if only a source package is available, it should. That's precisely what happens for source builds that depend on a non-system compiler on say Gentoo or MacPorts. What I'm saying is that the packaging system should *always be prepared* to offer that service (perhaps via plugins to OS distros' PMSes). Whether a particular package does, or not, presumably is up to the package's maintainer. Even if a binary package is available, it may only be partial. Indeed, by some definitions it alway will be (it will depend on an OS being installed!) Such a package should *also* offer the ability to fix up an incomplete system, by building from source if needed. Why can I say "should" here? Because if the packaging standard is decent and appropriate tools provided, there will be a source package because it's the easiest way to create a distribution! Such tools will surely be able to search the system for a preinstalled dependency, or a binary package cache for an installable package. A "complete" binary package would just provide the package cache on its distribution medium. > So why should I need to install Visual Studio just to *use* lxml? Because the packaging standard cannot mandate "high quality" packages from any given user's perspective, it can only provide the necessary features to implement them. If the lxml maintainer chooses to depend on a pre-installed libxml2, AFAICS you're SOL -- you need to go elsewhere for the library. VS + libxml2 source is just the most reliable way to go elsewhere in some sense (prebuilt binaries have a habit of showing up late or built with incompatible compiler options or the like). > But I do think that there's a risk that the discussion, because it > is necessarily driven by developers, forgets that "end users" > really don't have some tools that a developer would consider > "trivial" to have. I don't understand the problem. As long as binary packages have a default spec to depend on nothing but a browser to download the MSI, all you need is a buildbot that has a minimal Windows installation, and it won't be forgotten. The developers may forget to check, but the bot will remember! I certainly agree that the *default* spec should be that if you can get your hands on binary installer, that's all you need -- it will do *all* the work needed. Heck, it's not clear to me what else the default spec might be for a binary package. OTOH, if the developers make a conscious decision to depend on a given library, and/or a compiler, being pre-installed on the target system, what can Python or the packaging standard do about that? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ssh://h...@hg.python.org/cpython unstable?
Yes. The network link on dinsdale is maxed out, since it hosts too many services (and since there was a massive increase in traffic over the last year). As a consequence, connections may time out. We are working on migrating services to a new machine. Unfortunately, the current hold-up is that OSU/OSL is slow in assigning IP addresses, which means that we cannot create new VMs as quickly as we wish. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
