Re: [Python-Dev] Coroutines and PEP 380

[Mark Shannon]

Matt Joiner wrote:
Just to clarify, this differs in functionality from enhanced generators 
by allowing you to yield from an arbitrary call depth rather than having 
to "yield from" through a chain of calling generators? Furthermore 
there's no syntactical change except to the bottommost frame doing a 
co_yield? Does this capture the major differences?



Yes.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Coroutines and PEP 380

[Mark Shannon]

Glyph wrote:

On Jan 17, 2012, at 5:03 PM, Mark Shannon wrote:


Lets start controversially: I don't like PEP 380, I think it's a kludge.


Too late; it's already accepted.  There's not much point in making 
controversial statements about it now.


Why is it too late? Presenting this as a fait accompli does not make it 
any better. The PEP mailing list is closed to most people, so what forum 
for debate is there?




I think that CPython should have proper coroutines, rather than add 
more bits and pieces to generators in an attempt to make them more 
like coroutines.


By "proper" coroutines, you mean implicit coroutines (cooperative 
threads) rather than explicit coroutines (cooperative generators). 

Nothing "implicit" about it.
 Python has been going in the "explicit" direction on this question for 
a long time.  (And, in my opinion, this is the right direction to go, 
but that's not really relevant here.)


You can use asymmetric coroutines with a scheduler to provide 
cooperative threads if you want, but coroutines not have to be used as 
threads.


The key advantages of my coroutine implmentation over PEP 380 are:

1. No syntax change.
2. Code can be used in coroutines without modification.
3. No stack unwinding is required at a yield point.



I think this discussion would be more suitable for python-ideas though, 
since you have a long row to hoe here.  There's already a PEP - 
http://www.python.org/dev/peps/pep-0219/ - apparently deferred and not 
rejected, which you may want to revisit.


There are several libraries which can give you cooperative threading 
already; I assume you're already aware of greenlet and stackless, but I 
didn't see what advantages your proposed implementation provides over 
those.  I would guess that one of the first things you should address on 
python-ideas is why adopting your implementation would be a better idea 
than just bundling one of those with the standard library :).


Already been discussed:
http://mail.python.org/pipermail/python-ideas/2011-October/012571.html

All of the objections to coroutines (as I propose) also apply to PEP 380.

The advantage of my implementation over greenlets is portability.

I suspect stackless is actually fairly similar to what I have done,
I haven't checked in detail.

Cheers,
Mark.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

[Victor Stinner]

2012/1/18 "Martin v. Löwis" :
> For 3.3 onwards, I'm skeptical whether all this configuration support is
> really necessary. I think a much smaller patch which leaves no choice
> would be more appropriate.

The configuration helps unit testing: see changes on Lib/test/*.py in
my last patch. I hesitate to say that the configuration is required
for tests. Anyway, users upgrading from Python 3.2 to 3.3 may need to
keep the same hash function and don't care of security (e.g. programs
running locally with trusted data).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

[Hrvoje Niksic]

On 01/17/2012 09:29 PM, "Martin v. Löwis" wrote:

I(0) = H&  MASK
PERTURB(0) = H
I(n+1) = (5*I(n) + 1 + PERTURB(n))&  MASK
PERTURN(n+1) = PERTURB(n)>>  5

So if two objects O1 and O2 have the same hash value H, the sequence of
probed indices is the same for any MASK value. It will be a different
sequence, yes, but they will still collide on each and every slot.

This is the very nature of open addressing.


Open addressing can still deploy a collision resolution mechanism 
without this property. For example, double hashing uses a different hash 
function (applied to the key) to calculate PERTURB(0). To defeat it, the 
attacker would have to produce keys that hash the same using both hash 
functions.


Double hashing is not a good general solution for Python dicts because 
it complicates the interface of hash tables that support arbitrary keys. 
Still, it could be considered for dicts with known key types (built-ins 
could hardcode the alternative hash function) or for SafeDicts, if they 
are still considered.


Hrvoje
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Antoine Pitrou]

On Wed, 18 Jan 2012 07:52:20 +
Paul Moore  wrote:
> On 18 January 2012 07:46, Georg Brandl  wrote:
> >> But I am dubious that releases that are obsolete in 6 months and lack
> >> 3rd party support will see much production use.
> >
> > Whether people would use the releases is probably something that only
> > they can tell us -- that's why a community survey is mentioned in the
> > PEP.
> 
> The class of people who we need to consider carefully is those who
> want to use the latest release, but are limited by the need for other
> parties to release stuff that works with that release (usually, this
> means Windows binaries of extensions, or platform vendor packaged
> releases of modules/packages).

Well, do consider, though, that anyone not using third-party C
extensions under Windows (either Windows users that are content with
pure Python libs, or users of other platforms) won't have that problem.
That should be quite a lot of people already.

As for vendors, they have their own release management independent of
ours already, so this PEP wouldn't change anything for them.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Antoine Pitrou]

On Wed, 18 Jan 2012 11:37:08 +0900
"Stephen J. Turnbull"  wrote:
>  > availability of release management volunteers,
> 
> Dramatic increase here.  It may look like RM is not so demanding --
> run a few scripts to put out the alphas/betas/releases.  But the RM
> needs to stay on top of breaking news, make decisions.  That takes
> time, interrupts other work, etc.

Georg and Barry may answer you here: they are release managers and PEP
co-authors.

>  > quick availability of new features (and behavioural changes),
> 
> These are already *available*, just not *tested*.
> 
> Since testing is the bottleneck on what users consider to be
> "available for me", you cannot decrease the amount of testing (alpha,
> beta releases) by anywhere near the amount you're increasing
> frequency, or you're just producing "as is" snapshots.

The point is to *increase* the amount of testing by making features
available in stable releases on a more frequent basis. Not decrease it.

Alphas and betas never produce much feedback, because people are
reluctant to install them for anything else than toying around. Python
is not emacs or Firefox, you don't use it in a vacuum and therefore
installing non-stable versions is dangerous.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Nick Coghlan]

This won't be a surprise to Antoine or Georg (since I've already
expressed the same opinion privately), but I'm -1 on the idea of
official releases of the whole shebang every 6 months. We're not
Ubuntu, Fedora, Chrome or Firefox with a for-profit company (or large
foundation) with multiple paid employees kicking around to really
drive the QA process. If we had official support from Red Hat or
Canonical promising to devote paid QA and engineering resources to
keeping things on track my opinion might be different, but that is
highly unlikely. I'm also wholly in agreement with Ezio that using the
same versioning scheme for both full releases and interim releases is
thoroughly confusing for users (for example, I consider Red Hat's
completely separate branding and versioning for Fedora and RHEL a
better model for end users than Canonical's more subtle 'Ubuntu' and
'Ubuntu LTS' distinction, and that's been my opinion since long before
I started working for RH).

My original suggestion to Antoine and Georg for 3.4 was that we simply
propose to Larry Hastings (the 3.4 RM) that we spread out the release
cycle, releasing the first alpha after ~6 months, the second after
about ~12, then rolling into the regular release cycle of a final
alpha, some beta releases, one or two release candidates and then the
actual release. However, I'm sympathetic to Antoine's point that early
alphas aren't likely to be at all interesting to folks that would like
a fully supported stdlib update to put into production and no longer
think that suggestion makes much sense on its own.

Instead, if the proposal involves instituting a PEP 3003 style
moratorium (i.e. stdlib changes only) for all interim releases, then
we're essentially talking about splitting the versioning of the core
language (and the CPython C API) and the standard library. If we're
going to discuss that, we may as well go a bit further and just split
development of the two out onto separate branches, with the current
numbering scheme applying to full language version releases and
switching to a date-based versioning scheme for the standard library
(i.e. if 3.3 goes out in August as planned, then it would be "Python
3.3 with the 12.08 stdlib release").

What might such a change mean?


1. For 3.3, the following releases would be made:
- 3.2.x is cut from the 3.2 branch (1 rc + 1 release)
- 3.3.0 + PyStdlib 12.08 is created from the default branch (1
alpha, 2 betas, 1+ rc, 1 release)
- the 3.3 maintenance branch is created
- the stdlib development branch is created

2. Once 3.2 goes into security-fix only mode, this would then leave us
with 4 active branches:
- 2.7 (maintenance)
- 3.3 (maintenance)
- stdlib (Python 3.3 compatible, PEP 3003 compliant updates)
- default (3.4 development)

The 2.7 branch would remain a separate head of development, but for
3.x development the update flow would become:
Bug fixes: 3.3->stdlib->default
Stdlib features: stdlib->default
Language changes: default

3. Somewhere around February 2013, we prepare to release Python 3.4a1
and 3.3.1, along with PyStdlib 13.02:
- 3.3.1 + PyStdlib 12.08 is cut from the 3.3 branch (1 rc + 1 release)
- 3.3.1 + PyStdlib 13.02 comes from the stdlib branch (1 alpha, 1
beta, 1+ rc, 1 release)
- 3.4.0a1 comes from the default branch (may include additional
stdlib changes)

4. Around August 2013 this process repeats:
- 3.3.2 + PyStdlib 12.08 is cut from the 3.3 branch
- 3.3.2 + PyStdlib 13.08 comes from the stdlib branch (final 3.3
compatible stdlib release)
- 3.4.0a2 comes from the default branch

5. And then in February 2014, we gear up for a new major release:
- 3.3.3 is cut from the 3.3 branch and the 3.3 branch enters
security fix only mode
- 3.4.0 + PyStdlib 14.02 is created from the default branch (1
alpha, 2 betas, 1+ rc, 1 release)
- the 3.4 maintenance branch is created and merged into the stdlib branch

(alternatively, Feb 2014 could be another interim release of 3.4 alpha
and a 3.3 compatible stdlib updated, with 3.4 delayed until August
2014)

I believe this approach would get to the core of what the PEP authors
want (i.e. more frequent releases of the standard library), while
being quite explicit in *avoiding* the concerns associated with more
frequent releases of the core language itself. The rate of updates on
the language spec, the C API (and ABI), the bytecode format and the
AST would remain largely unchanged at 18-24 months. Other key
protocols (e.g. default pickle formats) could also be declared
ineligible for changes in interim releases.

If a critical security problem is found, then additional releases may
be cut for the maintenance branch and for the stdlib branch.

There's a slight annoyance in having all development filtered through
an additional branch, but there's a large advantage in that having a
stable core in the stdlib branch makes it more likely we'll be able to
use it as a venue for collaboration with the

Re: [Python-Dev] Coroutines and PEP 380

[Glyph]

On Jan 18, 2012, at 4:23 AM, Mark Shannon wrote:

> Glyph wrote:
>> On Jan 17, 2012, at 5:03 PM, Mark Shannon wrote:
>>> Lets start controversially: I don't like PEP 380, I think it's a kludge.
>> Too late; it's already accepted.  There's not much point in making 
>> controversial statements about it now.
> 
> Why is it too late?

Because discussion happens before the PEP is accepted.  See the description of 
the workflow in .  The time to object 
to PEP 380 was when those threads were going on.

> Presenting this as a fait accompli does not make it any better.

But it is[1] a fait accompli, whether you like it or not; I'm first and 
foremost informing you of the truth, not trying to make you feel better (or 
worse).  Secondly, I am trying to forestall a long and ultimately pointless 
conversation :).

> The PEP mailing list is closed to most people,

The PEP mailing list is just where you submit your PEPs, and where the PEP 
editors do their work.  I'm not on it, but to my understanding of the process, 
there's not really any debate there.

> so what forum for debate is there?

python-ideas, and then this mailing list, in that order.  Regarding PEP 380 
specifically, there's been quite a bit.  See for example 
.  Keep in 
mind that the purpose of debate in this context is to inform Guido's opinion.  
There's no voting involved, although he will occasionally delegate decisions 
about particular PEPs to people knowledgeable in a relevant area.

>> I think this discussion would be more suitable for python-ideas though [...]
> Already been discussed:
> http://mail.python.org/pipermail/python-ideas/2011-October/012571.html

If you're following the PEP process, then the next step would be for you 
(having built some support) to author a new PEP, or to resurrect the deferred 
Stackless PEP with some new rationale - personally I'd recommend the latter.

My brief skimming of the linked thread doesn't indicate you have a lot of 
strong support though, just some people who would be somewhat interested.  So I 
still think it bears more discussion there, especially on the motivation / 
justification side of things.

> All of the objections to coroutines (as I propose) also apply to PEP 380.

You might want to see the video of Guido's "Fireside Chat" last year 
.  Skip to a little before 15:00.  He mentions the 
point that coroutines that can implicitly switch out from under you have the 
same non-deterministic property as threads: you don't know where you're going 
to need a lock or lock-like construct to update any variables, so you need to 
think about concurrency more deeply than if you could explicitly always see a 
'yield'.  I have more than one "painful event in my past" (as he refers to it) 
indicating that microthreads have the same problem as real threads :).

(And yes, they're microthreads, even if you don't have an elaborate scheduling 
construct.  If you can switch to another stack by making a function call, then 
you are effectively context switching, and it can become arbitrarily complex.  
Any coroutine in a system may introduce an arbitrarily complex microthread 
scheduler just by calling a function that yields to it.)

-glyph

([1]: Well actually it isn't, note the dashed line from "Accepted" to 
"Rejected" in the workflow diagram.  But you have to have a really darn good 
reason, and championing the rejection of a pep that Guido has explicitly 
accepted and has liked from pretty much the beginning is going to be very, very 
hard.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hashing proposal: change only string-only dicts

[Barry Warsaw]

On Jan 18, 2012, at 08:19 AM, Martin v. Löwis wrote:

>My concern is not about breaking doctests: this proposal will also break
>them. My concern is about applications that assume that hash(s) is
>stable across runs, and we do have reports that it will break
>applications.

I am a proponent of doctests, and thus use them heavily.  I can tell you that
the issue of dict hashing (non-)order has been well known for *years* and I
have convenience functions in my own doctests to sort and print dict
elements.  Back in my Launchpad days (which has oodles of doctests), many
years ago we went on a tear to fix dict printing when some change in Python
caused them to break.  So I'm not personally worried that such a change would
break any of my own code.

Even though I hope anybody who uses doctests has their own workarounds for
this, I still support being conservative in default behavior for stable
releases, because it's the right thing to do for our users.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Antoine Pitrou]

Le mercredi 18 janvier 2012 à 21:26 +1000, Nick Coghlan a écrit :
> I'm also wholly in agreement with Ezio that using the
> same versioning scheme for both full releases and interim releases is
> thoroughly confusing for users 

It's a straight-forward way to track the feature support of a release.
How do you suggest all these "sys.version_info >= (3, 2)" - and the
corresponding documentation snippets a.k.a "versionadded" or
"versionchanged" tags - be spelt otherwise?

> for example, I consider Red Hat's
> completely separate branding and versioning for Fedora and RHEL a
> better model for end users

It's not only branding and versioning, is it? They're completely
different projects with different goals (and different commercial
support).

If you're suggesting we do only short-term releases and leave the
responsibility of long-term support to another project or entity, I'm
not against it, but it's far more radical than what we are proposing in
the PEP :-)

> Instead, if the proposal involves instituting a PEP 3003 style
> moratorium (i.e. stdlib changes only) for all interim releases, then
> we're essentially talking about splitting the versioning of the core
> language (and the CPython C API) and the standard library. If we're
> going to discuss that, we may as well go a bit further and just split
> development of the two out onto separate branches, with the current
> numbering scheme applying to full language version releases and
> switching to a date-based versioning scheme for the standard library
> (i.e. if 3.3 goes out in August as planned, then it would be "Python
> 3.3 with the 12.08 stdlib release").

Well, you're opposing the PEP on the basis that it's workforce-intensive
but you're proposing something much more workforce-intensive :-)

Splitting the stdlib:
- requires someone to do the splitting (highly non-trivial given the
interactions of some modules with interpreter details or low-level C
code)
- requires setting up separate resources (continuous integration with N
stdlib versions and M interpreter versions, for example)
- requires separate maintenance and releases for the stdlib (but with
non-trivial interaction with interpreter maintenance, since they will
affect each other and must be synchronized for Python to be usable at
all)
- requires more attention by users since there are now *two* release
schedules and independent version numbers to track

The former two are one-time costs, but the latter two are recurring
costs.

Therefore, splitting the stdlib is much more complicated and involved
than many people think; it's not just "move a few directories around and
be done".
And it's not even obvious it would have an actual benefit, since
developers of other implementations are busy doing just that (see Jeff
Hardy's message in this thread).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Stephen J. Turnbull]

Antoine Pitrou writes:

 > > Since testing is the bottleneck on what users consider to be
 > > "available for me", you cannot decrease the amount of testing (alpha,
 > > beta releases) by anywhere near the amount you're increasing
 > > frequency, or you're just producing "as is" snapshots.
 > 
 > The point is to *increase* the amount of testing by making features
 > available in stable releases on a more frequent basis. Not decrease
 > it.

We're talking about different kinds of testing.  You're talking about
(what old-school commercial software houses meant by "beta") testing
in a production or production prototype environment.  I'd love to see
more of that, too!  My claim is that I don't expect much uptake if you
don't do close to as many of what are called "alpha" and "beta" tests
on python-dev as are currently done.

 > Alphas and betas never produce much feedback, because people are
 > reluctant to install them for anything else than toying around. Python
 > is not emacs or Firefox, you don't use it in a vacuum
 > and therefore installing non-stable versions is dangerous.

Exactly my point, except that the PEP authors seem to think that we
can cut back on the number of alpha and beta prereleases and still
achieve the stability that such users expect from a Python release.  I
don't think that's right.  I expect that unless quite substantial
resources (far more than "proportional to 1/frequency") are devoted to
each non-LTS release, a large fraction of such users to avoid non-LTS
releases the way they avoid betas now.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Antoine Pitrou]

Le mercredi 18 janvier 2012 à 21:48 +0900, Stephen J. Turnbull a écrit :
> My claim is that I don't expect much uptake if you
> don't do close to as many of what are called "alpha" and "beta" tests
> on python-dev as are currently done.

You claim people won't use stable releases because of not enough alphas?
That sounds completely unrelated. I don't know of any users who would
bother about that.
(you can produce flimsy software with many alphas, too)

>  > Alphas and betas never produce much feedback, because people are
>  > reluctant to install them for anything else than toying around. Python
>  > is not emacs or Firefox, you don't use it in a vacuum
>  > and therefore installing non-stable versions is dangerous.
> 
> Exactly my point, except that the PEP authors seem to think that we
> can cut back on the number of alpha and beta prereleases and still
> achieve the stability that such users expect from a Python release.  I
> don't think that's right.

Sure, and we think it is :)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Nick Coghlan]

On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou  wrote:
> Splitting the stdlib:
> - requires someone to do the splitting (highly non-trivial given the
> interactions of some modules with interpreter details or low-level C
> code)
> - requires setting up separate resources (continuous integration with N
> stdlib versions and M interpreter versions, for example)
> - requires separate maintenance and releases for the stdlib (but with
> non-trivial interaction with interpreter maintenance, since they will
> affect each other and must be synchronized for Python to be usable at
> all)
> - requires more attention by users since there are now *two* release
> schedules and independent version numbers to track

Did you read what I actually proposed? I specifically *didn't* propose
separate stdlib releases (for all the reasons you point out), only
separate date based stdlib *versioning*. Distribution of the CPython
interpreter + stdlib would remain monolithic, as it is today. Any
given stdlib release would only be supported for the most recent
language release. The only difference is that between language
releases, where we currently only release maintenance builds, we'd
*also* release a second version of each maintenance build with an
updated standard library, along with an alpha release of the next
language version (with the last part being entirely optional, but I
figured I may as well make the suggestion since I like the idea to
encourage getting syntax updates and the like out for earlier
experimentation).

When you initially pitched the proposal via email, you didn't include
the "language moratarium applies to interim releases" idea. That one
additional suggestion makes the whole concept *much* more appealing to
me, but I only like it on the condition that we decouple the stdlib
versioning from the language definition versioning (even though I
recommend we only officially support very specific combinations of the
two). My suggestion is really just a concrete proposal for
implementing Ezio's idea of only bumping the Python version for
releases with long term support, and using some other mechanism to
distinguish the interim releases.

So, assuming a 2 year LTS cycle, the released versions up to February
2015 with my suggestion would end up being:

>From the default branch:
Python 3.3.0 + stdlib 12.08.0  (~August 2012)
Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
Python 3.4.0a2 + stdlib 14.08.0a2 (~August 2013)
Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)
Python 3.4.0a4 + stdlib 14.08.0a4  (~2014)
Python 3.4.0b1 + stdlib 14.08.0b1  (~2014)
Python 3.4.0b2 + stdlib 14.08.0b2  (~2014)
Python 3.4.0c1 + stdlib 14.08.0c1  (~2014)
Python 3.4.0 + stdlib 14.08  (~August 2014)
Python 3.5.0a1 + stdlib 16.08.0a1  (~February 2015)

>From the 3.3 maintenance branch (these are maintenance updates to the
"LTS" release):
Python 3.3.1 + stdlib 12.08.1  (~February 2013)
Python 3.3.2 + stdlib 12.08.2  (~August 2013)
Python 3.3.3 + stdlib 12.08.3  (~February 2014)
Python 3.3.4 + stdlib 12.08.4  (~August 2014) (and 3.3 branch enters
security patch only mode)

>From the 3.4 maintenance branch (these are maintenance updates to the
"LTS" release):
Python 3.4.1 + stdlib 14.08.1  (~February 2015)

>From the stdlib feature development branch (these are the new interim
releases with standard library updates only as proposed by PEP 407):
Python 3.3.1 + stdlib 13.02.0  (~February 2013)
Python 3.3.2 + stdlib 13.08.0  (~August 2013)
Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path
from here is to make the jump to 3.4.0)
-- 3.4.0 + 12.08.0 is released from default branch --
Python 3.4.1 + stdlib 15.02.0  (~February 2015)

If we have to make "brown paper bag" releases for the maintenance or
stdlib branches then the micro versions get bumped - the date based
version of the standard library versions relates to when that
particular *API* was realised, not when bugs were last fixed in it. If
a target release date slips, then the stdlib version would be
increased accordingly (cf. Ubuntu 6.06).

Yes, we'd have an extra set of active buildbots to handle the stdlib
branch, but a) that's no harder than creating the buildbots for a new
maintenance branch and b) the interim release proposal will need to
separate language level changes from stdlib level changes *anyway*.

As far as how sys.version checks would be updated, I would propose a
simple API addition to track the new date-based standard lib
versioning: sys.stdlib_version. People could choose to just depend on
a specific Python version (implicitly depending on the stdlib version
that was originally shipped with that version of CPython), or they may
instead decide to depend on a specific stdlib version (implicitly
depending on the first Python version that was shipped with that
stdlib).

The reason I like this scheme is that it allows us (and users) to
precisely track the things that can vary at the two different rates.
At least the following would still be governed by 

Re: [Python-Dev] [Python-checkins] Daily reference leaks (12de1ad1cee8): sum=6024

[Nick Coghlan]

On Wed, Jan 18, 2012 at 2:31 PM,   wrote:
> results for 12de1ad1cee8 on branch "default"
> 
>
> test_capi leaked [2008, 2008, 2008] references, sum=6024

Yikes, you weren't kidding about that new subinterpreter code
execution test upsetting the refleak detection...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Stephen J. Turnbull]

Antoine Pitrou writes:

 > You claim people won't use stable releases because of not enough
 > alphas?  That sounds completely unrelated.

Surely testing is related to user perceptions of stability.  More
testing helps reduce bugs in released software, which improves user
perception of stability, encouraging them to use the software in
production.  Less testing, then, will have the opposite effect.  But
you understand that theory, I'm sure.  So what do you mean to say?

 > (you can produce flimsy software with many alphas, too)

The problem is the converse: can you produce Python-release-quality
software with much less pre-release testing than current feature
releases get?

 > Sure, and we think it is [possible to do that] :)

Given the relative risk of rejecting PEP 407 and me being wrong (the
status quo really isn't all that bad AFAICS), vs. accepting PEP 407
and you being wrong, I don't find a smiley very convincing.  In fact,
I don't find the PEP itself convincing -- and I'm not the only one.

We'll see what Barry and Georg have to say.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Antoine Pitrou]

Le jeudi 19 janvier 2012 à 00:25 +0900, Stephen J. Turnbull a écrit :
> 
>  > You claim people won't use stable releases because of not enough
>  > alphas?  That sounds completely unrelated.
> 
> Surely testing is related to user perceptions of stability.  More
> testing helps reduce bugs in released software, which improves user
> perception of stability, encouraging them to use the software in
> production.

I have asked a practical question, a theoretical answer isn't exactly
what I was waiting for.

>  > Sure, and we think it is [possible to do that] :)
> 
> Given the relative risk of rejecting PEP 407 and me being wrong (the
> status quo really isn't all that bad AFAICS), vs. accepting PEP 407
> and you being wrong, I don't find a smiley very convincing.

I don't care to convince *you*, since you are not involved in Python
development and release management (you haven't ever been a contributor
AFAIK). Unless you produce practical arguments, saying "I don't think
you can do it" is plain FUD and certainly not worth answering to.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Senthil Kumaran]

On Wed, Jan 18, 2012 at 09:26:19PM +1000, Nick Coghlan wrote:
> My original suggestion to Antoine and Georg for 3.4 was that we simply
> propose to Larry Hastings (the 3.4 RM) that we spread out the release
> cycle, releasing the first alpha after ~6 months, the second after
> about ~12, then rolling into the regular release cycle of a final
> alpha, some beta releases, one or two release candidates and then the
> actual release. However, I'm sympathetic to Antoine's point that early
> alphas aren't likely to be at all interesting to folks that would like
> a fully supported stdlib update to put into production and no longer
> think that suggestion makes much sense on its own.

This looks like a 'good bridge' of suggestion between rapid releases
and stable releases. 

What would be purpose of alpha release. Would we encourage people to
use it or test it? Which the rapid relase cycle, the encouragement is
to use rather than test.

-- 
Senthil
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Antoine Pitrou]

On Thu, 19 Jan 2012 01:06:07 +1000
Nick Coghlan  wrote:
> On Wed, Jan 18, 2012 at 2:31 PM,   wrote:
> > results for 12de1ad1cee8 on branch "default"
> > 
> >
> > test_capi leaked [2008, 2008, 2008] references, sum=6024
> 
> Yikes, you weren't kidding about that new subinterpreter code
> execution test upsetting the refleak detection...

Well, these are real leaks, but I expect them to be quite difficult to
track (I've found a couple of them), because they can be scattered
around in C module initialization routines and the like. I suggest we
skip this test on refleak runs.

cheers

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hashing proposal: change only string-only dicts

[PJ Eby]

On Tue, Jan 17, 2012 at 7:58 PM, "Martin v. Löwis" wrote:

> Am 17.01.2012 22:26, schrieb Antoine Pitrou:
> > Only 2 bits are used in ob_sstate, meaning 30 are left. These 30 bits
> > could cache a "hash perturbation" computed from the string and the
> > random bits:
> >
> > - hash() would use ob_shash
> > - dict_lookup() would use ((ob_shash * 103) ^ (ob_sstate & ~3))
> >
> > This way, you cache almost all computations, adding only a computation
> > and a couple logical ops when looking up a string in a dict.
>
> That's a good idea. For Unicode, it might be best to add another slot
> into the object, even though this increases the object size.
>

Wouldn't that break the ABI in 2.x?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Brett Cannon]

On Wed, Jan 18, 2012 at 10:56, Antoine Pitrou  wrote:

> On Thu, 19 Jan 2012 01:06:07 +1000
> Nick Coghlan  wrote:
> > On Wed, Jan 18, 2012 at 2:31 PM,   wrote:
> > > results for 12de1ad1cee8 on branch "default"
> > > 
> > >
> > > test_capi leaked [2008, 2008, 2008] references, sum=6024
> >
> > Yikes, you weren't kidding about that new subinterpreter code
> > execution test upsetting the refleak detection...
>
> Well, these are real leaks, but I expect them to be quite difficult to
> track (I've found a couple of them), because they can be scattered
> around in C module initialization routines and the like. I suggest we
> skip this test on refleak runs.
>

Do we have any general strategy to help make it more fine-grained to detect
where the leak might be coming from? We could then maybe try to get some
people pound on this at the PyCon sprints. Otherwise I'm reluctant to skip
it since they are legitimate leaks that should be get fixed.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Antoine Pitrou]

On Wed, 18 Jan 2012 11:14:50 -0500
Brett Cannon  wrote:

> On Wed, Jan 18, 2012 at 10:56, Antoine Pitrou  wrote:
> 
> > On Thu, 19 Jan 2012 01:06:07 +1000
> > Nick Coghlan  wrote:
> > > On Wed, Jan 18, 2012 at 2:31 PM,   wrote:
> > > > results for 12de1ad1cee8 on branch "default"
> > > > 
> > > >
> > > > test_capi leaked [2008, 2008, 2008] references, sum=6024
> > >
> > > Yikes, you weren't kidding about that new subinterpreter code
> > > execution test upsetting the refleak detection...
> >
> > Well, these are real leaks, but I expect them to be quite difficult to
> > track (I've found a couple of them), because they can be scattered
> > around in C module initialization routines and the like. I suggest we
> > skip this test on refleak runs.
> >
> 
> Do we have any general strategy to help make it more fine-grained to detect
> where the leak might be coming from?

Unfortunately not. I've tried to track down the remaining leaks (*) by
using gc.get_objects(), but apart from a couple of false positives
(dead weakrefs lingering in some tp_subclasses slots until the next
subclasses take their place ;-)), most refleaks seem to be either on
long-lived objects (meaning the leaks are not severe) or on
non-gc-tracked objects.

(*)

$ ./python -m test -R 3:2 test_capi
[1/1] test_capi
beginning 5 repetitions
12345
.
test_capi leaked [152, 152] references, sum=304


> We could then maybe try to get some
> people pound on this at the PyCon sprints. Otherwise I'm reluctant to skip
> it since they are legitimate leaks that should be get fixed.

Well it's the old well-known issue with pseudo-"permanent" references
not being appropriately managed/cleaned up. Which only shows when
calling Py_Initialize/Py_Finalize multiple times, or using
sub-interpreters.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Brett Cannon]

On Wed, Jan 18, 2012 at 11:27, Antoine Pitrou  wrote:

> On Wed, 18 Jan 2012 11:14:50 -0500
> Brett Cannon  wrote:
>
> > On Wed, Jan 18, 2012 at 10:56, Antoine Pitrou 
> wrote:
> >
> > > On Thu, 19 Jan 2012 01:06:07 +1000
> > > Nick Coghlan  wrote:
> > > > On Wed, Jan 18, 2012 at 2:31 PM,   wrote:
> > > > > results for 12de1ad1cee8 on branch "default"
> > > > > 
> > > > >
> > > > > test_capi leaked [2008, 2008, 2008] references, sum=6024
> > > >
> > > > Yikes, you weren't kidding about that new subinterpreter code
> > > > execution test upsetting the refleak detection...
> > >
> > > Well, these are real leaks, but I expect them to be quite difficult to
> > > track (I've found a couple of them), because they can be scattered
> > > around in C module initialization routines and the like. I suggest we
> > > skip this test on refleak runs.
> > >
> >
> > Do we have any general strategy to help make it more fine-grained to
> detect
> > where the leak might be coming from?
>
> Unfortunately not. I've tried to track down the remaining leaks (*) by
> using gc.get_objects(), but apart from a couple of false positives
> (dead weakrefs lingering in some tp_subclasses slots until the next
> subclasses take their place ;-)), most refleaks seem to be either on
> long-lived objects (meaning the leaks are not severe) or on
> non-gc-tracked objects.
>
> (*)
>
> $ ./python -m test -R 3:2 test_capi
> [1/1] test_capi
> beginning 5 repetitions
> 12345
> .
> test_capi leaked [152, 152] references, sum=304
>
>
> > We could then maybe try to get some
> > people pound on this at the PyCon sprints. Otherwise I'm reluctant to
> skip
> > it since they are legitimate leaks that should be get fixed.
>
> Well it's the old well-known issue with pseudo-"permanent" references
> not being appropriately managed/cleaned up. Which only shows when
> calling Py_Initialize/Py_Finalize multiple times, or using
> sub-interpreters.
>

Could we tweak the report to somehow ignore the permanent refcounts for
just this test? If not then we might as well leave it out since that number
will never hit 0.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Antoine Pitrou]

On Wed, 18 Jan 2012 11:39:42 -0500
Brett Cannon  wrote:
> >
> > > We could then maybe try to get some
> > > people pound on this at the PyCon sprints. Otherwise I'm reluctant to
> > skip
> > > it since they are legitimate leaks that should be get fixed.
> >
> > Well it's the old well-known issue with pseudo-"permanent" references
> > not being appropriately managed/cleaned up. Which only shows when
> > calling Py_Initialize/Py_Finalize multiple times, or using
> > sub-interpreters.
> >
> 
> Could we tweak the report to somehow ignore the permanent refcounts for
> just this test? If not then we might as well leave it out since that number
> will never hit 0.

I can't think of any way to specifically ignore them (if we knew where
they are we could just fix the refleaks :-)).

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Stephen J. Turnbull]

Nick Coghlan writes:

 > >From the stdlib feature development branch (these are the new interim
 > releases with standard library updates only as proposed by PEP 407):
 > Python 3.3.1 + stdlib 13.02.0  (~February 2013)
 > Python 3.3.2 + stdlib 13.08.0  (~August 2013)
 > Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path
 > from here is to make the jump to 3.4.0)
 > -- 3.4.0 + 12.08.0 is released from default branch --

Typo? -> 3.4.0 + 14.08.0, right?

 > Python 3.4.1 + stdlib 15.02.0  (~February 2015)

It seems to me there could be considerable divergence between the
stdlib code in

 > the default branch:
 > Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
 > Python 3.4.0a2 + stdlib 14.08.0a2  (~August 2013)
 > Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)

and

 > the stdlib feature development branch
 > Python 3.3.1 + stdlib 13.02.0  (~February 2013)
 > Python 3.3.2 + stdlib 13.08.0  (~August 2013)
 > Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path

because 14.08.0a* will be targeting 3.4, and *should* use new language
constructs and APIs where they are appropriate, while 13.02.0 ...
14.02.0 will be targeting the 3.3 API, and mustn't use them.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Dirkjan Ochtman]

On Tuesday, January 17, 2012, Antoine Pitrou  wrote:
> We would like to propose the following PEP to change (C)Python's release
> cycle. Discussion is welcome, especially from people involved in the
> release process, and maintainers from third-party distributions of
> Python.

As a Gentoo packager, this would mean much more work for us, unless all the
non-LTS releases promised to be backwards compatible. I.e. the hard part
for us is managing all the incompatibilities in other packages,
compatibility with Python.

As a user of Python, I would rather dislike the change from 18 to 24 months
for LTS release cycles. And the limiting factor for my use of Python
features is largely old Python versions still in use, not the availability
of newer features in the newest Python. So I'm much more interested in
finding ways of improving 2.7/3.2 uptake than adding more feature releases.
I also think that it would be sensible to wait with something like this
process change until the 3.x adoption curve is much further along.

Cheers,

Dirkjan




> Regards
>
> Antoine.
>
>
> PEP: 407
> Title: New release cycle and introducing long-term support versions
> Version: $Revision$
> Last-Modified: $Date$
> Author: Antoine Pitrou ,
>Georg Brandl ,
>Barry Warsaw 
> Status: Draft
> Type: Process
> Content-Type: text/x-rst
> Created: 2012-01-12
> Post-History:
> Resolution: TBD
>
>
> Abstract
> 
>
> Finding a release cycle for an open-source project is a delicate
> exercise in managing mutually contradicting constraints: developer
> manpower, availability of release management volunteers, ease of
> maintenance for users and third-party packagers, quick availability of
> new features (and behavioural changes), availability of bug fixes
> without pulling in new features or behavioural changes.
>
> The current release cycle errs on the conservative side.  It is
> adequate for people who value stability over reactivity.  This PEP is
> an attempt to keep the stability that has become a Python trademark,
> while offering a more fluid release of features, by introducing the
> notion of long-term support versions.
>
>
> Scope
> =
>
> This PEP doesn't try to change the maintenance period or release
> scheme for the 2.7 branch.  Only 3.x versions are considered.
>
>
> Proposal
> 
>
> Under the proposed scheme, there would be two kinds of feature
> versions (sometimes dubbed "minor versions", for example 3.2 or 3.3):
> normal feature versions and long-term support (LTS) versions.
>
> Normal feature versions would get either zero or at most one bugfix
> release; the latter only if needed to fix critical issues.  Security
> fix handling for these branches needs to be decided.
>
> LTS versions would get regular bugfix releases until the next LTS
> version is out.  They then would go into security fixes mode, up to a
> termination date at the release manager's discretion.
>
> Periodicity
> ---
>
> A new feature version would be released every X months.  We
> tentatively propose X = 6 months.
>
> LTS versions would be one out of N feature versions.  We tentatively
> propose N = 4.
>
> With these figures, a new LTS version would be out every 24 months,
> and remain supported until the next LTS version 24 months later.  This
> is mildly similar to today's 18 months bugfix cycle for every feature
> version.
>
> Pre-release versions
> 
>
> More frequent feature releases imply a smaller number of disruptive
> changes per release.  Therefore, the number of pre-release builds
> (alphas and betas) can be brought down considerably.  Two alpha builds
> and a single beta build would probably be enough in the regular case.
> The number of release candidates depends, as usual, on the number of
> last-minute fixes before final release.
>
>
> Effects
> ===
>
> Effect on development cycle
> ---
>
> More feature releases might mean more stress on the development and
> release management teams.  This is quantitatively alleviated by the
> smaller number of pre-release versions; and qualitatively by the
> lesser amount of disruptive changes (meaning less potential for
> breakage).  The shorter feature freeze period (after the first beta
> build until the final release) is easier to accept.  The rush for
> adding features just before feature freeze should also be much
> smaller.
>
> Effect on bugfix cycle
> --
>
> The effect on fixing bugs should be minimal with the proposed figures.
> The same number of branches would be simultaneously open for regular
> maintenance (two until 2.x is terminated, then one).
>
> Effect on workflow
> --
>
> The workflow for new features would be the same: developers would only
> commit them on the ``default`` branch.
>
> The workflow for bug fixes would be slightly updated: developers would
> commit bug fixes to the current LTS branch (for example ``3.3``) and
> then merge them into ``default``.
>
_

Re: [Python-Dev] Hashing proposal: change only string-only dicts

[Martin v. Löwis]

Am 18.01.2012 17:01, schrieb PJ Eby:
> On Tue, Jan 17, 2012 at 7:58 PM, "Martin v. Löwis"  > wrote:
> 
> Am 17.01.2012 22:26, schrieb Antoine Pitrou:
> > Only 2 bits are used in ob_sstate, meaning 30 are left. These 30 bits
> > could cache a "hash perturbation" computed from the string and the
> > random bits:
> >
> > - hash() would use ob_shash
> > - dict_lookup() would use ((ob_shash * 103) ^ (ob_sstate & ~3))
> >
> > This way, you cache almost all computations, adding only a computation
> > and a couple logical ops when looking up a string in a dict.
> 
> That's a good idea. For Unicode, it might be best to add another slot
> into the object, even though this increases the object size.
> 
> 
> Wouldn't that break the ABI in 2.x?

I was thinking about adding the field at the end, so I thought it
shouldn't. However, if somebody inherits from PyUnicodeObject, it still
might - so my new proposal is to add the extra hash into the str block,
either at str[-1], or after the terminating 0. This would cause an
average increase of four bytes of the storage (0 bytes in 50% of the
cases, 8 bytes because of padding in the other 50%).

What do you think?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hashing proposal: change only string-only dicts

[Martin v. Löwis]

Am 18.01.2012 13:30, schrieb Barry Warsaw:
> On Jan 18, 2012, at 08:19 AM, Martin v. Löwis wrote:
> 
>> My concern is not about breaking doctests: this proposal will also break
>> them. My concern is about applications that assume that hash(s) is
>> stable across runs, and we do have reports that it will break
>> applications.
> 
> I am a proponent of doctests, and thus use them heavily.  I can tell you that
> the issue of dict hashing (non-)order has been well known for *years* and I
> have convenience functions in my own doctests to sort and print dict
> elements.

Indeed. So that breakage may actually be less than people expect.

As for cases that still rely on dict order: none of the proposed
solutions preserve full compatibility in dict order. The only solution
(not actually proposed so far) is to add an AVL tree into the hash
table, to track keys that collide on hash values (rather than hash
slots). Such a tree would be only used if there is an actual collision,
which, in practical dict usage, never occurs.

I've been seriously considering implementing a balanced tree inside
the dict (again for string-only dicts, as ordering can't be guaranteed
otherwise). However, this would be a lot of code for a security fix.
It *would* solve the issue for good, though.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Brett Cannon]

On Wed, Jan 18, 2012 at 09:08, Nick Coghlan  wrote:

> On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou 
> wrote:
> > Splitting the stdlib:
> > - requires someone to do the splitting (highly non-trivial given the
> > interactions of some modules with interpreter details or low-level C
> > code)
> > - requires setting up separate resources (continuous integration with N
> > stdlib versions and M interpreter versions, for example)
> > - requires separate maintenance and releases for the stdlib (but with
> > non-trivial interaction with interpreter maintenance, since they will
> > affect each other and must be synchronized for Python to be usable at
> > all)
> > - requires more attention by users since there are now *two* release
> > schedules and independent version numbers to track
>
> Did you read what I actually proposed? I specifically *didn't* propose
> separate stdlib releases (for all the reasons you point out), only
> separate date based stdlib *versioning*. Distribution of the CPython
> interpreter + stdlib would remain monolithic, as it is today. Any
> given stdlib release would only be supported for the most recent
> language release. The only difference is that between language
> releases, where we currently only release maintenance builds, we'd
> *also* release a second version of each maintenance build with an
> updated standard library, along with an alpha release of the next
> language version (with the last part being entirely optional, but I
> figured I may as well make the suggestion since I like the idea to
> encourage getting syntax updates and the like out for earlier
> experimentation).


> When you initially pitched the proposal via email, you didn't include
> the "language moratarium applies to interim releases" idea. That one
> additional suggestion makes the whole concept *much* more appealing to
> me, but I only like it on the condition that we decouple the stdlib
> versioning from the language definition versioning (even though I
> recommend we only officially support very specific combinations of the
> two). My suggestion is really just a concrete proposal for
> implementing Ezio's idea of only bumping the Python version for
> releases with long term support, and using some other mechanism to
> distinguish the interim releases.
>

IOW we would have a language moratorium every 2 years (i.e. between LTS
releases) while switching to a 6 month release cycle for language/VM
bugfixes and full stdlib releases? I would support that as it has several
benefits from several angles.

>From a VM perspective, it gives other VMs 2 years to catch up to the next
release instead of 18 months; not a big switch, but still better than
shortening it.

It also makes disruptive language changes less frequent so people have more
time to catch up, update books/docs, etc. We can also let them bake longer
and we all get more experience with them.

Doing a release every 6 months that includes updates to the stdlib and
bugfixes to the language/VM also benefits other VMs by getting
compatibility fixes in faster. All of the other VM maintainers have told me
that keeping the stdlib non-CPython compliant is the biggest hurdle. This
kind of switch means they could release a VM that supports a release 6
months or a year after a language change release (e.g. 1 to 2 releases in)
so as to get changes in faster and lower the need to keep their own fork.

It should also increase the chances of external developers of projects
being willing to become core developers and contributing their project to
Python. If they get to keep a 6 month release cycle we could consider
pulling in project like httplib2 and others that have resisted inclusion in
the stdlib because painfully long (for them) wait between releases.


>
> So, assuming a 2 year LTS cycle, the released versions up to February
> 2015 with my suggestion would end up being:
>
> >From the default branch:
> Python 3.3.0 + stdlib 12.08.0  (~August 2012)
> Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
> Python 3.4.0a2 + stdlib 14.08.0a2 (~August 2013)
> Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)
> Python 3.4.0a4 + stdlib 14.08.0a4  (~2014)
> Python 3.4.0b1 + stdlib 14.08.0b1  (~2014)
> Python 3.4.0b2 + stdlib 14.08.0b2  (~2014)
> Python 3.4.0c1 + stdlib 14.08.0c1  (~2014)
> Python 3.4.0 + stdlib 14.08  (~August 2014)
> Python 3.5.0a1 + stdlib 16.08.0a1  (~February 2015)
>
> >From the 3.3 maintenance branch (these are maintenance updates to the
> "LTS" release):
> Python 3.3.1 + stdlib 12.08.1  (~February 2013)
> Python 3.3.2 + stdlib 12.08.2  (~August 2013)
> Python 3.3.3 + stdlib 12.08.3  (~February 2014)
> Python 3.3.4 + stdlib 12.08.4  (~August 2014) (and 3.3 branch enters
> security patch only mode)
>
> >From the 3.4 maintenance branch (these are maintenance updates to the
> "LTS" release):
> Python 3.4.1 + stdlib 14.08.1  (~February 2015)
>
> >From the stdlib feature development branch (these are the new interim
> releases with standard library updat

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Antoine Pitrou]

Hello Dirkjan,

On Wed, 18 Jan 2012 18:32:22 +0100
Dirkjan Ochtman  wrote:
> On Tuesday, January 17, 2012, Antoine Pitrou  wrote:
> > We would like to propose the following PEP to change (C)Python's release
> > cycle. Discussion is welcome, especially from people involved in the
> > release process, and maintainers from third-party distributions of
> > Python.
> 
> As a Gentoo packager, this would mean much more work for us, unless all the
> non-LTS releases promised to be backwards compatible. I.e. the hard part
> for us is managing all the incompatibilities in other packages,
> compatibility with Python.

It might need to be spelt clearly in the PEP, but one of my assumptions
is that packagers choose on what release series they want to
synchronize. So packagers can synchronize on the LTS releases if it's
more practical for them, or if it maps better to their own release
model (e.g. Debian).

Do you think that's a valid answer to Gentoo's concerns?

> So I'm much more interested in
> finding ways of improving 2.7/3.2 uptake than adding more feature releases.

That would be nice as well, but I think it's orthogonal to the PEP.
Besides, I'm afraid there's not much we (python-dev) can do about it.
Some vendors (Debian, Redhat) will always lag behind the bleeding-edge
feature releases.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Georg Brandl]

Am 18.01.2012 16:25, schrieb Stephen J. Turnbull:
> Antoine Pitrou writes:
> 
>  > You claim people won't use stable releases because of not enough
>  > alphas?  That sounds completely unrelated.
> 
> Surely testing is related to user perceptions of stability.  More
> testing helps reduce bugs in released software, which improves user
> perception of stability, encouraging them to use the software in
> production.  Less testing, then, will have the opposite effect.  But
> you understand that theory, I'm sure.  So what do you mean to say?
> 
>  > (you can produce flimsy software with many alphas, too)
> 
> The problem is the converse: can you produce Python-release-quality
> software with much less pre-release testing than current feature
> releases get?
>
>  > Sure, and we think it is [possible to do that] :)
> 
> Given the relative risk of rejecting PEP 407 and me being wrong (the
> status quo really isn't all that bad AFAICS), vs. accepting PEP 407
> and you being wrong, I don't find a smiley very convincing.

"The status quo really isn't all that bad" applies to any PEP.  Also,
compared to most PEPs, it is quite easy to revert to the previous
state of things if they don't work out as wanted.

> In fact,
> I don't find the PEP itself convincing -- and I'm not the only one.

That is noted.  And I think Antoine was a little harsh earlier; of
course we also need to convince users that the new cycle is
advantageous and not detrimental.

> We'll see what Barry and Georg have to say.

Two things:

a) The release manager's job is not as bad as you might believe.  We
have an incredibly helpful and active core of developers which means
that the RM job is more or less "reduced" to pronouncing on changes
during the rc phase, and actually producing the releases.

b) I did not have the impression (maybe someone can underline that
with tracker stats?) that there were a lot more bug reports than
usual during the alpha and early beta stages of Python 3.2.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Georg Brandl]

Am 18.01.2012 18:56, schrieb Brett Cannon:

> IOW we would have a language moratorium every 2 years (i.e. between LTS
> releases) while switching to a 6 month release cycle for language/VM bugfixes
> and full stdlib releases?

That is certainly a possibility (it's listed as an open issue in the PEP).

> I would support that as it has several benefits from
> several angles.
> 
> From a VM perspective, it gives other VMs 2 years to catch up to the next
> release instead of 18 months; not a big switch, but still better than 
> shortening it.
> 
> It also makes disruptive language changes less frequent so people have more 
> time
> to catch up, update books/docs, etc. We can also let them bake longer and we 
> all
> get more experience with them.

Yes.  In the end, the moratorium really was a good idea, and this would be
carrying on the spirit.

> Doing a release every 6 months that includes updates to the stdlib and 
> bugfixes
> to the language/VM also benefits other VMs by getting compatibility fixes in
> faster. All of the other VM maintainers have told me that keeping the stdlib
> non-CPython compliant is the biggest hurdle. This kind of switch means they
> could release a VM that supports a release 6 months or a year after a language
> change release (e.g. 1 to 2 releases in) so as to get changes in faster and
> lower the need to keep their own fork.
> 
> It should also increase the chances of external developers of projects being
> willing to become core developers and contributing their project to Python. If
> they get to keep a 6 month release cycle we could consider pulling in project
> like httplib2 and others that have resisted inclusion in the stdlib because
> painfully long (for them) wait between releases.

Exactly!

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hashing proposal: change only string-only dicts

[Glenn Linderman]

On 1/18/2012 9:52 AM, "Martin v. Löwis" wrote:

I've been seriously considering implementing a balanced tree inside
the dict (again for string-only dicts, as ordering can't be guaranteed
otherwise). However, this would be a lot of code for a security fix.
It*would*  solve the issue for good, though.


To handle keys containing non-orderable keys along with strings, which 
are equally vulnerable to string-only keys, especially if the non-string 
components can have fixed values during an attack, you could simply use 
their hash value as an orderable proxy for the non-orderable key components.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Antoine Pitrou]

Well, they should be fixed now :-)

Regards

Antoine.


On Wed, 18 Jan 2012 17:42:15 +0100
Antoine Pitrou  wrote:
> On Wed, 18 Jan 2012 11:39:42 -0500
> Brett Cannon  wrote:
> > >
> > > > We could then maybe try to get some
> > > > people pound on this at the PyCon sprints. Otherwise I'm reluctant to
> > > skip
> > > > it since they are legitimate leaks that should be get fixed.
> > >
> > > Well it's the old well-known issue with pseudo-"permanent" references
> > > not being appropriately managed/cleaned up. Which only shows when
> > > calling Py_Initialize/Py_Finalize multiple times, or using
> > > sub-interpreters.
> > >
> > 
> > Could we tweak the report to somehow ignore the permanent refcounts for
> > just this test? If not then we might as well leave it out since that number
> > will never hit 0.
> 
> I can't think of any way to specifically ignore them (if we knew where
> they are we could just fix the refleaks :-)).
> 
> Regards
> 
> Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[fwierzbi...@gmail.com]

On Wed, Jan 18, 2012 at 9:56 AM, Brett Cannon  wrote:

> Doing a release every 6 months that includes updates to the stdlib and
> bugfixes to the language/VM also benefits other VMs by getting compatibility
> fixes in faster. All of the other VM maintainers have told me that keeping
> the stdlib non-CPython compliant is the biggest hurdle. This kind of switch
> means they could release a VM that supports a release 6 months or a year
> after a language change release (e.g. 1 to 2 releases in) so as to get
> changes in faster and lower the need to keep their own fork.
As one of the other VM maintainers I agree with everything Brett has
said here. The proposal sounds very good to me from that perspective.

-Frank
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Steven D'Aprano]

Antoine Pitrou wrote:

Le jeudi 19 janvier 2012 à 00:25 +0900, Stephen J. Turnbull a écrit :

 > You claim people won't use stable releases because of not enough
 > alphas?  That sounds completely unrelated.

Surely testing is related to user perceptions of stability.  More
testing helps reduce bugs in released software, which improves user
perception of stability, encouraging them to use the software in
production.


I have asked a practical question, a theoretical answer isn't exactly
what I was waiting for.

[...]

I don't care to convince *you*, since you are not involved in Python
development and release management (you haven't ever been a contributor
AFAIK). Unless you produce practical arguments, saying "I don't think
you can do it" is plain FUD and certainly not worth answering to.


Pardon me, but people like Stephen Turnbull are *users* of Python, exactly the 
sort of people you DO have to convince that moving to an accelerated or more 
complex release process will result in a better product. The risk is that you 
will lose users, or fragment the user base even more than it is now with 2.x 
vs 3.x.


Quite frankly, I like the simplicity and speed of the current release cycle. 
All this talk about separate LTS releases and parallel language releases and 
library releases makes my head spin. I fear the day that people asking 
questions on the tutor or python-list mailing lists will have to say (e.g.) 
"I'm using Python 3.4.1 and standard library 1.2.7" in order to specify the 
version they're using.


I fear change, because the current system works well and for every way to make 
it better there are a thousand ways to make it worse. Dismissing fears like 
this as FUD doesn't do anyone any favours.


One on-going complaint is that Python-Dev doesn't have the manpower or time to 
do everything that needs to be done. Bugs languish for months or years because 
nobody has the time to look at it. Will going to a more rapid release cycle 
give people more time, or just increase their workload? You're hoping that a 
more rapid release cycle will attract more developers, and there is a chance 
that you could be right; but a more rapid release cycle WILL increase the 
total work load. So you're betting that this change will attract enough new 
developers that the work load per person will decrease even as the total work 
load increases. I don't think that's a safe bet.




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Steven D'Aprano]

Brett Cannon wrote:


And honestly, if we don't go with this I'm with Georg's comment in another
email of beginning to consider stripping the stdlib down to core libraries
to help stop with the bitrot (sorry, Paul). If we can't attract new
replacements for modules we can't ditch because of backwards compatibility
I start to wonder if I should even care about improving the stdlib outside
of core code required to make Python simply function.



Do we have any evidence of this alleged bitrot? I spend a lot of time on the 
comp.lang.python newsgroup and I see no evidence that people using Python 
believe the standard library is rotting from lack of attention.


I do see people having trouble with installing third party packages. I see 
that stripping back the standard library and forcing people to rely more on 
external libraries will hurt, rather than help, the experience they have with 
Python.




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Matt Joiner]

On Wed, Jan 18, 2012 at 6:55 PM, Georg Brandl  wrote:
> The main reason is changes in the library.  We have been getting complaints
> about the standard library bitrotting for years now, and one of the main
> reasons it's so hard to a) get decent code into the stdlib and b) keep it
> maintained is that the release cycles are so long.  It's a tough thing for
> contributors to accept that the feature you've just implemented will only
> be in a stable release in 16 months.
>
> If the stdlib does not get more reactive, it might just as well be cropped
> down to a bare core, because 3rd-party libraries do everything as well and
> do it before we do.  But you're right that if Python came without batteries,
> the current release cycle would be fine.

I think this is the real issue here. The batteries in Python are so
important because:
1) The stability and quality of 3rd party libraries is not guaranteed.
2) The mechanism used to obtain 3rd party libraries, is not popular or
considered reliable.

Much of the "bitrot" is that standard library modules have been
deprecated by third party ones that are of a much higher
functionality. Rather than importing these libraries, it needs to be
trivial to obtain them.

Putting some of these higher quality 3rd party modules into lock step
with Python is an unpopular move, and hampers their future growth.

From the top of my head, libraries such as LXML, argparse, and
requests are such popular libraries that shouldn't be baked in. In the
long term, it would be nice to see these kinds of libraries dropped
from the standard installation, and made available through the new
distribute package systems etc.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Writable __doc__

[Ethan Furman]

Is there a reason why normal classes can't have their __doc__ strings 
rewritten?  Creating a do-nothing metaclass seems like overkill for such 
a simple operation.


Python 3.2 ... on win32
--> class Test():
...   __doc__ = 'am I permanent?'
...
--> Test.__doc__
'am I permanent?'
--> Test.__doc__ = 'yes'
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: attribute '__doc__' of 'type' objects is not writable
--> type(Test)


--> class Meta(type):
...   "only for exists to allow writable __doc__"
...
--> class Test(metaclass=Meta):
...   __doc__ = 'am I permanent?'
...
--> Test.__doc__
'am I permanent?'
--> Test.__doc__ = 'No!'
--> Test.__doc__
'No!'
--> type(Test)


Should I create a bug report?

~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Writable __doc__

[Benjamin Peterson]

2012/1/18 Ethan Furman :
> Is there a reason why normal classes can't have their __doc__ strings
> rewritten?  Creating a do-nothing metaclass seems like overkill for such a
> simple operation.
>
> Python 3.2 ... on win32
> --> class Test():
> ...   __doc__ = 'am I permanent?'
> ...
> --> Test.__doc__
> 'am I permanent?'
> --> Test.__doc__ = 'yes'
> Traceback (most recent call last):
>  File "", line 1, in 
> AttributeError: attribute '__doc__' of 'type' objects is not writable
> --> type(Test)
> 
>
> --> class Meta(type):
> ...   "only for exists to allow writable __doc__"
> ...
> --> class Test(metaclass=Meta):
> ...   __doc__ = 'am I permanent?'
> ...
> --> Test.__doc__
> 'am I permanent?'
> --> Test.__doc__ = 'No!'
> --> Test.__doc__
> 'No!'
> --> type(Test)
> 
>
> Should I create a bug report?

 $ ./python
Python 3.3.0a0 (default:095de2293f39, Jan 18 2012, 10:34:18)
[GCC 4.5.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Test:
... __doc__ = "time machine"
...
>>> Test.__doc__ = "strikes again"
>>> Test.__doc__
'strikes again'




-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Coroutines and PEP 380

[Matt Joiner]

PEP380 and Mark's coroutines could coexist, so I really don't "it's
too late" matters. Furthermore, PEP380 has utility in its own right
without considering its use for "explicit coroutines".

I would like to see these coroutines considered, but as someone else
mentioned, coroutines via PEP380 enhanced generators have some
interesting characteristics, from my experimentations they feel
monadic.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Nick Coghlan]

On Thu, Jan 19, 2012 at 7:31 AM, fwierzbi...@gmail.com
 wrote:
> On Wed, Jan 18, 2012 at 9:56 AM, Brett Cannon  wrote:
>
>> Doing a release every 6 months that includes updates to the stdlib and
>> bugfixes to the language/VM also benefits other VMs by getting compatibility
>> fixes in faster. All of the other VM maintainers have told me that keeping
>> the stdlib non-CPython compliant is the biggest hurdle. This kind of switch
>> means they could release a VM that supports a release 6 months or a year
>> after a language change release (e.g. 1 to 2 releases in) so as to get
>> changes in faster and lower the need to keep their own fork.
> As one of the other VM maintainers I agree with everything Brett has
> said here. The proposal sounds very good to me from that perspective.

Yes, with the addition of the idea of a PEP 3003 style language change
moratorium for interim releases, I've been converted from an initial
opponent of the idea (since we don't want to give the wider community
whiplash) to a supporter (since some parts of the community,
especially web service developers that deploy to tightly controlled
environments, aren't well served by the standard library's inability
to keep up with externally maintained standards and recommended
development practices).

It means PEP 407 can end up serving two goals:

1. Speeding up the rate of release for the standard library, allowing
enhanced features to be made available to end users sooner.
2. Slowing down (slightly) the rate of release of changes to the core
language and builtins, providing more time for those changes to filter
out through the wider Python ecosystem.

Agreeing with those goals in principle then leaves two key questions
to be addressed:

1. How would we have to update our development practices to make such
a dual versioning scheme feasible?
2. How can we best communicate a new approach to versioning without
unduly confusing developers that have built up certain expectations
about Python's release cycle over the past 20+ years?

For the first point, I think having two active development branches
(one for stdlib updates, one for language updates) will prove to be
absolutely essential. Otherwise all language updates would have to be
landed in the 6 month window between the last stdlib release for a
given language version and the next language release, which seems to
me a crazy way to go about things. As a consequence, I think we'd be
obliged to do something to avoid conflicts on Misc/NEWS (this could be
as simple as splitting it out into NEWS and NEWS_STDLIB, but if we're
restructuring those files anyway, we may also want to do something
about the annoying conflicts between maintenance releases and
development releases).

That then leaves the question of how to best communicate such a change
to the rest of the Python community. This is more a political and
educational question than it is a technical one. A few different
approaches have already been suggested:

1. I believe the PEP currently proposes just taking the "no more than
9" limit off the minor version of the language. Feature releases would
just come out every 6 months, with every 4th release flagged as a
language release. This could even be conveyed programmatically by
offering "sys.lang_version" and "sys.lang_version_info" attributes
that define the *language* version of a given release - 3.3, 3.4, 3.5
and 3.6 would all have something like sys.lang_version == '3.3', and
then in 3.7 (the next language release) it would be updated to say
sys.lang_version == '3.7'.

This approach would require that some policies (such as the
deprecation cycle) by updated to refer to changes in the language
version (sys.lang_version) rather than change in the stdlib version
(sys.version).

I don't like this scheme because it tries to use one number (the minor
version field) to cover two very different concepts (stdlib updates
and language updates). While technically feasible, this is
unnecessarily obscure and confusing for end users.

2. Brett's alternative proposal is that we switch to using the major
version for language releases and the minor version for stdlib
releases. We would then release 3.3, 3.4, 3.5 and 3.6 at 6 month
intervals, with 4.0 then being released in August 2014 as a new
language version.

Without taking recent history into acount, I actually like this scheme
- it fits well with traditional usage of major.minor.micro version
numbering. However, I'm not confident that the "python" name will
refer to Python 3 on a majority of systems by 2014 and accessing
Python 4.0 through the "python3" name would just be odd.

It also means we lose our ability to signal to the community when we
plan to make a backwards incompatible language release (making the
assumtion that we're never going to want to do that again would be
incredibly naive). On a related note, we'd also be setting ourselves
to have to explain to everyone that "no, no, Python 3 -> 4 is like
upgrading from Python 3.2 -> 3.3, not 2.7

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Nick Coghlan]

On Thu, Jan 19, 2012 at 10:19 AM, Steven D'Aprano  wrote:
> Brett Cannon wrote:
> Do we have any evidence of this alleged bitrot? I spend a lot of time on the
> comp.lang.python newsgroup and I see no evidence that people using Python
> believe the standard library is rotting from lack of attention.

IMO, it's a problem mainly with network (especially web) protocols and
file formats. It can take the stdlib a long time to catch up with
external developments due to the long release cycle, so people are
often forced to switch to third party libraries that better track the
latest versions of relevant standards (de facto or otherwise).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407 / splitting the stdlib

[Terry Reedy]

On 1/18/2012 8:06 PM, Nick Coghlan wrote:

On Thu, Jan 19, 2012 at 10:19 AM, Steven D'Aprano  wrote:

Do we have any evidence of this alleged bitrot? I spend a lot of time on the
comp.lang.python newsgroup and I see no evidence that people using Python
believe the standard library is rotting from lack of attention.


IMO, it's a problem mainly with network (especially web) protocols and
file formats. It can take the stdlib a long time to catch up with
external developments due to the long release cycle, so people are
often forced to switch to third party libraries that better track the
latest versions of relevant standards (de facto or otherwise).


Some of those modules are more that 2 years out of date and I guess what 
Brett is saying is that the people interested and able to update them 
will not do so in the stdlib because they want to be able to push out 
feature updates whenever they are needed and available and not be tied 
to a slow release schedule. Morever, since the external standards will 
continue to evolve for the foreseeable future, the need to track them 
more quickly will also continue.


We could relax the ban on new features in micro releases and designate 
such modules as volatile and let them get new features in each x.y.z 
release. In a sense, this would be less drastic than inventing a new 
type of release. Code can require an x.y.z release, as it must if it 
depends on a bug fix not in x.y.0.


I also like the idea of stretching out the alpha release cycle. I would 
like to see 3.3.0a1 appear along with 3.2.3 (in February?). If alpha 
releases are released with all buildbots green, they are as good, at 
least with respect to old features, as a corresponding bugfix release. 
All releases will become more dependable as test coverage improves. 
Again, this idea avoids inventing a new type of release with new release 
designations.


I think one reason people avoid alpha releases is that they so quickly 
become obsolete. If one sat for 3 to 6 months, it might get more 
attention. As for any alpha stigma, we should emphasize that alpha only 
mean not feature frozen.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Writable __doc__

[Eric Snow]

On Wed, Jan 18, 2012 at 5:01 PM, Ethan Furman  wrote:
> Is there a reason why normal classes can't have their __doc__ strings
> rewritten?  Creating a do-nothing metaclass seems like overkill for such a
> simple operation.
>
> Python 3.2 ... on win32
> --> class Test():
> ...   __doc__ = 'am I permanent?'
> ...
> --> Test.__doc__
> 'am I permanent?'
> --> Test.__doc__ = 'yes'
> Traceback (most recent call last):
>  File "", line 1, in 
> AttributeError: attribute '__doc__' of 'type' objects is not writable
> --> type(Test)
> 
>
> --> class Meta(type):
> ...   "only for exists to allow writable __doc__"
> ...
> --> class Test(metaclass=Meta):
> ...   __doc__ = 'am I permanent?'
> ...
> --> Test.__doc__
> 'am I permanent?'
> --> Test.__doc__ = 'No!'
> --> Test.__doc__
> 'No!'
> --> type(Test)
> 
>
> Should I create a bug report?

http://bugs.python.org/issue12773  :)

-eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Writable __doc__

[Sreenivas Reddy T]

this is happening on python 2.6 too.

Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Test(type):
...   __doc__=
  File "", line 2
__doc__=
   ^
SyntaxError: invalid syntax
>>> class Test(type):
...   __doc__='asasdas'
...
>>>
>>> Test.__doc__='sadfsdff'
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: attribute '__doc__' of 'type' objects is not writable
>>> type(Test)

>>>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Writable __doc__

[Noufal Ibrahim]

Sreenivas Reddy T  writes:

> this is happening on python 2.6 too.
>
> Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
> [GCC 4.4.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
 class Test(type):
> ...   __doc__=
>   File "", line 2
> __doc__=
>^
> SyntaxError: invalid syntax
 class Test(type):
> ...   __doc__='asasdas'
> ...



I don't get any syntax errors (Python2.7 and 2.6)

>>> class Test(object):
...   __doc__ = "Something"
... 
>>> 
>>> help(Test)

>>> class Test(type):
...   __doc__ = "something"
... 
>>> help(Test)

>>> Test.__doc__
'something'

 Test.__doc__='sadfsdff'
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: attribute '__doc__' of 'type' objects is not writable
 type(Test)
> 


The __name__, __bases__, __module__, __abstractmethods__, __dict__ and
__doc__ attributes have custom getters and setters in the type object
definition. __doc__ has only a getter. No setter and no deleter.

http://hg.python.org/cpython/file/0b5ce36a7a24/Objects/typeobject.c#l658

That is why you're seeing this. What's the question here?

[...]


-- 
~noufal
http://nibrahim.net.in

May I ask a question?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Writable __doc__

[Noufal Ibrahim]

Noufal Ibrahim  writes:

[...]

> That is why you're seeing this. What's the question here?

[...]

My apologies. I didn't read the whole thread. 

-- 
~noufal
http://nibrahim.net.in

Some bird populations soaring down -Headline of an article in Science News, 
page 126, February 20, 1993.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Stephen J. Turnbull]

Georg Brandl writes:

 > "The status quo really isn't all that bad" applies to any PEP.  Also,
 > compared to most PEPs, it is quite easy to revert to the previous
 > state of things if they don't work out as wanted.

That depends on how "doesn't work out" plays out.  If meeting the
schedule *and* producing a good release regularly is just more work
than expected, of course you're right.

If you stick to the schedule with insufficient resources, and lack of
testing produces a really bad release (or worse, a couple of sorta bad
releases in succession), reverting Python's reputation for stability
is going to be non-trivial.

 > a) The release manager's job is not as bad as you might believe.  We
 > have an incredibly helpful and active core of developers which means
 > that the RM job is more or less "reduced" to pronouncing on changes
 > during the rc phase, and actually producing the releases.

I've done release management and I've been watching Python do release
management since PEP 263; I'm well aware that Python has a truly
excellent process in place, and I regularly recommend studying to
friends interested in improving their own projects' processes.

But I've also (twice) been involved (as RM) in a major revision of RM
procedures, and both times it was a lot more work than anybody
expected.  Finally, the whole point of this exercise is to integrate a
lot more stdlib changes (including whole packages) than in the past on
a much shorter timeline, and to do it repeatedly.  "Every six months"
still sounds like a long time if you are a "leaf" project still
working on your changes on your own schedule and chafing at the bit
waiting to get them in to the core project's releases, but it's
actually quite short for the RM.

I'm not against this change (especially since, as Antoine so
graciously pointed out, I'm not going to be actually doing the work in
the foreseeable future), but I do advise that the effort required
seemed to be dramatically underestimated.

 > b) I did not have the impression (maybe someone can underline that
 > with tracker stats?) that there were a lot more bug reports than
 > usual during the alpha and early beta stages of Python 3.2.

Yeah, but the question for Python's stability reputation is "were
there more than zero?"  Every bug that gets through is a risk.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > Pardon me, but people like Stephen Turnbull are *users* of Python, exactly 
 > the 
 > sort of people you DO have to convince that moving to an accelerated or more 
 > complex release process will result in a better product.

Well, to be fair, Antoine is right in excluding me from the user base
he's trying to attract (as I understand it).  I do not maintain
products or systems that depend on Python working 99.9% of the
time, and in fact in many of my personal projects I use trunk.

One of the problems with this kind of discussion is that the targets
of the new procedures are not clear in everybody's mind, but all of us
tend to use generic terms like "users" when we mean to discuss
benefits or costs to a specific class of users.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Daily reference leaks (12de1ad1cee8): sum=6024

[Gregory P. Smith]

On Wed, Jan 18, 2012 at 12:50 PM, Antoine Pitrou  wrote:
>
> Well, they should be fixed now :-)
>
> Regards
>
> Antoine.

awesome! :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com