[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2014-11-28 - 2014-12-05)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open4666 ( -2)
  closed 30095 (+39)
  total  34761 (+37)

Open issues with patches: 2173 


Issues opened (28)
==

#9179: Lookback with group references incorrect (two issues?)
http://bugs.python.org/issue9179  reopened by serhiy.storchaka

#22619: Possible implementation of negative limit for traceback functi
http://bugs.python.org/issue22619  reopened by vlth

#22922: asyncio: call_soon() should raise an exception if the event lo
http://bugs.python.org/issue22922  reopened by haypo

#22964: dbm.open(..., "x")
http://bugs.python.org/issue22964  opened by Antony.Lee

#22968: Lib/types.py nit: isinstance != PyType_IsSubtype
http://bugs.python.org/issue22968  opened by gmt

#22969: Compile fails with --without-signal-module
http://bugs.python.org/issue22969  opened by KHH

#22970: Cancelling wait() after notification leaves Condition in an in
http://bugs.python.org/issue22970  opened by dcoles

#22971: test_pickle: "Fatal Python error: Cannot recover from stack ov
http://bugs.python.org/issue22971  opened by haypo

#22972: Timeout making ajax calls to SimpleHTTPServer from internet ex
http://bugs.python.org/issue22972  opened by Andrew.Burrows

#22976: multiprocessing Queue empty() is broken on Windows
http://bugs.python.org/issue22976  opened by Radosław.Szkodziński

#22977: Unformatted “Windows Error 0x%X” exception message on Wine
http://bugs.python.org/issue22977  opened by vadmium

#22980: C extension naming doesn't take bitness into account
http://bugs.python.org/issue22980  opened by pitrou

#22981: Use CFLAGS when extracting multiarch
http://bugs.python.org/issue22981  opened by pitrou

#22982: BOM incorrectly inserted before writing, after seeking in text
http://bugs.python.org/issue22982  opened by MarkIngramUK

#22983: Cookie parsing should be more permissive
http://bugs.python.org/issue22983  opened by demian.brecht

#22984: test_json.test_endless_recursion(): "Fatal Python error: Canno
http://bugs.python.org/issue22984  opened by haypo

#22985: Segfault on time.sleep
http://bugs.python.org/issue22985  opened by Omer.Katz

#22986: Improved handling of __class__ assignment
http://bugs.python.org/issue22986  opened by njs

#22988: No error when yielding from `finally`
http://bugs.python.org/issue22988  opened by fov

#22989: HTTPResponse.msg not as documented
http://bugs.python.org/issue22989  opened by bastik

#22990: bdist installation dialog
http://bugs.python.org/issue22990  opened by Alan

#22991: test_gdb leaves the terminal in raw mode with gdb 7.8.1
http://bugs.python.org/issue22991  opened by xdegaye

#22992: Adding a git developer's guide to Mercurial to devguide
http://bugs.python.org/issue22992  opened by demian.brecht

#22993: Plistlib fails on certain valid plist values
http://bugs.python.org/issue22993  opened by Connor.Wolf

#22995: Restrict default pickleability
http://bugs.python.org/issue22995  opened by serhiy.storchaka

#22996: Order of _io objects finalization can lose data in reference c
http://bugs.python.org/issue22996  opened by pitrou

#22997: Minor improvements to "Functional API" section of Enum documen
http://bugs.python.org/issue22997  opened by simeon.visser

#22998: inspect.Signature and default arguments
http://bugs.python.org/issue22998  opened by doerwalter



Most recent 15 issues with no replies (15)
==

#22990: bdist installation dialog
http://bugs.python.org/issue22990

#22989: HTTPResponse.msg not as documented
http://bugs.python.org/issue22989

#22985: Segfault on time.sleep
http://bugs.python.org/issue22985

#22981: Use CFLAGS when extracting multiarch
http://bugs.python.org/issue22981

#22970: Cancelling wait() after notification leaves Condition in an in
http://bugs.python.org/issue22970

#22969: Compile fails with --without-signal-module
http://bugs.python.org/issue22969

#22964: dbm.open(..., "x")
http://bugs.python.org/issue22964

#22962: ipaddress: Add optional prefixlen argument to ip_interface and
http://bugs.python.org/issue22962

#22958: Constructors of weakref mapping classes don't accept "self" an
http://bugs.python.org/issue22958

#22956: Improved support for prepared SQL statements
http://bugs.python.org/issue22956

#22947: Enable 'imageop' - "Multimedia Srvices Feature module" for 64-
http://bugs.python.org/issue22947

#22942: Language Reference - optional comma
http://bugs.python.org/issue22942

#22928: HTTP header injection in urrlib2/urllib/httplib/http.client
http://bugs.python.org/issue22928

#22907: Misc/python-config.sh.in: ensure sed invocations only match be
http://bugs.python.org/issue22907

#22893: Idle: __future__ does not work in startup code.
http://bugs.python.org/issue22893



Most recent 15 issues waiting for review (15)
=

[Python-Dev] PEP 479 (Change StopIteration handling inside generators) -- hopefully final text

[Guido van Rossum]

For those who haven't followed along, here's the final text of PEP 479,
with a brief Acceptance section added. The basic plan hasn't changed, but
there's a lot more clarifying text and discussion of a few
counter-proposals. Please send suggestions for editorial improvements to
p...@python.org. The official reference version of the PEP is at
https://www.python.org/dev/peps/pep-0479/; the repo is
https://hg.python.org/peps/ (please check out the repo and send diffs
relative to the repo if you have edits).


PEP: 479
Title: Change StopIteration handling inside generators
Version: $Revision$
Last-Modified: $Date$
Author: Chris Angelico , Guido van Rossum <
gu...@python.org>
Status: Accepted
Type: Standards Track
Content-Type: text/x-rst
Created: 15-Nov-2014
Python-Version: 3.5
Post-History: 15-Nov-2014, 19-Nov-2014, 5-Dec-2014


Abstract


This PEP proposes a change to generators: when ``StopIteration`` is
raised inside a generator, it is replaced it with ``RuntimeError``.
(More precisely, this happens when the exception is about to bubble
out of the generator's stack frame.)  Because the change is backwards
incompatible, the feature is initially introduced using a
``__future__`` statement.


Acceptance
==

This PEP was accepted by the BDFL on November 22.  Because of the
exceptionally short period from first draft to acceptance, the main
objections brought up after acceptance were carefully considered and
have been reflected in the "Alternate proposals" section below.
However, none of the discussion changed the BDFL's mind and the PEP's
acceptance is now final.  (Suggestions for clarifying edits are still
welcome -- unlike IETF RFCs, the text of a PEP is not cast in stone
after its acceptance, although the core design/plan/specification
should not change after acceptance.)


Rationale
=

The interaction of generators and ``StopIteration`` is currently
somewhat surprising, and can conceal obscure bugs.  An unexpected
exception should not result in subtly altered behaviour, but should
cause a noisy and easily-debugged traceback.  Currently,
``StopIteration`` can be absorbed by the generator construct.

The main goal of the proposal is to ease debugging in the situation
where an unguarded ``next()`` call (perhaps several stack frames deep)
raises ``StopIteration`` and causes the iteration controlled by the
generator to terminate silently.  (When another exception is raised, a
traceback is printed pinpointing the cause of the problem.)

This is particularly pernicious in combination with the ``yield from``
construct of PEP 380 [1]_, as it breaks the abstraction that a
subgenerator may be factored out of a generator.  That PEP notes this
limitation, but notes that "use cases for these [are] rare to non-
existent".  Unfortunately while intentional use is rare, it is easy to
stumble on these cases by accident::

import contextlib

@contextlib.contextmanager
def transaction():
print('begin')
try:
yield from do_it()
except:
print('rollback')
raise
else:
print('commit')

def do_it():
print('Refactored preparations')
yield # Body of with-statement is executed here
print('Refactored finalization')

def gene():
for i in range(2):
with transaction():
yield i
# return
raise StopIteration  # This is wrong
print('Should not be reached')

for i in gene():
print('main: i =', i)

Here factoring out ``do_it`` into a subgenerator has introduced a
subtle bug: if the wrapped block raises ``StopIteration``, under the
current behavior this exception will be swallowed by the context
manager; and, worse, the finalization is silently skipped!  Similarly
problematic behavior occurs when an ``asyncio`` coroutine raises
``StopIteration``, causing it to terminate silently.

Additionally, the proposal reduces the difference between list
comprehensions and generator expressions, preventing surprises such as
the one that started this discussion [2]_.  Henceforth, the following
statements will produce the same result if either produces a result at
all::

a = list(F(x) for x in xs if P(x))
a = [F(x) for x in xs if P(x)]

With the current state of affairs, it is possible to write a function
``F(x)`` or a predicate ``P(x)`` that causes the first form to produce
a (truncated) result, while the second form raises an exception
(namely, ``StopIteration``).  With the proposed change, both forms
will raise an exception at this point (albeit ``RuntimeError`` in the
first case and ``StopIteration`` in the second).

Finally, the proposal also clears up the confusion about how to
terminate a generator: the proper way is ``return``, not
``raise StopIteration``.

As an added bonus, the above changes bring generator functions much
more in line with regular functions.  If you wish to take a piece of
code presented as a

[Python-Dev] My thinking about the development process

[Brett Cannon]

This is a bit long as I espoused as if this was a blog post to try and give
background info on my thinking, etc. The TL;DR folks should start at the
"Ideal Scenario" section and read to the end.

P.S.: This is in Markdown and I have put it up at
https://gist.github.com/brettcannon/a9c9a5989dc383ed73b4 if you want a
nicer formatted version for reading.

# History lesson
Since I signed up for the python-dev mailing list way back in June 2002,
there seems to be a cycle where we as a group come to a realization that
our current software development process has not kept up with modern
practices and could stand for an update. For me this was first shown when
we moved from SourceForge to our own infrastructure, then again when we
moved from Subversion to Mercurial (I led both of these initiatives, so
it's somewhat a tradition/curse I find myself in this position yet again).
And so we again find ourselves at the point of realizing that we are not
keeping up with current practices and thus need to evaluate how we can
improve our situation.

# Where we are now
Now it should be realized that we have to sets of users of our development
process: contributors and core developers (the latter whom can play both
roles). If you take a rough outline of our current, recommended process it
goes something like this:

1. Contributor clones a repository from hg.python.org
2. Contributor makes desired changes
3. Contributor generates a patch
4. Contributor creates account on bugs.python.org and signs the
   [contributor agreement](https://www.python.org/psf/contrib/contrib-form/)
4. Contributor creates an issue on bugs.python.org (if one does not already
exist) and uploads a patch
5. Core developer evaluates patch, possibly leaving comments through our
[custom version of Rietveld](http://bugs.python.org/review/)
6. Contributor revises patch based on feedback and uploads new patch
7. Core developer downloads patch and applies it to a clean clone
8. Core developer runs the tests
9. Core developer does one last `hg pull -u` and then commits the changes
to various branches

I think we can all agree it works to some extent, but isn't exactly smooth.
There are multiple steps in there -- in full or partially -- that can be
automated. There is room to improve everyone's lives.

And we can't forget the people who help keep all of this running as well.
There are those that manage the SSH keys, the issue tracker, the review
tool, hg.python.org, and the email system that let's use know when stuff
happens on any of these other systems. The impact on them needs to also be
considered.

## Contributors
I see two scenarios for contributors to optimize for. There's the simple
spelling mistake patches and then there's the code change patches. The
former is the kind of thing that you can do in a browser without much
effort and should be a no-brainer commit/reject decision for a core
developer. This is what the GitHub/Bitbucket camps have been promoting
their solution for solving while leaving the cpython repo alone.
Unfortunately the bulk of our documentation is in the Doc/ directory of
cpython. While it's nice to think about moving the devguide, peps, and even
breaking out the tutorial to repos hosting on Bitbucket/GitHub, everything
else is in Doc/ (language reference, howtos, stdlib, C API, etc.). So
unless we want to completely break all of Doc/ out of the cpython repo and
have core developers willing to edit two separate repos when making changes
that impact code **and** docs, moving only a subset of docs feels like a
band-aid solution that ignores the big, white elephant in the room: the
cpython repo, where a bulk of patches are targeting.

For the code change patches, contributors need an easy way to get a hold of
the code and get their changes to the core developers. After that it's
things like letting contributors knowing that their patch doesn't apply
cleanly, doesn't pass tests, etc. As of right now getting the patch into
the issue tracker is a bit manual but nothing crazy. The real issue in this
scenario is core developer response time.

## Core developers
There is a finite amount of time that core developers get to contribute to
Python and it fluctuates greatly. This means that if a process can be found
which allows core developers to spend less time doing mechanical work and
more time doing things that can't be automated -- namely code reviews --
then the throughput of patches being accepted/rejected will increase. This
also impacts any increased patch submission rate that comes from improving
the situation for contributors because if the throughput doesn't change
then there will simply be more patches sitting in the issue tracker and
that doesn't benefit anyone.

# My ideal scenario
If I had an infinite amount of resources (money, volunteers, time, etc.),
this would be my ideal scenario:

1. Contributor gets code from wherever; easiest to just say "fork on GitHub
or Bitbucket" as they would be official mirrors of hg.python.org and are

Re: [Python-Dev] My thinking about the development process

[Donald Stufft]

> On Dec 5, 2014, at 3:04 PM, Brett Cannon  wrote:
> 

This looks like a pretty good write up, seems to pretty fairly evaluate the 
various sides and the various concerns.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 2/3 porting HOWTO has been updated

[Brett Cannon]

It now promotes using tooling as much as possible to automate the process
of making code by Python 2/3 source-compatible:
https://docs.python.org/3.5/howto/pyporting.html

Blog post about it at
http://nothingbutsnark.svbtle.com/commentary-on-getting-your-code-to-run-on-python-23
.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2/3 porting HOWTO has been updated

[Benjamin Peterson]

On Fri, Dec 5, 2014, at 16:04, Brett Cannon wrote:
> It now promotes using tooling as much as possible to automate the process
> of making code by Python 2/3 source-compatible:
> https://docs.python.org/3.5/howto/pyporting.html

Are you going to update the 2.7 copy of the howto, too?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Eric Snow]

Very nice, Brett.

On Fri, Dec 5, 2014 at 1:04 PM, Brett Cannon  wrote:
> And we can't forget the people who help keep all of this running as well.
> There are those that manage the SSH keys, the issue tracker, the review
> tool, hg.python.org, and the email system that let's use know when stuff
> happens on any of these other systems. The impact on them needs to also be
> considered.

It sounds like Guido would rather as much of this was done by a
provider rather than relying on volunteers.  That makes sense though
there are concerns about control of certain assents.  However, that
applies only to some, like hg.python.org.

>
> ## Contributors
> I see two scenarios for contributors to optimize for. There's the simple
> spelling mistake patches and then there's the code change patches. The
> former is the kind of thing that you can do in a browser without much effort
> and should be a no-brainer commit/reject decision for a core developer. This
> is what the GitHub/Bitbucket camps have been promoting their solution for
> solving while leaving the cpython repo alone. Unfortunately the bulk of our
> documentation is in the Doc/ directory of cpython. While it's nice to think
> about moving the devguide, peps, and even breaking out the tutorial to repos
> hosting on Bitbucket/GitHub, everything else is in Doc/ (language reference,
> howtos, stdlib, C API, etc.). So unless we want to completely break all of
> Doc/ out of the cpython repo and have core developers willing to edit two
> separate repos when making changes that impact code **and** docs, moving
> only a subset of docs feels like a band-aid solution that ignores the big,
> white elephant in the room: the cpython repo, where a bulk of patches are
> targeting.

With your ideal scenario this would be a moot point, right?  There
would be no need to split out doc-related repos.

>
> For the code change patches, contributors need an easy way to get a hold of
> the code and get their changes to the core developers. After that it's
> things like letting contributors knowing that their patch doesn't apply
> cleanly, doesn't pass tests, etc.

This is probably more work than it seems at first.

> As of right now getting the patch into the
> issue tracker is a bit manual but nothing crazy. The real issue in this
> scenario is core developer response time.
>
> ## Core developers
> There is a finite amount of time that core developers get to contribute to
> Python and it fluctuates greatly. This means that if a process can be found
> which allows core developers to spend less time doing mechanical work and
> more time doing things that can't be automated -- namely code reviews --
> then the throughput of patches being accepted/rejected will increase. This
> also impacts any increased patch submission rate that comes from improving
> the situation for contributors because if the throughput doesn't change then
> there will simply be more patches sitting in the issue tracker and that
> doesn't benefit anyone.

This is the key concern I have with only addressing the contributor
side of things.  I'm all for increasing contributions, but not if they
are just going to rot on the tracker and we end up with disillusioned
contributors.

>
> # My ideal scenario
> If I had an infinite amount of resources (money, volunteers, time, etc.),
> this would be my ideal scenario:
>
> 1. Contributor gets code from wherever; easiest to just say "fork on GitHub
> or Bitbucket" as they would be official mirrors of hg.python.org and are
> updated after every commit, but could clone hg.python.org/cpython if they
> wanted
> 2. Contributor makes edits; if they cloned on Bitbucket or GitHub then they
> have browser edit access already
> 3. Contributor creates an account at bugs.python.org and signs the CLA

There's no real way around this, is there?  I suppose account creation
*could* be automated relative to a github or bitbucket user, though it
probably isn't worth the effort.  However, the CLA part is pretty
unavoidable.

> 3. The contributor creates an issue at bugs.python.org (probably the one
> piece of infrastructure we all agree is better than the other options,
> although its workflow could use an update)

I wonder if issue creation from a PR (where no issue # is in the
message) could be automated too without a lot of extra work.

> 4. If the contributor used Bitbucket or GitHub, they send a pull request
> with the issue # in the PR message
> 5. bugs.python.org notices the PR, grabs a patch for it, and puts it on
> bugs.python.org for code review
> 6. CI runs on the patch based on what Python versions are specified in the
> issue tracker, letting everyone know if it applied cleanly, passed tests on
> the OSs that would be affected, and also got a test coverage report
> 7. Core developer does a code review
> 8. Contributor updates their code based on the code review and the updated
> patch gets pulled by bugs.python.org automatically and CI runs again
> 9. Once the patch is acceptable a

Re: [Python-Dev] Python 2/3 porting HOWTO has been updated

[Brett Cannon]

On Fri Dec 05 2014 at 4:07:46 PM Benjamin Peterson 
wrote:

>
>
> On Fri, Dec 5, 2014, at 16:04, Brett Cannon wrote:
> > It now promotes using tooling as much as possible to automate the process
> > of making code by Python 2/3 source-compatible:
> > https://docs.python.org/3.5/howto/pyporting.html
>
> Are you going to update the 2.7 copy of the howto, too?
>

Have not decided yet. All the Google searches I have tried that bring up
the HOWTO use the Python 3 version. Plus I know people are going to find
mistakes that require fixing so I would rather wait until it stabilizes
before I bother backporting to 2.7.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2/3 porting HOWTO has been updated

[Donald Stufft]

> 
> On Dec 5, 2014, at 6:10 PM, Brett Cannon  wrote:
> 
> 
> 
> On Fri Dec 05 2014 at 4:07:46 PM Benjamin Peterson  > wrote:
> 
> 
> On Fri, Dec 5, 2014, at 16:04, Brett Cannon wrote:
> > It now promotes using tooling as much as possible to automate the process
> > of making code by Python 2/3 source-compatible:
> > https://docs.python.org/3.5/howto/pyporting.html 
> > 
> 
> Are you going to update the 2.7 copy of the howto, too?
> 
> Have not decided yet. All the Google searches I have tried that bring up the 
> HOWTO use the Python 3 version. Plus I know people are going to find mistakes 
> that require fixing so I would rather wait until it stabilizes before I 
> bother backporting to 2.7. 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


Do we need to update it? Can it just redirect to the 3 version?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Ian Cordasco]

On Dec 5, 2014 4:18 PM, "Eric Snow"  wrote:
>
> Very nice, Brett.
>
> On Fri, Dec 5, 2014 at 1:04 PM, Brett Cannon  wrote:
> > And we can't forget the people who help keep all of this running as
well.
> > There are those that manage the SSH keys, the issue tracker, the review
> > tool, hg.python.org, and the email system that let's use know when stuff
> > happens on any of these other systems. The impact on them needs to also
be
> > considered.
>
> It sounds like Guido would rather as much of this was done by a
> provider rather than relying on volunteers.  That makes sense though
> there are concerns about control of certain assents.  However, that
> applies only to some, like hg.python.org.
>
> >
> > ## Contributors
> > I see two scenarios for contributors to optimize for. There's the simple
> > spelling mistake patches and then there's the code change patches. The
> > former is the kind of thing that you can do in a browser without much
effort
> > and should be a no-brainer commit/reject decision for a core developer.
This
> > is what the GitHub/Bitbucket camps have been promoting their solution
for
> > solving while leaving the cpython repo alone. Unfortunately the bulk of
our
> > documentation is in the Doc/ directory of cpython. While it's nice to
think
> > about moving the devguide, peps, and even breaking out the tutorial to
repos
> > hosting on Bitbucket/GitHub, everything else is in Doc/ (language
reference,
> > howtos, stdlib, C API, etc.). So unless we want to completely break all
of
> > Doc/ out of the cpython repo and have core developers willing to edit
two
> > separate repos when making changes that impact code **and** docs, moving
> > only a subset of docs feels like a band-aid solution that ignores the
big,
> > white elephant in the room: the cpython repo, where a bulk of patches
are
> > targeting.
>
> With your ideal scenario this would be a moot point, right?  There
> would be no need to split out doc-related repos.
>
> >
> > For the code change patches, contributors need an easy way to get a
hold of
> > the code and get their changes to the core developers. After that it's
> > things like letting contributors knowing that their patch doesn't apply
> > cleanly, doesn't pass tests, etc.
>
> This is probably more work than it seems at first.
>
> > As of right now getting the patch into the
> > issue tracker is a bit manual but nothing crazy. The real issue in this
> > scenario is core developer response time.
> >
> > ## Core developers
> > There is a finite amount of time that core developers get to contribute
to
> > Python and it fluctuates greatly. This means that if a process can be
found
> > which allows core developers to spend less time doing mechanical work
and
> > more time doing things that can't be automated -- namely code reviews --
> > then the throughput of patches being accepted/rejected will increase.
This
> > also impacts any increased patch submission rate that comes from
improving
> > the situation for contributors because if the throughput doesn't change
then
> > there will simply be more patches sitting in the issue tracker and that
> > doesn't benefit anyone.
>
> This is the key concern I have with only addressing the contributor
> side of things.  I'm all for increasing contributions, but not if they
> are just going to rot on the tracker and we end up with disillusioned
> contributors.
>
> >
> > # My ideal scenario
> > If I had an infinite amount of resources (money, volunteers, time,
etc.),
> > this would be my ideal scenario:
> >
> > 1. Contributor gets code from wherever; easiest to just say "fork on
GitHub
> > or Bitbucket" as they would be official mirrors of hg.python.org and are
> > updated after every commit, but could clone hg.python.org/cpython if
they
> > wanted
> > 2. Contributor makes edits; if they cloned on Bitbucket or GitHub then
they
> > have browser edit access already
> > 3. Contributor creates an account at bugs.python.org and signs the CLA
>
> There's no real way around this, is there?  I suppose account creation
> *could* be automated relative to a github or bitbucket user, though it
> probably isn't worth the effort.  However, the CLA part is pretty
> unavoidable.
>
> > 3. The contributor creates an issue at bugs.python.org (probably the one
> > piece of infrastructure we all agree is better than the other options,
> > although its workflow could use an update)
>
> I wonder if issue creation from a PR (where no issue # is in the
> message) could be automated too without a lot of extra work.
>
> > 4. If the contributor used Bitbucket or GitHub, they send a pull request
> > with the issue # in the PR message
> > 5. bugs.python.org notices the PR, grabs a patch for it, and puts it on
> > bugs.python.org for code review
> > 6. CI runs on the patch based on what Python versions are specified in
the
> > issue tracker, letting everyone know if it applied cleanly, passed
tests on
> > the OSs that would be affected, and also got a test coverage report

Re: [Python-Dev] My thinking about the development process

[Nick Coghlan]

On 6 December 2014 at 06:24, Donald Stufft  wrote:
>
> On Dec 5, 2014, at 3:04 PM, Brett Cannon  wrote:
> 
>
> This looks like a pretty good write up, seems to pretty fairly evaluate the
> various sides and the various concerns.

Agreed - thanks for taking this on Brett!

For my part, I realised that if I want my Kallithea based proposal to
work out, I actually need to *be* an upstream Kallithea contributor,
so I posted to the Kallithea list laying out the kinds of features I'd
be pushing for and why:
http://lists.sfconservancy.org/pipermail/kallithea-general/2014q4/60.html

I only posted that a few minutes ago, so we'll see what the existing
Kallithea contributors think of the idea :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2/3 porting HOWTO has been updated

[Benjamin Peterson]

On Fri, Dec 5, 2014, at 18:16, Donald Stufft wrote:
> > 
> > On Dec 5, 2014, at 6:10 PM, Brett Cannon  wrote:
> > 
> > 
> > 
> > On Fri Dec 05 2014 at 4:07:46 PM Benjamin Peterson  > > wrote:
> > 
> > 
> > On Fri, Dec 5, 2014, at 16:04, Brett Cannon wrote:
> > > It now promotes using tooling as much as possible to automate the process
> > > of making code by Python 2/3 source-compatible:
> > > https://docs.python.org/3.5/howto/pyporting.html 
> > > 
> > 
> > Are you going to update the 2.7 copy of the howto, too?
> > 
> > Have not decided yet. All the Google searches I have tried that bring up 
> > the HOWTO use the Python 3 version. Plus I know people are going to find 
> > mistakes that require fixing so I would rather wait until it stabilizes 
> > before I bother backporting to 2.7. 
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
> 
> 
> Do we need to update it? Can it just redirect to the 3 version?

Technically, yes, of course. However, that would unexpected take you out
of the Python 2 docs "context". Also, that doesn't solve the problem for
the downloadable versions of the docs.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Shorya Raj]

Hi All
I just want to put my two cents into this.

This would definitely be a great step to take. I have been discussing PEP
462 with Nick, and the automation was definitely something that would be
something that would be great to have - I mean, I was submitting a simple
documentation patch for build CPython on Windows, and it took several weeks
for the patch to be accepted, then a couple of months for the patch to
actually be merged in. As mentioned, automated testing to ensure that tests
pass, along with easier comitting of documentation patches, would obviously
be a great way to start to decrease this turn around.

Has there been any though on what sort off infrastructure we could use for
this? Obviously github / bitbucket could be used as mentioned by others for
repo management, but a lot of thought would have to go into the decisions
regarding CI tools. I think it would also be a good time to address the
issues with the current bug tracker - although it works, it is hardly as
useable as some of the other ones. As for the argument that we should use
open source tools to ensure that the owners of these tools aren't able to
cause us problems in the future - both Hadoop and Cassandra, along with a
lot of other Apache projects seem to be using JIRA just fine.



Thanks
Shorya Raj

On Sat, Dec 6, 2014 at 11:17 AM, Eric Snow 
wrote:

> Very nice, Brett.
>
> On Fri, Dec 5, 2014 at 1:04 PM, Brett Cannon  wrote:
> > And we can't forget the people who help keep all of this running as well.
> > There are those that manage the SSH keys, the issue tracker, the review
> > tool, hg.python.org, and the email system that let's use know when stuff
> > happens on any of these other systems. The impact on them needs to also
> be
> > considered.
>
> It sounds like Guido would rather as much of this was done by a
> provider rather than relying on volunteers.  That makes sense though
> there are concerns about control of certain assents.  However, that
> applies only to some, like hg.python.org.
>
> >
> > ## Contributors
> > I see two scenarios for contributors to optimize for. There's the simple
> > spelling mistake patches and then there's the code change patches. The
> > former is the kind of thing that you can do in a browser without much
> effort
> > and should be a no-brainer commit/reject decision for a core developer.
> This
> > is what the GitHub/Bitbucket camps have been promoting their solution for
> > solving while leaving the cpython repo alone. Unfortunately the bulk of
> our
> > documentation is in the Doc/ directory of cpython. While it's nice to
> think
> > about moving the devguide, peps, and even breaking out the tutorial to
> repos
> > hosting on Bitbucket/GitHub, everything else is in Doc/ (language
> reference,
> > howtos, stdlib, C API, etc.). So unless we want to completely break all
> of
> > Doc/ out of the cpython repo and have core developers willing to edit two
> > separate repos when making changes that impact code **and** docs, moving
> > only a subset of docs feels like a band-aid solution that ignores the
> big,
> > white elephant in the room: the cpython repo, where a bulk of patches are
> > targeting.
>
> With your ideal scenario this would be a moot point, right?  There
> would be no need to split out doc-related repos.
>
> >
> > For the code change patches, contributors need an easy way to get a hold
> of
> > the code and get their changes to the core developers. After that it's
> > things like letting contributors knowing that their patch doesn't apply
> > cleanly, doesn't pass tests, etc.
>
> This is probably more work than it seems at first.
>
> > As of right now getting the patch into the
> > issue tracker is a bit manual but nothing crazy. The real issue in this
> > scenario is core developer response time.
> >
> > ## Core developers
> > There is a finite amount of time that core developers get to contribute
> to
> > Python and it fluctuates greatly. This means that if a process can be
> found
> > which allows core developers to spend less time doing mechanical work and
> > more time doing things that can't be automated -- namely code reviews --
> > then the throughput of patches being accepted/rejected will increase.
> This
> > also impacts any increased patch submission rate that comes from
> improving
> > the situation for contributors because if the throughput doesn't change
> then
> > there will simply be more patches sitting in the issue tracker and that
> > doesn't benefit anyone.
>
> This is the key concern I have with only addressing the contributor
> side of things.  I'm all for increasing contributions, but not if they
> are just going to rot on the tracker and we end up with disillusioned
> contributors.
>
> >
> > # My ideal scenario
> > If I had an infinite amount of resources (money, volunteers, time, etc.),
> > this would be my ideal scenario:
> >
> > 1. Contributor gets code from wherever; easiest to just say "fork on
> GitHub
> > or Bitbucket" as they would be officia

Re: [Python-Dev] My thinking about the development process

[R. David Murray]

On Fri, 05 Dec 2014 15:17:35 -0700, Eric Snow  
wrote:
> On Fri, Dec 5, 2014 at 1:04 PM, Brett Cannon  wrote:
> > We don't exactly have a ton of people
> > constantly going "I'm so bored because everything for Python's development
> > infrastructure gets sorted so quickly!" A perfect example is that R. David
> > Murray came up with a nice update for our workflow after PyCon but then ran
> > out of time after mostly defining it and nothing ever became of it (maybe we
> > can rectify that at PyCon?). Eric Snow has pointed out how he has written
> > similar code for pulling PRs from I think GitHub to another code review
> > tool, but that doesn't magically make it work in our infrastructure or get
> > someone to write it and help maintain it (no offense, Eric).
> 
> None taken.  I was thinking the same thing when I wrote that. :)
> 
> >
> > IOW our infrastructure can do anything, but it can't run on hopes and
> > dreams. Commitments from many people to making this happen by a certain
> > deadline will be needed so as to not allow it to drag on forever. People
> > would also have to commit to continued maintenance to make this viable
> > long-term.

The biggest blocker to my actually working the proposal I made was that
people wanted to see it in action first, which means I needed to spin up
a test instance of the tracker and do the work there.  That barrier to
getting started was enough to keep me from getting started...even though
the barrier isn't *that* high (I've done it before, and it is easier now
than it was when I first did it), it is still a *lot* higher than
checking out CPython and working on a patch.

That's probably the biggest issue with *anyone* contributing to tracker
maintenance, and if we could solve that, I think we could get more
people interested in helping maintain it.  We need the equivalent of
dev-in-a-box for setting up for testing proposed changes to
bugs.python.org, but including some standard way to get it deployed so
others can look at a live system running the change in order to review
the patch.

Maybe our infrastructure folks will have a thought or two about this?
I'm willing to put some work into this if we can figure out what
direction to head in.  It could well be tied in to moving
bugs.python.org in with the rest of our infrastructure, something I know
Donald has been noodling with off and on; and I'm willing to help with
that as well.

It sounds like being able to propose and test changes to our Roundup
instance (and test other services talking to Roundup, before deploying
them for real) is going to be critical to improving our workflow no
matter what other decisions are made, so we need to make it easier to
do.

In other words, it seems like the key to improving the productivity of
our CPython patch workflow is to improve the productivity of the patch
workflow for our key workflow resource, bugs.python.org.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Donald Stufft]

> On Dec 5, 2014, at 8:26 PM, R. David Murray  wrote:
> 
> On Fri, 05 Dec 2014 15:17:35 -0700, Eric Snow  
> wrote:
>> On Fri, Dec 5, 2014 at 1:04 PM, Brett Cannon  wrote:
>>> We don't exactly have a ton of people
>>> constantly going "I'm so bored because everything for Python's development
>>> infrastructure gets sorted so quickly!" A perfect example is that R. David
>>> Murray came up with a nice update for our workflow after PyCon but then ran
>>> out of time after mostly defining it and nothing ever became of it (maybe we
>>> can rectify that at PyCon?). Eric Snow has pointed out how he has written
>>> similar code for pulling PRs from I think GitHub to another code review
>>> tool, but that doesn't magically make it work in our infrastructure or get
>>> someone to write it and help maintain it (no offense, Eric).
>> 
>> None taken.  I was thinking the same thing when I wrote that. :)
>> 
>>> 
>>> IOW our infrastructure can do anything, but it can't run on hopes and
>>> dreams. Commitments from many people to making this happen by a certain
>>> deadline will be needed so as to not allow it to drag on forever. People
>>> would also have to commit to continued maintenance to make this viable
>>> long-term.
> 
> The biggest blocker to my actually working the proposal I made was that
> people wanted to see it in action first, which means I needed to spin up
> a test instance of the tracker and do the work there.  That barrier to
> getting started was enough to keep me from getting started...even though
> the barrier isn't *that* high (I've done it before, and it is easier now
> than it was when I first did it), it is still a *lot* higher than
> checking out CPython and working on a patch.
> 
> That's probably the biggest issue with *anyone* contributing to tracker
> maintenance, and if we could solve that, I think we could get more
> people interested in helping maintain it.  We need the equivalent of
> dev-in-a-box for setting up for testing proposed changes to
> bugs.python.org, but including some standard way to get it deployed so
> others can look at a live system running the change in order to review
> the patch.
> 
> Maybe our infrastructure folks will have a thought or two about this?
> I'm willing to put some work into this if we can figure out what
> direction to head in.  It could well be tied in to moving
> bugs.python.org in with the rest of our infrastructure, something I know
> Donald has been noodling with off and on; and I'm willing to help with
> that as well.

Theoretically you could create a dev environment with the psf-salt stuff
once it’s actually done. It won’t be the most efficient use of your computer
resources because it’d expect to run several vagrant VMs locally but it would
also match “production” (in a salt-ified world) better. It wouldn’t be as
good as a dedicated dev setup for it, but it would probably be better than
a sort of “yea here’s a bunch of steps that sort of get you close YOLO”.

> 
> It sounds like being able to propose and test changes to our Roundup
> instance (and test other services talking to Roundup, before deploying
> them for real) is going to be critical to improving our workflow no
> matter what other decisions are made, so we need to make it easier to
> do.
> 
> In other words, it seems like the key to improving the productivity of
> our CPython patch workflow is to improve the productivity of the patch
> workflow for our key workflow resource, bugs.python.org.
> 
> --David
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Nick Coghlan]

On 6 December 2014 at 11:39, Donald Stufft  wrote:
>> Maybe our infrastructure folks will have a thought or two about this?
>> I'm willing to put some work into this if we can figure out what
>> direction to head in.  It could well be tied in to moving
>> bugs.python.org in with the rest of our infrastructure, something I know
>> Donald has been noodling with off and on; and I'm willing to help with
>> that as well.
>
> Theoretically you could create a dev environment with the psf-salt stuff
> once it’s actually done. It won’t be the most efficient use of your computer
> resources because it’d expect to run several vagrant VMs locally but it would
> also match “production” (in a salt-ified world) better. It wouldn’t be as
> good as a dedicated dev setup for it, but it would probably be better than
> a sort of “yea here’s a bunch of steps that sort of get you close YOLO”.

For demonstrating UI changes, either a single VM Vagrant setup
specifically for testing, or else something that works in the free
tier of a public PaaS may be a better option. The advantage of those
two approaches is that they'd be potentially acceptable as
contributions to the upstream Roundup project, rather than needing to
be CPython specific.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2/3 porting HOWTO has been updated

[Nick Coghlan]

On 6 December 2014 at 10:44, Benjamin Peterson  wrote:
> On Fri, Dec 5, 2014, at 18:16, Donald Stufft wrote:
>> Do we need to update it? Can it just redirect to the 3 version?
>
> Technically, yes, of course. However, that would unexpected take you out
> of the Python 2 docs "context". Also, that doesn't solve the problem for
> the downloadable versions of the docs.

As Benjamin says, we'll likely want to update the Python 2 version
eventually for the benefit of the downloadable version of the docs,
but Brett's also right it makes sense to wait for feedback on the
Python 3 version and then backport the most up to date text wholesale.

In terms of the text itself, this is a great update Brett - thanks!

A couple of specific notes:

* http://python-future.org/compatible_idioms.html is my favourite
short list of "What are the specific Python 2 only habits that I need
to unlearn in order to start writing 2/3 compatible code?". It could
be worth mentioning in addition to the What's New documents and the
full Python 3 Porting book.

* it's potentially worth explicitly noting the "bytes(index_value)"
and "str(bytes_value)" traps when discussing the bytes/text changes.
Those do rather different things in Python 2 & 3, but won't emit an
error or warning in either version.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 479 (Change StopIteration handling inside generators) -- hopefully final text

[Nick Coghlan]

On 6 December 2014 at 04:42, Guido van Rossum  wrote:
> For those who haven't followed along, here's the final text of PEP 479, with
> a brief Acceptance section added. The basic plan hasn't changed, but there's
> a lot more clarifying text and discussion of a few counter-proposals. Please
> send suggestions for editorial improvements to p...@python.org. The official
> reference version of the PEP is at
> https://www.python.org/dev/peps/pep-0479/; the repo is
> https://hg.python.org/peps/ (please check out the repo and send diffs
> relative to the repo if you have edits).

Thanks Guido, that explanation of the change looks great to me.

And thanks also to Chris and everyone else that helped with the rather
involved discussions!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2/3 porting HOWTO has been updated

[Nick Coghlan]

On 6 December 2014 at 14:40, Nick Coghlan  wrote:
> On 6 December 2014 at 10:44, Benjamin Peterson  wrote:
>> On Fri, Dec 5, 2014, at 18:16, Donald Stufft wrote:
>>> Do we need to update it? Can it just redirect to the 3 version?
>>
>> Technically, yes, of course. However, that would unexpected take you out
>> of the Python 2 docs "context". Also, that doesn't solve the problem for
>> the downloadable versions of the docs.
>
> As Benjamin says, we'll likely want to update the Python 2 version
> eventually for the benefit of the downloadable version of the docs,
> but Brett's also right it makes sense to wait for feedback on the
> Python 3 version and then backport the most up to date text wholesale.
>
> In terms of the text itself, this is a great update Brett - thanks!
>
> A couple of specific notes:
>
> * http://python-future.org/compatible_idioms.html is my favourite
> short list of "What are the specific Python 2 only habits that I need
> to unlearn in order to start writing 2/3 compatible code?". It could
> be worth mentioning in addition to the What's New documents and the
> full Python 3 Porting book.
>
> * it's potentially worth explicitly noting the "bytes(index_value)"
> and "str(bytes_value)" traps when discussing the bytes/text changes.
> Those do rather different things in Python 2 & 3, but won't emit an
> error or warning in either version.

Given that 3.4 and 2.7.9 will be the first exposure some users will
have had to pip, would it perhaps be worth explicitly mentioning the
"pip install " commands for the various tools? At least pylint's
PyPI page only gives the manual download instructions, including which
dependencies you will need to install.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Terry Reedy]

On 12/5/2014 3:04 PM, Brett Cannon wrote:


1. Contributor clones a repository from hg.python.org 
2. Contributor makes desired changes
3. Contributor generates a patch
4. Contributor creates account on bugs.python.org
 and signs the
[contributor
agreement](https://www.python.org/psf/contrib/contrib-form/)


I would like to have the process of requesting and enforcing the signing 
of CAs automated.



4. Contributor creates an issue on bugs.python.org
 (if one does not already exist) and uploads a patch


I would like to have patches rejected, or at least held up, until a CA 
is registered.  For this to work, a signed CA should be immediately 
registered on the tracker, at least as 'pending'.  It now can take a 
week or more to go through human processing.




5. Core developer evaluates patch, possibly leaving comments through our
[custom version of Rietveld](http://bugs.python.org/review/)
6. Contributor revises patch based on feedback and uploads new patch
7. Core developer downloads patch and applies it to a clean clone
8. Core developer runs the tests
9. Core developer does one last `hg pull -u` and then commits the
changes to various branches


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com