Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Martin v. Löwis]

On 02.06.2012 02:37, Mark Lawrence wrote:

On 01/06/2012 18:27, Brett Cannon wrote:

About the only thing I can think of from the language summit that we
discussed doing for Python 3.3 that has not come about is accepting the
regex module and getting it into the stdlib. Is this still being worked
towards?



Umpteen versions of regex have been available on pypi for years. Umpteen
bugs against the original re module have been fixed. If regex can't now
go into the standard library, what on earth can?


Something that isn't that big so that a maintainer can really read all 
of it.


I really wish the bug fixes had been made to SRE, instead of rewriting 
it all. So I'm -0 on this regex module. If this isn't added to 3.3,

I'll start encouraging people to contribute changes to SRE for 3.4, and
just ignore the regex module.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Martin v. Löwis]

On 01.06.2012 19:33, Brett Cannon wrote:

Are these dead in the water or are we going to try to change our
release cycle? I'm just asking since 3.3 final is due out in about 3
months and deciding on this along with shifting things if we do make
a change could end up taking that long and I suspect if we don't do
this for 3.3 we are probably never going to do it for Python 3 series
as a whole.


I'm -1 on both PEPs.

For PEP 407, I fail to see what problem it solves. The PEP is short on
rationale, so let me guess what the motivation for the PEP is:

- Some people (Barry in particular) are in favor of timed releases.
  I don't know what the actual motivation for timed releases is, but
  ISTM that PEP 407 is an attempt to make Python generate timed
  releases. I'm -1 on that because of the additional effort for release
  managers. In particular, a strict schedule will limit vacation time,
  and require the release team to coordinate their vacation plans.
  With two alpha, one beta, and one rc, plus LTS bugfix releases, there
  may well be one release of some Python version every month.

- Some contributors are worried about getting their contributions "out",
  and some core committers are worried that we get fewer contributions
  because of that.

  While I well recall the feeling of getting changes "out", the real
  concerns only exist for the very first contribution:
  * Those gurus on python-dev are certainly working on a fix for this
very important issue already, how could they not have noticed?
My work will be futile, and they'll fix it the day before I submit
the patch.
  * Now that the patch is uploaded, can somebody *please* review it?
How hard can it be to look over 20 lines of code?
  * Now that they committed it, when can I start telling my friends
about it? The next release takes ages, and waiting is not fun.

  While these concerns are all real, it's really a matter of contributor
  education to deal with them, The longer people contribute to open
  source (or participate in any kind of software development), the
  more they learn that this is just how things work. The PEP really
  only addresses the third concern, whereas I think that the second
  is much more relevant.

  As for us not getting enough contributions: can we please worry
  about that when we have all patches processed that already have
  been contributed?

I also think that the PEP will have negative effect on Python users:
incompatible changes will spread faster (people will think that it's
ok to break stuff since it was announced for three releases, when
it wasn't actually announced in the last LTS). Users will feel the
urgency of updating, and at the same time uneasiness about doing
so as it may break stuff. People *already* get behind by two or
three releases (in the 2.x series), getting behind 10 releases
just will make them feel sad.

For PEP 413, much the same concerns apply. In addition, I think it's
too complicated, both for users, and for the actual technical
implementation.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] JITted regex engine from pypy

[Maciej Fijalkowski]

Hi

I was reading a bit about the regex module and I would like to present some
other solution into speeding up the re module for Python.

So, as a bit of background - pypy has a re compatible module. It's also
JITted and it's also exportable as a C library (that is a library you can
call from C with C API, not a python extension module). I wonder if it
would be worth to put some work into it to make it a library that CPython
can use.

On the minus side, the JIT only works on x86 and x86_64, on the plus side,
since it's 100% API compatible, it can be used as a _xxx speedup module
relatively easy.

Do people have opinions?

Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Hynek Schlawack]

Am 03.06.12 13:22, schrieb "Martin v. Löwis":

> - Some contributors are worried about getting their contributions "out",
>   and some core committers are worried that we get fewer contributions
>   because of that.
> 
>   While I well recall the feeling of getting changes "out", the real
>   concerns only exist for the very first contribution:
>   * Those gurus on python-dev are certainly working on a fix for this
> very important issue already, how could they not have noticed?
> My work will be futile, and they'll fix it the day before I submit
> the patch.
>   * Now that the patch is uploaded, can somebody *please* review it?
> How hard can it be to look over 20 lines of code?
>   * Now that they committed it, when can I start telling my friends
> about it? The next release takes ages, and waiting is not fun.
> 
>   While these concerns are all real, it's really a matter of contributor
>   education to deal with them, The longer people contribute to open
>   source (or participate in any kind of software development), the
>   more they learn that this is just how things work. The PEP really
>   only addresses the third concern, whereas I think that the second
>   is much more relevant.

As a newish core developer I’d like to stress that Martin is 100% right
here.

Point three was never an issue to me – the biggest satisfaction is
seeing the actual commit with the own name and the appearing in ACKS –
you _can_ already tell your friends/tweet/blog about it at this point.
And people do.

OTOH point two is _very_ frustrating. The most colorful bikeshed is
still much better than ignored patches. Personally, I gave up on CPython
after my patches languished for weeks until Antoine revived the tickets
three months later.

I'm sure we've lost plenty of talent this way already and _if_ we want
to attract more talented contributors, _this_ is the issue to tackle.
The release process has nothing to do with that.

I guess the PEPs (especially 413) are more about the bad rap the stdlib
has been getting lately (e.g.
).

>   As for us not getting enough contributions: can we please worry
>   about that when we have all patches processed that already have
>   been contributed?

Realistically, that means "never".

Cheers,
Hynek
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Calvin Spealman]

On Sun, Jun 3, 2012 at 7:49 AM, Maciej Fijalkowski  wrote:
> Hi
>
> I was reading a bit about the regex module and I would like to present some
> other solution into speeding up the re module for Python.
>
> So, as a bit of background - pypy has a re compatible module. It's also
> JITted and it's also exportable as a C library (that is a library you can
> call from C with C API, not a python extension module). I wonder if it would
> be worth to put some work into it to make it a library that CPython can use.
>
> On the minus side, the JIT only works on x86 and x86_64, on the plus side,
> since it's 100% API compatible, it can be used as a _xxx speedup module
> relatively easy.
>
> Do people have opinions?

A few questions and comments about such an idea, from someone who
hasn't used PyPy yet and doesn't understand the setup involved.

1) Would PyPy be required to build this as a C-compatible library,
such that CPython could use it as an extension module? That is, would
it make PyPy a required part of building CPython?

2) Are there benchmarks comparing the performance of this
implementation to the existing re module and the proposed regex
module?

3) How would the maintenance work? Where would the module live
"officially"? Does CPython fork it or is it extracted from PyPy in a
way it can be installed as an external dependency? How does CPython
get changes upstream?

4) I may be remembering wrong, but I recall maintenance ease to be one
of the justifications for the regex module. How would your proposal
compare? Is a random developer looking to fix a bug in his way going
to find this easier or more difficult to get his head around?

The idea is interesting.

> Cheers,
> fijal
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Maciej Fijalkowski]

On Sun, Jun 3, 2012 at 3:06 PM, Calvin Spealman wrote:

> On Sun, Jun 3, 2012 at 7:49 AM, Maciej Fijalkowski 
> wrote:
> > Hi
> >
> > I was reading a bit about the regex module and I would like to present
> some
> > other solution into speeding up the re module for Python.
> >
> > So, as a bit of background - pypy has a re compatible module. It's also
> > JITted and it's also exportable as a C library (that is a library you can
> > call from C with C API, not a python extension module). I wonder if it
> would
> > be worth to put some work into it to make it a library that CPython can
> use.
> >
> > On the minus side, the JIT only works on x86 and x86_64, on the plus
> side,
> > since it's 100% API compatible, it can be used as a _xxx speedup module
> > relatively easy.
> >
> > Do people have opinions?
>
> A few questions and comments about such an idea, from someone who
> hasn't used PyPy yet and doesn't understand the setup involved.
>
> 1) Would PyPy be required to build this as a C-compatible library,
> such that CPython could use it as an extension module? That is, would
> it make PyPy a required part of building CPython?
>

It depends a bit how we organize stuff. PyPy (as the pypy repository
checkout, not the pypy interpreter) would be requires to build necessary C
files (and as such also maintenance since the C files are not
hand-editable), but pypy would not be required to compile C files.


>
> 2) Are there benchmarks comparing the performance of this
> implementation to the existing re module and the proposed regex
> module?
>

I don't think so. It really is reasonably fast in a lot of cases and it can
definitely be made faster in more cases. The main power comes from JITting
- so you compile a piece of assembler per regex created. I doubt C library
can come close to this approach-wise. Of course there will be cases and
cases, but generally speaking the approach is superior. It would be cool if
someone do the benchmarks how they look like *right now*.


>
> 3) How would the maintenance work? Where would the module live
> "officially"? Does CPython fork it or is it extracted from PyPy in a
> way it can be installed as an external dependency? How does CPython
> get changes upstream?
>

I would honestly hope it can be maintained as a part of pypy and then
cpython would just use it. But those are just hopes.


>
> 4) I may be remembering wrong, but I recall maintenance ease to be one
> of the justifications for the regex module. How would your proposal
> compare? Is a random developer looking to fix a bug in his way going
> to find this easier or more difficult to get his head around?
>

I think it's relatively easy since it's python code after all, but what do
I know. Someone has to have a look, it lives here -
https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rsre I would like
people to have opinions themselves whether it's more or less maintenance
effort. On our side, we'll maintain this particular part of code anyway (so
it's also easier because you leave it to others).

Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Nick Coghlan]

The embedded (in both senses of the term) use cases for CPython pretty much
kill the idea, I'm afraid.

Those cases are also one of the biggest question marks over incorporating
regex wholesale instead of incrementally updating the existing engine to
achieve feature parity.

Publishing such a JIT compiled module via PyPI would be great, though.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Maciej Fijalkowski]

On Sun, Jun 3, 2012 at 3:46 PM, Nick Coghlan  wrote:

> The embedded (in both senses of the term) use cases for CPython pretty
> much kill the idea, I'm afraid.
>

As I said it can (and should) definitely be optional.


> Those cases are also one of the biggest question marks over incorporating
> regex wholesale instead of incrementally updating the existing engine to
> achieve feature parity.
>
> Publishing such a JIT compiled module via PyPI would be great, though.
>
> Cheers,
> Nick.
>
> --
> Sent from my phone, thus the relative brevity :)
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Martin v. Löwis]

On the minus side, the JIT only works on x86 and x86_64, on the plus
side, since it's 100% API compatible, it can be used as a _xxx
speedup module relatively easy.

Do people have opinions?


The main concern for re is not speed, but functionality. The Python re
module needs to grow a number of features, and correct a number of bugs.
So 100% compatible is actually not good enough. 95% compatible (with
the features added and the bugs fixed) would be better.

OTOH, sharing the re code with PyPy would be a desirable goal, as would
be writing the re code in Python (although SRE already implements
significant parts in Python).

As a speedup module, it's uninteresting - we want to simplify 
maintenance, not complicate it. So this can only work if it replaces

SRE.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Maciej Fijalkowski]

On Sun, Jun 3, 2012 at 5:21 PM, "Martin v. Löwis" wrote:

>  On the minus side, the JIT only works on x86 and x86_64, on the plus
>> side, since it's 100% API compatible, it can be used as a _xxx
>> speedup module relatively easy.
>>
>> Do people have opinions?
>>
>
> The main concern for re is not speed, but functionality. The Python re
> module needs to grow a number of features, and correct a number of bugs.
> So 100% compatible is actually not good enough. 95% compatible (with
> the features added and the bugs fixed) would be better.
>
> OTOH, sharing the re code with PyPy would be a desirable goal, as would
> be writing the re code in Python (although SRE already implements
> significant parts in Python).
>

We did not reimplement those parts in RPython, they're still in python (so
the sre engine does not accept regex, but instead the lower-level
description, etc. etc.)


>
> As a speedup module, it's uninteresting - we want to simplify maintenance,
> not complicate it. So this can only work if it replaces
> SRE.
>
> Regards,
> Martin
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 3.3 on Windows 2000

[Martin v. Löwis]

It seems that by moving to VS 2010, we have killed Windows 2000 support,
for the same reason VS 2012 would kill XP support: Windows 2000
apparently won't recognize the .exe files as executables anymore.
I haven't actually tested this: can somebody please confirm?

A year ago, Brian put a statement into PEP 11 that 3.3 would support
Windows 2000 still, but with a warning. Under my recent PEP change,
Windows 2000 does not need to be supported anymore since 13.07.2010,
when Microsoft's extended support expired.

So I propose to just remove the claim from the PEP that 3.3 would
still be supported, and not go through any notification period.

Objections?

As a consequence, we could then change some of the deferred-loading
stuff for "new" (i.e. XP+) API into proper linking.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Barry Warsaw]

On Jun 03, 2012, at 01:22 PM, Martin v. Löwis wrote:

>- Some people (Barry in particular) are in favor of timed releases.
>   I don't know what the actual motivation for timed releases is, but

Timed releases in general can provide much better predictability for others
depending on those releases.  E.g. folks working on things to go into Python
can plan better how to make sure their stuff is ready in time, and downstreams
can *much* better plan on which Python versions to include in their products
and releases.

Having said that, unless there's widespread consensus within the Python
developers for timed releases, then it's not going to work, either within the
context of those PEPs or not.  After the last round of mostly negative
feedback, I don't personally have much motivation to push these through.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Benjamin Peterson]

2012/6/3 Maciej Fijalkowski :
> Hi
>
> I was reading a bit about the regex module and I would like to present some
> other solution into speeding up the re module for Python.

IMO, the most important feature of the regex module is that it fixes
long standing bugs and includes long requested features especially
with respect to Unicode. That it's faster is only windfall.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Terry Reedy]

On 6/3/2012 7:22 AM, "Martin v. Löwis" wrote:

On 01.06.2012 19:33, Brett Cannon wrote:

Are these dead in the water or are we going to try to change our
release cycle? I'm just asking since 3.3 final is due out in about 3
months and deciding on this along with shifting things if we do make
a change could end up taking that long and I suspect if we don't do
this for 3.3 we are probably never going to do it for Python 3 series
as a whole.


I'm -1 on both PEPs.


I pretty much agree. There is certainly no consensus and the possible 
benefit is not obviously substantially more than the cost.



For PEP 407, I fail to see what problem it solves. The PEP is short on
rationale, so let me guess what the motivation for the PEP is:

...

While I well recall the feeling of getting changes "out", the real
concerns only exist for the very first contribution:

...

* Now that the patch is uploaded, can somebody *please* review it?
How hard can it be to look over 20 lines of code?


Example http://bugs.python.org/issue13598
OP submitted revised patch in response to review 4 months ago


As for us not getting enough contributions: can we please worry
about that when we have all patches processed that already have
been contributed?


I suspect that having too many unattended patches sitting on the tracker 
discourages one from writing and submitting more. I also suspect, for 
instance, that applying some of Roger Serwy's Idle patches has 
encouraged him to write more.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.3 on Windows 2000

[Larry Hastings]

On 06/03/2012 08:32 AM, "Martin v. Löwis" wrote:

So I propose to just remove the claim from the PEP that 3.3 would
still be supported, and not go through any notification period.


Did you mean

   s/3.3/Windows 2000/

?


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.3 on Windows 2000

[Martin v. Löwis]

On 03.06.2012 18:18, Larry Hastings wrote:

On 06/03/2012 08:32 AM, "Martin v. Löwis" wrote:

So I propose to just remove the claim from the PEP that 3.3 would
still be supported, and not go through any notification period.


Did you mean

s/3.3/Windows 2000/

?


I meant "that 3.3 would support Windows 2000".

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Gregory P. Smith]

On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan  wrote:

> ipaddress really made it in because I personally ran into the limitations
> of not having IP address support in the stdlib. I ended up doing quite a
> bit of prompting to ensure the process of cleaning up the API to modern
> stdlib standards didn't stall (even now, generating a module reference from
> the docstrings is still a pending task)
>
> With regex, the pain isn't there, since re already covers such a large
> subset of what regex provides.
>
That last statement basically suggests that something like regex would
never be accepted until a CPython core developer was actually running into
pain with the many flaws in the re module (especially when it comes to
Unicode).  I disagree with that.

Per the language summit, I think we need to just do it.  Put it in as re
and rename the existing re module to sre.

We could pull the plug on it and leave it out if substantial as yet unknown
problems that can't be fixed in time for release crop up during the beta 1
or 2 (release manager's decision).

> My perspective is that it's now too late to make a change that big for
> 3.3, but the in principle approval holds for anyone that wants to work with
> MRAB and get the idea written up as a PEP for 3.4.
>
Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd
according to PEP 398) it can go in.

I don't like to claim that a PEP for this one is *strictly* necessary but
Nick raises good questions to be answered and has good suggestions for what
to write up in the PEP in his earlier response that I certainly would
prefer to have gathered up and documented so that is the route I suggest.

The issue seems to be primarily one of "who is volunteering to do it?"

-gps
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] JITted regex engine from pypy

[Bill Janssen]

Maciej Fijalkowski  wrote:

> On Sun, Jun 3, 2012 at 5:21 PM, "Martin v. Löwis" wrote:
> 
> >  On the minus side, the JIT only works on x86 and x86_64, on the plus
> >> side, since it's 100% API compatible, it can be used as a _xxx
> >> speedup module relatively easy.
> >>
> >> Do people have opinions?
> >>
> >
> > The main concern for re is not speed, but functionality. The Python re
> > module needs to grow a number of features, and correct a number of bugs.
> > So 100% compatible is actually not good enough. 95% compatible (with
> > the features added and the bugs fixed) would be better.

From my point of view, for textual data reduction, the MRAB regex now
has substantial improvements which enable very different kinds of uses,
like "named lists" and "fuzzy" matching, which I don't believe occur
(together) in any other RE library.  Along with features it shares with
the existing CPython "re" library, such as the ability to handle very
large RE's (which IronPython, for instance, is unable to handle,
apparently due to its use of the standard .NET RE library).  And do so
fairly efficiently.

Bill

> >
> > OTOH, sharing the re code with PyPy would be a desirable goal, as would
> > be writing the re code in Python (although SRE already implements
> > significant parts in Python).
> >
> 
> We did not reimplement those parts in RPython, they're still in python (so
> the sre engine does not accept regex, but instead the lower-level
> description, etc. etc.)
> 
> 
> >
> > As a speedup module, it's uninteresting - we want to simplify maintenance,
> > not complicate it. So this can only work if it replaces
> > SRE.
> >
> > Regards,
> > Martin
> >
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/bill%40janssen.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Nick Coghlan]

On Sun, Jun 3, 2012 at 9:22 PM, "Martin v. Löwis"  wrote:
> On 01.06.2012 19:33, Brett Cannon wrote:
>>
>> Are these dead in the water or are we going to try to change our
>> release cycle? I'm just asking since 3.3 final is due out in about 3
>> months and deciding on this along with shifting things if we do make
>> a change could end up taking that long and I suspect if we don't do
>> this for 3.3 we are probably never going to do it for Python 3 series
>> as a whole.
>
>
> I'm -1 on both PEPs.

Unsurprisingly, I'm -1 on PEP 407. Perhaps surprisingly, I'm also -0
on my own PEP 413 (I wrote it to present what I consider a more
tolerable alternative to an idea I really don't like)

I think marking both as Rejected would be an accurate reflection of
python-dev's collective opinion.

> For PEP 413, much the same concerns apply. In addition, I think it's
> too complicated, both for users, and for the actual technical
> implementation.

Yup (although I think PEP 407 would need to be *at least* as
complicated in practice as PEP 413 in order to make the implementation
manageable, but currently glosses over the technical details).

The one thing I actually *would* like to see change is for the cadence
of *alpha* releases to be increased to match that of maintenance
releases (that is, I'd like to see Python 3.4a1 released at the same
time as Python 3.3.1: around 6 months after the release of 3.3.0). I
think keeping the trunk closer to a "releasable" state will help
encourage a more regular rate of contributions and provide earlier
deadlines for big changes (e.g. it's significantly easier to say "we
want to have the compiler changes in place for 3.4a1 in April" than it
is to say "we want to have these changes in place by April, but that's
just an arbitrary point in time, since the nearest release deadline
will still be at least 12 months away". Scheduling things like sprints
and bug days also becomes more focused, since they have a nearer term
goal of getting things fixed for an alpha release that's only a few
months away rather than one that's more than a year out).

It also lowers the bar for getting people to tinker with and provide
feedback on new syntax like PEP 380 and core features like pyvenv and
pysetup that behave differently when installed instead of being run
from a source checkout. At the moment, the criteria for providing
early feedback on new syntax is "interested in the feature, and can
build CPython from source", while the criteria for installed features
is "interested in the feature, can build CPython from source, and can
install the result on a target system". Early alphas means that the
criteria for providing feedback becomes simply: "interested in the
feature, and has access to a system that can tolerate having the alpha
release installed".

These alpha releases can also feed into vendor schemes such as Red
Hat's tech preview program: while the system Python would always be a
released version, an alpha version may still be an adequate foundation
for a tech preview. As the other Python implementations catch up to
the 3.x series, the alphas would also provide clear recommended
synchronisation points that may make it easier for them to start
targeting CPython release compatibility *before* we publish the final
version.

As I see it, such an approach would achieve most of the benefits of a
regular release cadence with basically *none* of the seriously
disruptive effects created by the more ambitious schemes described in
PEP 407 or 413. I also consider it an excellent test run: if we can't
even produce alpha releases of the upcoming version every 6 months or
so, how on earth could we ever consider trying to create *production*
releases on that schedule?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Nick Coghlan]

On Mon, Jun 4, 2012 at 7:02 AM, Nick Coghlan  wrote:
> I think marking both as Rejected would be an accurate reflection of
> python-dev's collective opinion.

Slight correction: I think it would accurately reflect python-dev's
*divided* opinion, using the principle of "Status quo wins a
stalemate". The costs for either scheme are high, the benefits are not
proven, thus the default is to stick with the status quo.

Releasing alphas early, OTOH, doesn't require any real changes to our
development process at all, aside from imposing a bit more discipline
on trunk development in the first 12 months of the release cycle (I'm
inclined to place that particular detail on the "benefit" side of the
ledger, rather than the "cost" side). The *total* number of releases
from the release managers and installer builders shouldn't increase
much, if at all - I'd suggest we just stick with Georg's practice of 4
alpha releases, and merely space them out over the course of the
release cycle rather than clustered together at the end.

If Larry doesn't want to try this for 3.4, then I'll most likely
volunteer as 3.5 RM and try it out then.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Nick Coghlan]

On Mon, Jun 4, 2012 at 6:25 AM, Gregory P. Smith  wrote:
>
> On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan  wrote:
>>
>> ipaddress really made it in because I personally ran into the limitations
>> of not having IP address support in the stdlib. I ended up doing quite a bit
>> of prompting to ensure the process of cleaning up the API to modern stdlib
>> standards didn't stall (even now, generating a module reference from the
>> docstrings is still a pending task)
>>
>> With regex, the pain isn't there, since re already covers such a large
>> subset of what regex provides.
>
> That last statement basically suggests that something like regex would never
> be accepted until a CPython core developer was actually running into pain
> with the many flaws in the re module (especially when it comes to Unicode).
>  I disagree with that.

No, that's not really what I meant. Driving integration of a module
takes *time* and *effort*. The decision to commit that effort has to
be driven by something, and personal annoyance is a great motivator.
In the case of PEP 3144, I happened to be in a position to do
something about a gap in the standard library after the omission was
made glaringly obvious [1].

Getting this done was a combined effort from Peter (in getting the
module API updated), myself and others (esp. Antoine) in reviewing the
reference implementation's API and requesting changes and more
recently Sandro Tosi has been doing most of the heavy lifting in
getting the docs up to scratch.

> Per the language summit, I think we need to just do it.  Put it in as re and
> rename the existing re module to sre.

No. We almost burned Jesse out dropping multiprocessing into 2.6 at
the last minute, and many longstanding issues with that module are
only being addressed now that Richard has the time to be involved
again. SRE already suffers from a lack of maintenance, and we've had
zero indication that regex will make that situation better (and
several indications that it will actually make it worse. Matthew's
silence on the topic is *not* encouraging, and nobody else has even
volunteered to write a PEP, let alone agree to maintain the module).

> We could pull the plug on it and leave it out if substantial as yet unknown
> problems that can't be fixed in time for release crop up during the beta 1
> or 2 (release manager's decision).

Unwinding changes to the build process is yet more work that may not
be needed. We need to remember the purpose of the standard library:
most of the time, it is *not* intended to be all things to all people.
The status quo is that, if you're doing basic, primarily ASCII,
regular expression processing, then "import re" will serve you just
fine. If you're doing more than that, then you'll probably need to do
"pip install regex" (or platform specific equivalent) and change your
import to "import regex as re".

That's not *great* (as the number of open Unicode bugs against SRE can
attest), but it's far from unworkable. I consider it preferable to
adding yet another big ball of C code to the stdlib in the absence of
a PEP addressing the concerns already raised.

>> My perspective is that it's now too late to make a change that big for
>> 3.3, but the in principle approval holds for anyone that wants to work with
>> MRAB and get the idea written up as a PEP for 3.4.
>
> Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd
> according to PEP 398) it can go in.
>
> I don't like to claim that a PEP for this one is strictly necessary

Why not? Requiring a PEP is the norm, not the exception. Even when
there's agreement that something *should* be done, there's plenty of
details to be thrashed out in turning in principle agreement into a
concrete plan of action.

> but Nick
> raises good questions to be answered and has good suggestions for what to
> write up in the PEP in his earlier response that I certainly would prefer to
> have gathered up and documented so that is the route I suggest.
>
> The issue seems to be primarily one of "who is volunteering to do it?"

Correct, both in figuring out the integration details and in agreeing
to maintain it in the future.

Remember, now is better than never, but never is often better than
*right* now :)

Cheers,
Nick.

[1] 
http://git.fedorahosted.org/git/?p=pulpdist.git;a=blob;f=src/pulpdist/core/validation.py;h=ebccf354c5bbec376258681a345fb73129eeeb95;hb=736250d85b758a11e1d09f70ec3877d1c022aa9a#l77


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 405 (built-in virtualenv) status

[Éric Araujo]

Hi,

Le 02/06/2012 12:59, Christian Tismer a écrit :
> One urgent question: will this feature be backported to Python 2.7?

Features are never backported to the stable versions.  virtualenv still
exists as a standalone project which is compatible with 2.7 though.

Regards
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Gregory P. Smith]

On Sun, Jun 3, 2012 at 2:38 PM, Nick Coghlan  wrote:

> On Mon, Jun 4, 2012 at 6:25 AM, Gregory P. Smith  wrote:
> >
> > On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan  wrote:
> >>
> >> ipaddress really made it in because I personally ran into the
> limitations
> >> of not having IP address support in the stdlib. I ended up doing quite
> a bit
> >> of prompting to ensure the process of cleaning up the API to modern
> stdlib
> >> standards didn't stall (even now, generating a module reference from the
> >> docstrings is still a pending task)
> >>
> >> With regex, the pain isn't there, since re already covers such a large
> >> subset of what regex provides.
> >
> > That last statement basically suggests that something like regex would
> never
> > be accepted until a CPython core developer was actually running into pain
> > with the many flaws in the re module (especially when it comes to
> Unicode).
> >  I disagree with that.
>
> No, that's not really what I meant. Driving integration of a module
> takes *time* and *effort*. The decision to commit that effort has to
> be driven by something, and personal annoyance is a great motivator.
> In the case of PEP 3144, I happened to be in a position to do
> something about a gap in the standard library after the omission was
> made glaringly obvious [1].
>
> Getting this done was a combined effort from Peter (in getting the
> module API updated), myself and others (esp. Antoine) in reviewing the
> reference implementation's API and requesting changes and more
> recently Sandro Tosi has been doing most of the heavy lifting in
> getting the docs up to scratch.
>
> > Per the language summit, I think we need to just do it.  Put it in as re
> and
> > rename the existing re module to sre.
>
> No. We almost burned Jesse out dropping multiprocessing into 2.6 at
> the last minute, and many longstanding issues with that module are
> only being addressed now that Richard has the time to be involved
> again. SRE already suffers from a lack of maintenance, and we've had
> zero indication that regex will make that situation better (and
> several indications that it will actually make it worse. Matthew's
> silence on the topic is *not* encouraging, and nobody else has even
> volunteered to write a PEP, let alone agree to maintain the module).
>
> > We could pull the plug on it and leave it out if substantial as yet
> unknown
> > problems that can't be fixed in time for release crop up during the beta
> 1
> > or 2 (release manager's decision).
>
> Unwinding changes to the build process is yet more work that may not
> be needed. We need to remember the purpose of the standard library:
> most of the time, it is *not* intended to be all things to all people.
> The status quo is that, if you're doing basic, primarily ASCII,
> regular expression processing, then "import re" will serve you just
> fine. If you're doing more than that, then you'll probably need to do
> "pip install regex" (or platform specific equivalent) and change your
> import to "import regex as re".
>
> That's not *great* (as the number of open Unicode bugs against SRE can
> attest), but it's far from unworkable. I consider it preferable to
> adding yet another big ball of C code to the stdlib in the absence of
> a PEP addressing the concerns already raised.
>
> >> My perspective is that it's now too late to make a change that big for
> >> 3.3, but the in principle approval holds for anyone that wants to work
> with
> >> MRAB and get the idea written up as a PEP for 3.4.
> >
> > Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd
> > according to PEP 398) it can go in.
> >
> > I don't like to claim that a PEP for this one is strictly necessary
>
> Why not? Requiring a PEP is the norm, not the exception. Even when
> there's agreement that something *should* be done, there's plenty of
> details to be thrashed out in turning in principle agreement into a
> concrete plan of action.
>
> > but Nick
> > raises good questions to be answered and has good suggestions for what to
> > write up in the PEP in his earlier response that I certainly would
> prefer to
> > have gathered up and documented so that is the route I suggest.
> >
> > The issue seems to be primarily one of "who is volunteering to do it?"
>
> Correct, both in figuring out the integration details and in agreeing
> to maintain it in the future.
>
> Remember, now is better than never, but never is often better than
> *right* now :)
>
>
heh.  indeed.  regardless, the module is available on pypi whether it goes
in or not so we do at least have something to point people to when they
need more than the existing undermaintained re (sre) module.

There are also other options with different properties such as
http://pypi.python.org/pypi/re2/.

-gps
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/

Re: [Python-Dev] PEP 405 (built-in virtualenv) status

[Nick Coghlan]

On Sun, Jun 3, 2012 at 3:33 AM, Christian Tismer  wrote:
> As an old windows guy, I very much agree with Kristjan. The venv
> approach is great. Windows is just a quite weird situation to handle
> in some cases, and a super-simple way to get rid of *any* built-in behavior
> concerning setup would be great.
>
> The idea of moving path setup stuff into the python.exe stub
> makes very much sense to me. This would make pythonxx.dll
> a really useful library to be shared.

It's mainly Py_Initialize() that triggers the magic. What may be worth
exploring is a variant on that which allows embedding applications to
explicitly pass in *everything* that would otherwise be guessed by
inspecting the environment. (Some things can be forced to particular
values already, but certainly not everything).

> Python has IMHO too much behavior like this:
> 'by default, look into xxx, but if a yyy exists, behave differently'.
> I don't like this, because the absense of a simple file changes the whole
> system behavior.
> I would do it the other way round:
> As soon as you introduce the venv.cfg file, enforce its existence
> completely! If that file is not there, then python exits with an error
> message.
> This way you can safely ensure its existence, and the file can be made
> read-only and so on. A non-existent file is just a bad thing and is hard to
> make
> read-only ;-)
> So please let's abandon the old 'if exists ...' pattern, at least this one
> time.
> By the explicit cfg file, the file can clearly say if there is a virtual env
> or not.

Backwards compatibility constraints mean we simply can't do that.
However, as noted above, it may make sense to provide more ways for
embedding applications to selectively access the behaviour through the
C API.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 405 (built-in virtualenv) status

[Christian Tismer]

On 04.06.12 00:28, Nick Coghlan wrote:

...
Backwards compatibility constraints mean we simply can't do that.
However, as noted above, it may make sense to provide more ways for
embedding applications to selectively access the behaviour through the
C API.


Why that???
I don't see this.
If you have a new python version with a new file that has-to-be-there,
what is then the problem? The new version carries the new file, so I
don't see a compatibility issue, because this version does not want
to be backward-compatible. It just introduces the new file constraint,
and it produces what it needs.

Am I somehow blinded, maybe?

(yes, you all know that I am, so please be patient with me) -- Chris

--
Christian Tismer :^)
tismerysoft GmbH : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key ->  http://pgp.uni-mainz.de
work +49 173 24 18 776  mobile +49 173 24 18 776  fax n.a.
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Martin v. Löwis]

That last statement basically suggests that something like regex would
never be accepted until a CPython core developer was actually running
into pain with the many flaws in the re module (especially when it comes
to Unicode).  I disagree with that.

Per the language summit, I think we need to just do it.  Put it in as re
and rename the existing re module to sre.


There are really places where "we" just doesn't work, even in a
community project. "We" will never commit anything to revision control.
Individual committers commit.

So if *you* want to commit it, go ahead - I think there is general
approval for that. Take the praise when it works, and take the (likely)
blame for when it fails in some significant way, and then work on fixing
it.


The issue seems to be primarily one of "who is volunteering to do it?"


I don't think anybody is, or will be for the coming years. I wish I had
trust into MRAB to stay around and work on this for the next ten years
(and I think the author of the regex module really needs to commit for
that timespan, see SRE's history), but I don't. So whoever commits the
change now is in charge, and will either have to work hard on fixing the
problems, or will be responsible for breaking Python 3 in a serious way.

That's why nobody volunteers.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Martin v. Löwis]

heh.  indeed.  regardless, the module is available on pypi whether it
goes in or not so we do at least have something to point people to when
they need more than the existing undermaintained re (sre) module.


I completely disagree that SRE is unmaintained. It has about monthly 
commits to it, to fix reported bugs, by various people.


It may be aged software, but that has the advantage that more people
are familiar with the code base now than back in the days when /F
was still maintaining it.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Nick Coghlan]

I apologise, "unmaintained" is too strong a word. I mean "lacking an owner
sufficiently confident in their authority and expertise and with sufficient
time and energy to add,or approve the addition of, substantial new features
which may require significant refactoring of internal details".

Perhaps "unowned" would be a better word? Saying yes or no to major feature
requests isn't the same as fixing errors in existing features. (Compare
regular email package maintenance to RDM's recent updates)

--
Sent from my phone, thus the relative brevity :)
On Jun 4, 2012 8:55 AM, Martin v. Löwis  wrote:

> heh.  indeed.  regardless, the module is available on pypi whether it
>> goes in or not so we do at least have something to point people to when
>> they need more than the existing undermaintained re (sre) module.
>>
>
> I completely disagree that SRE is unmaintained. It has about monthly
> commits to it, to fix reported bugs, by various people.
>
> It may be aged software, but that has the advantage that more people
> are familiar with the code base now than back in the days when /F
> was still maintaining it.
>
> Regards,
> Martin
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

[Steven D'Aprano]

Gregory P. Smith wrote:


Per the language summit, I think we need to just do it.  Put it in as re
and rename the existing re module to sre.


I thought that the plan was to put the regex module in as regex, leaving re 
unchanged for backwards compatibility, with any backwards-incompatible 
renaming to be done some time in the future.




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Nick Coghlan]

On Mon, Jun 4, 2012 at 7:11 AM, Nick Coghlan  wrote:
> On Mon, Jun 4, 2012 at 7:02 AM, Nick Coghlan  wrote:
>> I think marking both as Rejected would be an accurate reflection of
>> python-dev's collective opinion.
>
> Slight correction: I think it would accurately reflect python-dev's
> *divided* opinion, using the principle of "Status quo wins a
> stalemate". The costs for either scheme are high, the benefits are not
> proven, thus the default is to stick with the status quo.
>
> Releasing alphas early, OTOH, doesn't require any real changes to our
> development process at all, aside from imposing a bit more discipline
> on trunk development in the first 12 months of the release cycle (I'm
> inclined to place that particular detail on the "benefit" side of the
> ledger, rather than the "cost" side). The *total* number of releases
> from the release managers and installer builders shouldn't increase
> much, if at all - I'd suggest we just stick with Georg's practice of 4
> alpha releases, and merely space them out over the course of the
> release cycle rather than clustered together at the end.
>
> If Larry doesn't want to try this for 3.4, then I'll most likely
> volunteer as 3.5 RM and try it out then.

After an off-list discussion with Larry, I'm now planning to expand on
this concept in PEP form (superceding 413). There's actually a little
bit more to it than just releasing the alphas early - it's about
harnessing the power of external deadlines to help counter innate
tendencies towards procrastination :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?

[Terry Reedy]

On 6/3/2012 5:02 PM, Nick Coghlan wrote:


The one thing I actually *would* like to see change is for the cadence
of *alpha* releases to be increased to match that of maintenance
releases (that is, I'd like to see Python 3.4a1 released at the same
time as Python 3.3.1: around 6 months after the release of 3.3.0). I
think keeping the trunk closer to a "releasable" state will help
encourage a more regular rate of contributions and provide earlier
deadlines for big changes (e.g. it's significantly easier to say "we
want to have the compiler changes in place for 3.4a1 in April" than it
is to say "we want to have these changes in place by April, but that's
just an arbitrary point in time, since the nearest release deadline
will still be at least 12 months away". Scheduling things like sprints
and bug days also becomes more focused, since they have a nearer term
goal of getting things fixed for an alpha release that's only a few
months away rather than one that's more than a year out).


I like this idea. The main thing that makes alpha releases not 
'production' releases is not having more bugs, because they generally do 
not, but instability of new features. So I think this might have many of 
the benefits of the non-accepted PEPs with much lower cost.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com