Re: [Python-Dev] Proposed schedule for Python 3.4

[Christian Heimes]

Am 03.10.2012 13:54, schrieb Larry Hastings:
> As for the rest of it, my understanding was that there is no longer any
> great plan written in the stars for Python releases.  Python releases
> are comprised of whatever features people propose, implement, and are
> willing to support, that they can get done in time for the beta cutoff. 
> From that perspective, the schedule drives the features more than the
> other-way around.

I have some crypto and security related features planed for Python 3.4.
I'm planing a PEP on password hashing algorithms (password based key
derivation functions) like pbkdf2, bcrypt and scrypt. The topic was
discussed on python-ideas about half a year ago. We agree that it's
worth to have accelerated C implementations in the core.

Two days ago NIST announced the SHA-3 contest winner. My wrapper of
keccak https://bitbucket.org/tiran/pykeccak/ is almost ready and just
needs some cleanup and more tests. Once it's done I'll remove the Python
3.2 and 2.x compatibility code and integrate it into 3.4.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Dirkjan Ochtman]

On Thu, Oct 4, 2012 at 12:32 PM, Christian Heimes  wrote:
> Two days ago NIST announced the SHA-3 contest winner. My wrapper of
> keccak https://bitbucket.org/tiran/pykeccak/ is almost ready and just
> needs some cleanup and more tests. Once it's done I'll remove the Python
> 3.2 and 2.x compatibility code and integrate it into 3.4.

See also https://github.com/bjornedstrom/python-sha3.

Cheers,

Dirkjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Christian Heimes]

Am 04.10.2012 12:49, schrieb Dirkjan Ochtman:
> On Thu, Oct 4, 2012 at 12:32 PM, Christian Heimes  
> wrote:
>> Two days ago NIST announced the SHA-3 contest winner. My wrapper of
>> keccak https://bitbucket.org/tiran/pykeccak/ is almost ready and just
>> needs some cleanup and more tests. Once it's done I'll remove the Python
>> 3.2 and 2.x compatibility code and integrate it into 3.4.
> 
> See also https://github.com/bjornedstrom/python-sha3.

Björn began his implementation about the same time I made my first
experiments. He even ran into the same issue with block_size. I've
compared his code with my code in ticket http://bugs.python.org/issue16113 .

Summary: My code works with Python 3.x (tested with 3.2 and 3.3) and
doesn't require additional Python subclasses. Björns code has better
unit tests as I haven't committed the test suite yet.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



The relevant part:

"""
ranlib libpython2.7.a
gcc   -o python \
Modules/python.o \
libpython2.7.a -lsocket -lnsl -ldl-lm
Traceback (most recent call last):
  File "./setup.py", line 8, in 
from platform import machine as platform_machine
  File
"/export/home/buildbot/32bits/2.7.cea-indiana-x86/build/Lib/platform.py",
line 116, in 
import sys,string,os,re,subprocess
  File
"/export/home/buildbot/32bits/2.7.cea-indiana-x86/build/Lib/subprocess.py",
line 429, in 
import select
ImportError: No module named select
"""

My patch adds a "select" dependency, and "select" is not compiled yet
at this stage.

Any suggestion about how to solve this?

Thanks!.

- -- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG2A/5lgi5GaxT1NAQL0KAQAgTFJc8owtoCAAAQGhhhTqGgrUuUegrM+
PCVZr/IVdnv+tNzAZWl84ng4ZQbfXBzU+6y5XIga0dpifqee9VOVNnLcHL4xv4fP
EWV4jZ9AWN8fPiAbgAe6MGQp3NYlRWTDRoPZTz/zXLqHxzMYzr9LpAOURqXr2/eB
VbYTRAhkGdM=
=m3j8
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Benjamin Peterson]

2012/10/4 Jesus Cea :
> Any suggestion about how to solve this?

Easy solutions include somehow removing the dependence on subprocess
or moving the import of subprocess into the function that uses it.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Using environment variables problematic for embedded python.

[Campbell Barton]

Checked Python 3.4 (latest hg checkout) and this area of the code
hasn't changed: pythonrun,c initstdio()

reported the issue:
http://bugs.python.org/issue16129

On Thu, Oct 4, 2012 at 2:51 PM, Guido van Rossum  wrote:
> This seems more fit for the tracker; can you file there? (Then post the
> issue link here.) I do think you have a legitimate use case to set the
> default encoding to utf-8. (Though there may be a way already.) Does Python
> 3.3 have te same bug?
>
>
> On Wednesday, October 3, 2012, Campbell Barton wrote:
>>
>> On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton 
>> wrote:
>> > Hi,
>> >
>> > We've run into an issue recently with blender3d on ms-windows where we
>> > want to enforce the encoding is UTF-8 with the embedded python
>> > interpreter.
>> > (the encoding defaults to cp437).
>> >
>> > I naively thought setting the environment variable before calling
>> > Py_Initialize() would work, but the way python DLL loads, it gets its
>> > own environment variables that cant be modified directly [1].
>> > eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
>> >
>> > We had bug reports by windows users not able to export files because
>> > the stdout errors on printing paths with unsupported encoding. [2],[3]
>> >
>> > ---
>> >
>> > Of course we could distribute blender with a bat file launcher that
>> > sets env variables, or ask the user to set their env variable - but I
>> > dont think this is really a good option.
>> >
>> > I tried overriding the stderr & stdout, but this caused another bug in
>> > a part of out code that catches exception messages from the stderr.
>> > [4]
>> > import sys, io
>> > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
>> > errors='surrogateescape', line_buffering=True)
>> > sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
>> > errors='surrogateescape', line_buffering=True)
>> >
>> >
>> >
>> > IMHO either of these solutions would be fine.
>> >
>> > * have a PyOS_PutEnv() function, gettext has gettext_putenv() to
>> > workaround this problem.
>> >
>> > * manage this the same as Py_GetPythonHome(), which can be defined by
>> > the embedding application to override the default.
>> >
>> >
>> > Id like to know if there is some known solution to workaround this
>> > issue, if not - would either of these would be acceptable in python
>> > (can write up a patch if it helps)
>> >
>> > Regards,
>> > Campbell
>> >
>> > ---
>> >
>> > [1]
>> > http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
>> > [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
>> > [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
>> > [4] http://projects.blender.org/tracker/?func=detail&aid=32720
>>
>> To follow up and give a correction to overwriting sys.stdout/stderr,
>> The issue seemed to be that __stderr__/__stdout__ was later
>> overwritten, loosing the original reference to the buffer (maybe
>> refcount issue here?), either way it would silence the output.
>>
>>
>> import sys, io
>> sys.__stdout__ = sys.stdout =
>> io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
>> encoding='utf-8', errors='surrogateescape', newline="\n",
>> line_buffering=True)
>> sys.__stderr__ = sys.stderr =
>> io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
>> encoding='utf-8', errors='surrogateescape', newline="\n",
>> line_buffering=True)
>>
>> This all works as expected without bug [4] (above), however on exit I
>> get an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
>> _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);
>>
>> I'd rather not loose more time debugging why this assert happens,
>> IMHO this is too low-level a way to change the encoing of stdio/stderr
>> and error-prone too, so some way to reliably set PYTHONIOENCODING from
>> a program embedding python is still needed.
>>
>> --
>> - Campbell
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
> --
> --Guido van Rossum (python.org/~guido)



-- 
- Campbell
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/10/12 14:34, Benjamin Peterson wrote:
> 2012/10/4 Jesus Cea :
>> Any suggestion about how to solve this?
> 
> Easy solutions include somehow removing the dependence on
> subprocess or moving the import of subprocess into the function
> that uses it.

Moving the import into the function that needs it solves the issue.

See .

Thanks for the suggestion, Benjamin.

- -- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG2MhJlgi5GaxT1NAQLiuAP+OkMW7wVO3rDvbpBGwes5hf6SfixrOyWu
E/ZMcnhoJ3q8VCdpgIR0GKgzBobc4NuNFotakXe10JH6tQKJe/lnZfgzlJwvj5Pc
G85anv08789oYrxzijmqcdrBJUrSoiqO1WEJBhELUFLjT9gtNnQ2RlEID81LMEzq
Su3+VieZcBk=
=flZu
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 03/10/12 17:27, Nick Coghlan wrote:
> I received a fair number of complaints from people that wanted to 
> experiment with yield from, but couldn't, because the first alpha 
> wasn't out yet and they weren't sufficiently interested to go to
> the effort of building their own copy of Python.

In my job, we do "alpha" releases quite frequently. The promises for
alpha is: a) no feature complete, b) the API, etc , can change before
release, c) features included are useable. The point is to use them
and experiment with them.

We don't call them "alphas", but "engineering previews". And they
expires after six months.

Delivering useable alphas requires to do a "minimal" of release
engineering, like disabling features half done, documenting known
bugs, etc. So yes, it it a bit of extra work.

In Python context, the idea of somebody using a "alpha" in production
is scary, and I have a few bad experiences doing a "make install" of a
alpha and when installing the final release keeping some alpha files
around causing erratic behaviour until I did a "\rm -r
/usr/local/SOME" and reinstall again. I think it happened to me with
2.6 :). There are some consequences here, like bytecode (pyc) changes,
modules renaming or vanishing before the final release, etc.

- -- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG2P6Jlgi5GaxT1NAQJLBgP9GRnurLjdx2IIcrnbjGuRC0EoQF5ISh5p
nVQldPRefb3c3MEpsVK0pmTDiPY0Nelepg8j8YMjZeri87AKHojMQ6Teu6NGvx/5
ZMRFsBve8YKfkqAr0ygVl8RlscOMXZsCq2iUHhc8I4L2Nqx7H7yoUDlMcak8JB4t
VxYAhugWOmE=
=/2aa
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Brett Cannon]

On Thu, Oct 4, 2012 at 9:17 AM, Jesus Cea  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 04/10/12 14:34, Benjamin Peterson wrote:
> > 2012/10/4 Jesus Cea :
> >> Any suggestion about how to solve this?
> >
> > Easy solutions include somehow removing the dependence on
> > subprocess or moving the import of subprocess into the function
> > that uses it.
>
> Moving the import into the function that needs it solves the issue.
>
> See .
>
> Thanks for the suggestion, Benjamin.


This used to be a no-no, though, because if someone called that function in
a thread during an import it would deadlock. Now Antoine reworked the locks
so I don't know if this is still true or not.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Nick Coghlan]

On Thu, Oct 4, 2012 at 7:54 PM, Brett Cannon  wrote:
> This used to be a no-no, though, because if someone called that function in
> a thread during an import it would deadlock. Now Antoine reworked the locks
> so I don't know if this is still true or not.

You can still technically deadlock, but you need a circular import
*and* two different threads that start at different parts of the
cycle.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stdlib and timezones, again

[Zachary Ware]

It occurred to me this morning that Python already ships a set of
timezone data with the Windows installer: Tcl/Tk's.  Is there any way
we could use that as the default on Windows?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stdlib and timezones, again

[Brian Curtin]

On Thu, Oct 4, 2012 at 12:01 PM, Zachary Ware
 wrote:
> It occurred to me this morning that Python already ships a set of
> timezone data with the Windows installer: Tcl/Tk's.  Is there any way
> we could use that as the default on Windows?

I would say no. You could choose not to include Tcl/Tk in your
installation, or you could build your own Python and not include those
sources. In either case, the lack of a GUI toolkit is now affecting
your use of timezone data - weird/bad.

Plus we only pull updated Tcl/Tk every few years.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Laurens Van Houtven]

Hello :)

That PEP is indeed pining to the fjords. I may be able to contribute (and
hopefully I can with the preliminary research and
talking-to-people-about-what-they-want that I've already done that maybe
hasn't made it into the draft PEP versions), but I can not currently spare
the cycles to carry it forth from its current state to a completed
implemented in 3.4 PEP.

cheers
lvh

On Thu, Oct 4, 2012 at 8:00 AM, Hynek Schlawack  wrote:

>
> Am 04.10.2012 um 03:38 schrieb R. David Murray :
>
> >>> Other proposed large-scale changes:
> >>> [...]
> >>> * A standard event-loop interface (PEP by Jim Fulton pending)
> >>
> >> Really? Was this discussed somewhere? I'd like to know more about it.
> >
> > I believe it was discussed at the Language Summit at the last Pycon.
> > As I recall there was at least one other person interested in helping
> > with it, but I don't remember who.
>
> T’was lvh and I remember doing some cheap spelling checking on it back at
> EuroPython 20_11_ but the work seems to has stalled 9 months ago.
>
> Adding him to the loop.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stdlib and timezones, again

[Lennart Regebro]

On Thu, Oct 4, 2012 at 7:01 PM, Zachary Ware
 wrote:
> It occurred to me this morning that Python already ships a set of
> timezone data with the Windows installer: Tcl/Tk's.  Is there any way
> we could use that as the default on Windows?

We could, but it wouldn't solve any of the problems that need solving,
so it's not very useful.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Split unicodeobject.c into subfiles?

[Victor Stinner]

Hi,

I would like to split the huge unicodeobject.c file into smaller
files. It's just the longest C file of CPython: 14,849 lines.

I don't know exactly how to split it, but first I would like to know
if you would agree with the idea.

Example:
 - Objects/unicode/codecs.c
 - Objects/unicode/mod_format.c
 - Objects/unicode/methods.c
 - Objects/unicode/operators.c
 - etc.

I don't know if it's better to use a subdirectory, or use a prefix for
new files: Objects/unicode_methods.c, Objects/unicode_codecs.c, etc.
There is already a Python/codecs.c file for example (same filename).

I would like to split the unicodeobject.c because it's hard to
navigate in this huge file between all functions, variables, types,
macros, etc. It's hard to add new code and to fix bugs. For example,
the implementation of str%args takes 1000 lines, 2 types and 10
functions (since my refactor yesterday, in Python 3.3 the main
function is 500 lines long :-)).

I only see one argument against such refactoring: it will be harder to
backport/forwardport bugfixes.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Andrew Svetlov]

I like the idea. From my perspective better to use subdirectory to
sake of easy finding in grep style.

On Thu, Oct 4, 2012 at 11:30 PM, Victor Stinner
 wrote:
> Hi,
>
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.
>
> I don't know exactly how to split it, but first I would like to know
> if you would agree with the idea.
>
> Example:
>  - Objects/unicode/codecs.c
>  - Objects/unicode/mod_format.c
>  - Objects/unicode/methods.c
>  - Objects/unicode/operators.c
>  - etc.
>
> I don't know if it's better to use a subdirectory, or use a prefix for
> new files: Objects/unicode_methods.c, Objects/unicode_codecs.c, etc.
> There is already a Python/codecs.c file for example (same filename).
>
> I would like to split the unicodeobject.c because it's hard to
> navigate in this huge file between all functions, variables, types,
> macros, etc. It's hard to add new code and to fix bugs. For example,
> the implementation of str%args takes 1000 lines, 2 types and 10
> functions (since my refactor yesterday, in Python 3.3 the main
> function is 500 lines long :-)).
>
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Benjamin Peterson]

2012/10/4 Victor Stinner :
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.

I imagine it could also prevent inlining of hot paths.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/10/12 18:38, Nick Coghlan wrote:
> You can still technically deadlock, but you need a circular import 
> *and* two different threads that start at different parts of the 
> cycle.

Is this true for 2.7 too, or only for recent 3.x?

- -- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG4BkJlgi5GaxT1NAQJ3vgP9EDK6d3IKYV02cx/7JFNehK/+MvSdgIUp
KXbf1hlvnMGsWnKgFFKZ4GC16+apyS5g0zXQU5qea/83l06/krcFaERSfZO++alY
jgMJ4AFNBH8r6svoC9Y12WvutV4spzntVWal7kGYh7qTroFceI2sfxHK0PZrsgvn
+qrCmBxspg0=
=2KBh
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[martin]

Zitat von Victor Stinner :


I only see one argument against such refactoring: it will be harder to
backport/forwardport bugfixes.


I'm opposed for a different reason: I think it will be *harder* to maintain.
The amount of code will not be reduced, but now you also need to guess what
file some piece of functionality may be in. Instead of having my text editor
(Emacs) search in one file, it will have to search across multiple files -
but not across all open buffers, but only some of them (since I will have
many other source files open as well).

I really fail to see what problem people have with large source files.
What is it that you want to do that can be done easier if it's multiple
files?

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Victor Stinner]

2012/10/4 Benjamin Peterson :
> 2012/10/4 Victor Stinner :
>> I only see one argument against such refactoring: it will be harder to
>> backport/forwardport bugfixes.
>
> I imagine it could also prevent inlining of hot paths.

It depends how the code is compiled. The stringlib is splitted in many
.h files, but it is able to use Py_LOCAL_INLINE.

If the code is grouped correctly, we may not loose any nice optimization at all.

FYI #include "test.c" is allowed in C ;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Benjamin Peterson]

2012/10/4 Victor Stinner :
> 2012/10/4 Benjamin Peterson :
>> 2012/10/4 Victor Stinner :
>>> I only see one argument against such refactoring: it will be harder to
>>> backport/forwardport bugfixes.
>>
>> I imagine it could also prevent inlining of hot paths.
>
> It depends how the code is compiled. The stringlib is splitted in many
> .h files, but it is able to use Py_LOCAL_INLINE.
>
> If the code is grouped correctly, we may not loose any nice optimization at 
> all.
>
> FYI #include "test.c" is allowed in C ;-)

Yes, but then compilation won't be any faster. ;)



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Chris Jerdonek]

On Thu, Oct 4, 2012 at 1:30 PM, Victor Stinner  wrote:
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.
> ...
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.

I am not siding with either side of the change yet, but an additional
argument against is that history may become less convenient to
navigate and track (e.g. hg annotate may lose information depending on
how the split is done).

Do we have a preferred way to split files?  For example, hg rename
could be used just for the largest chunk.  Or hg copy could be used on
all chunks but one.  I imagine (but have not confirmed) that the
latter would preserve hg annotate and let merges propagate to all
files, but it would also result in spurious merge conflicts in every
one of the files resulting from the split.

--Chris
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tru64 support

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 05/09/12 00:11, mar...@v.loewis.de wrote:
> We don't need to. It's perfectly fine if it breaks - we just can't 
> actively remove code to support the platform.
> 
> OTOH, if Tru64 was proposed for "unsupport", we could disable
> support in 3.4, and remove support in 3.5. Perhaps that would be a
> good thing to do.

In some other email in this thread you said that support was
(partially) deprecated in 3.0, so removal in 3.4 would be justified.

In any case, somebody have to take a decision and update PEP-11.

- -- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG4LKZlgi5GaxT1NAQKeCAQAg43Y6LgrYwJWP/TSDx4X9t+lSxE/47HF
J0cEdFtEhT3hm08qFi3fKhR4p7K8gTVFRxNqVBu89dacjCPUrbbnLVI3tTJjvrst
Joqx5RDZv1q3m/OwoFz+kIXEJFEtVNPY/geQlGGYkyoowc/FqFWl/ygpl8rTdjla
NvFmdoBGFUY=
=DIYs
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Brett Cannon]

On Thu, Oct 4, 2012 at 5:37 PM, Jesus Cea  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 04/10/12 18:38, Nick Coghlan wrote:
> > You can still technically deadlock, but you need a circular import
> > *and* two different threads that start at different parts of the
> > cycle.
>
> Is this true for 2.7 too, or only for recent 3.x?


3.3 only has the improvement.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Victor Stinner]

> I am not siding with either side of the change yet, but an additional
> argument against is that history may become less convenient to
> navigate and track (e.g. hg annotate may lose information depending on
> how the split is done).

If new files are created using "hg cp unicodeobject.c
unicode/newfile.c", the historic is kept.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Benjamin Peterson]

2012/10/4 Victor Stinner :
>> I am not siding with either side of the change yet, but an additional
>> argument against is that history may become less convenient to
>> navigate and track (e.g. hg annotate may lose information depending on
>> how the split is done).
>
> If new files are created using "hg cp unicodeobject.c
> unicode/newfile.c", the historic is kept.

Yes, but you can only create one file that way.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Victor Stinner]

2012/10/5 Benjamin Peterson :
> 2012/10/4 Victor Stinner :
>> If new files are created using "hg cp unicodeobject.c
>> unicode/newfile.c", the historic is kept.
>
> Yes, but you can only create one file that way.

You can create as many files as you want. Try:
---
hg cp unicodeobject.c unicode2.c
hg cp unicodeobject.c unicode3.c
hg ci -m "add new files"


hg ci -m "modify"
hg blame unicode2.c
hg blame unicode3.c
---

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Chris Jerdonek]

On Thu, Oct 4, 2012 at 4:31 PM, Benjamin Peterson  wrote:
> 2012/10/4 Victor Stinner :
>>> I am not siding with either side of the change yet, but an additional
>>> argument against is that history may become less convenient to
>>> navigate and track (e.g. hg annotate may lose information depending on
>>> how the split is done).
>>
>> If new files are created using "hg cp unicodeobject.c
>> unicode/newfile.c", the historic is kept.
>
> Yes, but you can only create one file that way.

You can create multiple files this way.  I just verified it.  But the
problem happens with merging.  You will create merge conflicts in the
deleted portions of every split file on every merge.  There may be a
way to avoid this that I don't know about though (i.e. to record that
merges into the deleted portions should no longer occur).

--Chris
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Eric V. Smith]

On 10/4/2012 4:30 PM, Victor Stinner wrote:
> Hi,
> 
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.

What problem are you trying to solve?

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Serhiy Storchaka]

On 03.10.12 20:10, MRAB wrote:

I downloaded the alphas to test the support for PEP 393 I'd added to
the regex module.


Hardly alpha would be useful to you if it released before PEP 393 
implementation.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Antoine Pitrou]

On Thu, 04 Oct 2012 23:46:57 +0200
mar...@v.loewis.de wrote:
> 
> Zitat von Victor Stinner :
> 
> > I only see one argument against such refactoring: it will be harder to
> > backport/forwardport bugfixes.
> 
> I'm opposed for a different reason: I think it will be *harder* to maintain.
> The amount of code will not be reduced, but now you also need to guess what
> file some piece of functionality may be in. Instead of having my text editor
> (Emacs) search in one file, it will have to search across multiple files -
> but not across all open buffers, but only some of them (since I will have
> many other source files open as well).
> 
> I really fail to see what problem people have with large source files.
> What is it that you want to do that can be done easier if it's multiple
> files?

Navigate, basically. That is, switch between different pieces of code,
without having to type in some text to search for.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[David Bolen]

mar...@v.loewis.de writes:

> We once had nightly builds of the Windows installers. It required a
> dedicated buildbot operator, since the process tended to break.

For what it's worth, I'd be willing to try to re-institute the daily
Windows installer builds if they'd see usage, but I suspect I'll need
assistance with any initial issues with the packaging process, which
is where the issues are most likely to be.

At the time we shut them down (Dec 2008 I think) they had been failing
for about 6 months without anyone noticing, so it didn't seem
repairing it again.

-- David

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Terry Reedy]

On 10/3/2012 12:02 PM, Larry Hastings wrote:


What it really comes down to: I'm a first-time RM, and I lack the
courage/wisdom to overrule what appears to be a reasonable status quo.
I feel I don't have to defend the decision to maintain the status quo; I
feel instead you have to make a case for changing it. So far all I
recall seeing from you are assertions.  I'd like to see some harder data.


On 10/03/2012 05:28 PM, Brian Curtin wrote:

The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
downloads of 3.3a2 Windows installers.


I am pretty sure I downloaded both, as well as most of the subsequent 
releases. Our alpha releases are as stable as the existing maintenance 
release, except for bugs in new features and possible interactions. In 
return, one gets access to the new features. We could promote the early 
releases a bit more than we do.


Building a full interpreter (including tkinter and other modules) on 
Windows is a challenge. Installing is another challenge yet. I installed 
.a1 and onwards for everyday use and so I could test IDLE patches on 
installed 3.3. The sooner 3.4.0a1 is released, the sooner I will be able 
to do the same for 3.4. Before that, I will have to hold off pushing 
idle patches from 3.3 to default or hope that all is well and wait until 
.a1 is out to find out for sure. I suppose you would call this 
integration testing for IDLE (and underneath that, tkinter) on Windows.



We simply don't have a fully-automated process to produce installers for
all platforms.  In fact, I fear we don't have a fully-automated process
to produce installers for /any/ platforms.


I really wish there were for Windows. If an installer were available 
monthly, I would use it.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bootstrap issue: "select" is compiled "too late"

[Antoine Pitrou]

On Thu, 4 Oct 2012 22:08:40 +0530
Nick Coghlan  wrote:
> On Thu, Oct 4, 2012 at 7:54 PM, Brett Cannon  wrote:
> > This used to be a no-no, though, because if someone called that function in
> > a thread during an import it would deadlock. Now Antoine reworked the locks
> > so I don't know if this is still true or not.
> 
> You can still technically deadlock, but you need a circular import
> *and* two different threads that start at different parts of the
> cycle.

Actually, this is detected and handled fine (a partial module is
returned to one of the two threads):
http://hg.python.org/cpython/file/b08416a31d15/Lib/test/test_threaded_import.py#l170
http://hg.python.org/cpython/file/b08416a31d15/Lib/test/test_importlib/test_locks.py#l43

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Ned Deily]

In article 
<20121003203328.horde.za5_tqgzi1vqbit43mhb...@webmail.df.eu>,
 mar...@v.loewis.de wrote:
> I wouldn't mind having alpha 1 in April 2013, and alpha 2 in October 2013.
> I share Larry's skepticism, and actually fear that it may confuse users
> (which find that they test something completely different from what gets
> released). However, I don't mind supporting this strategy anyway as an
> experiment.

I'm certainly also game to supporting earlier alphas.  Perhaps not on 
April 1, though.

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Stephen J. Turnbull]

Chris Jerdonek writes:

 > You can create multiple files this way.  I just verified it.  But the
 > problem happens with merging.  You will create merge conflicts in the
 > deleted portions of every split file on every merge.  There may be a
 > way to avoid this that I don't know about though (i.e. to record that
 > merges into the deleted portions should no longer occur).

"hg commit" will do that automatically, but you need to resolve that
conflict once manually.  If you also happen to be merging *from* a
feature branch *into* the trunk, you need to close the feature branch.
If it needs more work, close it and make a new branch.  Alternatively,
merge from trunk into the feature branch "immediately" to minimize the
accumulation of conflicts.  Then the eventual merge back to trunk will
only have "real" conflicts in it.

Note that "immediately" in the sense needed can be done at any time
because what you need to do is merge the revision created by the "hg
cp" operations.  If Victor tags with "hg tag unicode-refactored", then
you just do "hg merge -r unicode-refactored", and if you haven't made
any changes to the relevant files, you shouldn't get any conflicts.
If you do, then use "hg revert -r unicode-refactored  ...",
followed by "hg resolve --mark  ...", then fix other conflicts,
resolve, and commit as usual.

There's no other way to do it that I know of in any VCS because they
all track conflicts at the file level.  (It would be straightforward
to generalize git to handle this gracefully, but it would be a hugely
disruptive change.  I don't know if Mercurial would be susceptible to
such an extension.)

Specifically, AFAIK this kind of merge conflict will occur:

- if the branch being merged was forked before the "hg cp", and

- for each file in the branch containing changes in the deleted region.

I don't advocate this, just want to make the costs clearer.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split unicodeobject.c into subfiles?

[Benjamin Peterson]

2012/10/4 Antoine Pitrou :
> On Thu, 04 Oct 2012 23:46:57 +0200
> mar...@v.loewis.de wrote:
>>
>> Zitat von Victor Stinner :
>>
>> > I only see one argument against such refactoring: it will be harder to
>> > backport/forwardport bugfixes.
>>
>> I'm opposed for a different reason: I think it will be *harder* to maintain.
>> The amount of code will not be reduced, but now you also need to guess what
>> file some piece of functionality may be in. Instead of having my text editor
>> (Emacs) search in one file, it will have to search across multiple files -
>> but not across all open buffers, but only some of them (since I will have
>> many other source files open as well).
>>
>> I really fail to see what problem people have with large source files.
>> What is it that you want to do that can be done easier if it's multiple
>> files?
>
> Navigate, basically. That is, switch between different pieces of code,
> without having to type in some text to search for.

I find it's only possible to navigate without searching for extremely
small files.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should vars() return modifiable dict?

[Terry Reedy]

On 10/3/2012 11:34 AM, Steven D'Aprano wrote:

On 03/10/12 18:54, Serhiy Storchaka wrote:



Should behavior of vars() be corrected for locals?


I believe you are misinterpreting what you are seeing. In this case,
vars() simply returns locals(), which is an ordinary dict, but changes
to that dict are not guaranteed to propagate to the actual local
variables themselves. You make changes to that dict, then call vars()
again, which returns a fresh locals() dict. So what you are seeing is
simply a side-effect of the fact that changes to locals() may not
actually effect the local variables.

Note that in IronPython, the behaviour of your code is different:


steve@runes:~$ ipy
IronPython 2.6 Beta 2 DEBUG (2.6.0.20) on .NET 2.0.50727.1433
Type "help", "copyright", "credits" or "license" for more information.

def f():

... x = 42
... print(vars())
... vars()['x'] = 43
... vars()['y'] = 44
... print(x, vars())
...

f()

{'x': 42}
(43, {'y': 44, 'x': 43})




Should vars() for objects with __slots__ [1] returns modifiable or
non-modifiable dict?

[1] http://bugs.python.org/issue13290



You are assuming that the behaviour of vars(obj) should change. I don't
think
that is necessarily the case.

vars(obj) is defined as returning the object __dict__ attribute. If an
object
has no __dict__, vars() should (pick one):

1) Keep the current behaviour and raise an exception.

2) Return a regular dict containing {slot: value} for each of the slots.
Since the dict is a copy, changes to the dict will not effect the
original object.

3) Return a dictproxy containing {slot: value} for each of the slots.
Since the dictproxy does not support item assignment, you can't
modify it.

4) Return some other proxy object such that changes to the dict will
also change the object's slot attributes.


I find myself unable to choose between 2) and 4), which suggests that
the status quo wins and we keep the current behaviour.






--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] what´s new 3.3

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 30/09/12 12:55, Kristján Valur Jónsson wrote:
> Hi there.
> 
> Not having kept up, I realized I failed to contribute to the What's
> new thingie.
> 
> Here's stuff I remember working on and putting in:
> 
> 1. pickling support for built in iterators (#14288) 2. inter
> process socket duplication for windows (#14310) 3. Progress
> callback for gc module (#10576) 4. Faster locking on windows
> (#15038)
> 
> 
> 
> DiSome of this should probably be mentioned in the What's new
> document, even if only in its online version.

Please, update "Doc/whatsnew/3.3.rst" in 3.3 branch. The changes are
automatically available in the online documentation after a while (a
few hours).

- -- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG5GZplgi5GaxT1NAQKhaQP/RKzxA44RN2VZeD9MTIK/tiS8ddg/uAWA
9YPYYwVoDQf/sF1S+W/zb/Vf3YjuoHsFFLGTzWIHkY6G8hiVGGfks91/3Ur7n/GO
kE6CHQRpCzMYYnR2MvMjqlO99BD1++ex/+MKSiYFTUwsTIBVLSFicB5oG5WzWH/a
cKl8n0DCv8w=
=ULJD
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for Python 3.4

[Stephen J. Turnbull]

Executive summary:

Larry admits that doing an early alpha is a plausible experiment, he
just has reasons not to do it himself.  OTOH, Nick has a reason for
wanting to do it in *this* release cycle.

It's Larry's call; we should restrict ourselves to giving him the
information he requests to make a decision on which reasons are more
important to him.

Serhiy Storchaka writes:
 > On 03.10.12 20:10, MRAB wrote:
 > > I downloaded the alphas to test the support for PEP 393 I'd added to
 > > the regex module.
 > 
 > Hardly alpha would be useful to you if it released before PEP 393 
 > implementation.

Of course not.  But Nick is requesting this precisely because the PEPs
he wants to get extra testing are already implemented, or very close
to it, and presumably his (implied) promise to have "reasonably
complete" implementations available by alpha 1 is credible.  So your
counterfactual is hardly relevant.

MRAB's example of how alphas are useful is relevant because it's a
real example, with rationale, of the kind of third-party integration
testing Larry suspects *won't happen* if an alpha is released way
early.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Thread/garbage collection race in Popen

[Ben Leslie]

Hi all,

I have a Python program where I have many threads each calling Popen, and I
was hitting some trouble.

I've been seeing this on 3.2.3, however I believe the same issue is still
potentially a problem on head.

The error manifests itself when a call to os.close(errpipe_read) fails with
EBADF (
http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l1314
)

I believe the root cause of this problem is due to a double close() on a
different file descriptor (which is then reused as errpipe_read).

The file descriptors: p2cwrite, c2pread and errread are all closed at the
end of the _execute_child method:

http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l1351

However, these filedescriptors are wrapped up into file objects during
__init__ (see:
http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l725
)

As far as I can tell at the point where the garbage collector kicks in
Popen.{stdin,stdout,stderr} all go out of scope, and will be deallocated,
and the underlying filedescriptor closed.

However because the filedescriptor is already closed, and by this time is
actually reused, this deallocation closes what ends up being an incorrect
file-descriptor.

Since closing a file object where the underlying fd is already closed is
silenced (
http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Modules/_io/iobase.c#l235)
this would not normally be very apparent.

This race between a new filedescriptor being allocated and the garbage
collector deallocating the file descriptors certainly hits when using a few
threads, but I guess depending on the exact behaviour of the garbage
collector it could potentially also occur in a single threaded case as well.

I think a fix would be to remove the explicit close of these file
descriptors at the end of _execute_child, and let the garbage collector
close them. Of course that may leak file descriptors, if the GC doesn't
kick in for a while, so the other option would be to close the file object,
rather than just the file descriptor.

Hopefully someone more intimately familiar with the module can point me in
the right direction to verify this, and provide a fix.

Thanks,

Benno
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com