date:20081204

On Thu, Dec 4, 2008 at 2:36 AM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> I would like to merge mailing lists, now that the design and first
> implementation of Python 3000 is complete. In particular, I would

+1


  -Fred

-- 
Fred L. Drake, Jr.
"Chaos is the score upon which reality is written." --Henry Miller
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Ondrej Certik

On Thu, Dec 4, 2008 at 3:24 AM, Barry Warsaw <[EMAIL PROTECTED]> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On Dec 3, 2008, at 9:13 PM, Dotan Cohen wrote:
>
>> On this page:
>> http://www.python.org/download/releases/3.0/
>>
>> The text "This is a proeuction release" should probably read "This is
>> a production release". It would give a better first impression :)
>
> Fixed, thanks!

I tried to find the documentation here:

http://python.org/doc/

but clicking on the links:

http://docs.python.org/whatsnew/3.0.html
http://docs.python.org/3.0

gives me:

404 Not Found

Ondrej
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

Ondrej Certik wrote:
> I tried to find the documentation here:
> 
> http://python.org/doc/
> 
> but clicking on the links:
> 
> http://docs.python.org/whatsnew/3.0.html
> http://docs.python.org/3.0

These 404 for me as well. but the dev links have already rolled over to
3.1a0.

There are also no cross-links from the main 2.6 docs to the released
py3k docs.

I was going to suggest there needs to be something in PEP 101 about
checking the doc links, but it's already there :)

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r67511 - in python/trunk: Doc/library/logging.rst Lib/logging/init.py Lib/test/test_logging.py Misc/NEWS

vinay.sajip wrote:
> +def _showwarning(message, category, filename, lineno, file=None, line=None):
> +"""
> +Implementation of showwarnings which redirects to logging, which will 
> first
> +check to see if the file parameter is None. If a file is specified, it 
> will
> +delegate to the original warnings implementation of showwarning. 
> Otherwise,
> +it will call warnings.formatwarning and will log the resulting string to 
> a
> +warnings logger named "py.warnings" with level logging.WARNING.
> +"""
> +if file is not None:
> +if _warnings_showwarning is not None:
> +_warnings_showwarning(message, category, filename, lineno, file, 
> line)
> +else:
> +import warnings
> +s = warnings.formatwarning(message, category, filename, lineno, line)
> +logger = getLogger("py.warnings")
> +if not logger.handlers:
> +logger.addHandler(NullHandler())
> +logger.warning("%s", s)

I'd be careful here - this could deadlock if a thread spawned as a side
effect of importing a module happens to trigger a warning.

warnings is pulled into sys.modules as part of the interpreter startup -
having a global "import warnings" shouldn't have any real effect on
logging's import time.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging mailing lists

2008-12-04 Thread Christian Heimes


Martin v. Löwis wrote:

Any objections?


+1

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

On Wed, Dec 03, 2008 at 08:51:33PM -0500, Barry Warsaw wrote:
> On behalf of the Python development team and the Python community, I  
> am happy to announce the release of Python 3.0 final.

Yay!

> We are confident that Python 3.0 is of the same high quality as our  
> previous releases, such as the recently announced Python 2.6.  We will  
> continue to support and develop both Python 3 and Python 2 for the  
> foreseeable future, and you can safely choose either version (or both)  
> to use in your projects.  Which you choose depends on your own needs  
> and the availability of third-party packages that you depend on.  Some  
> other things to consider:

I think we should also have a statement upon on python.org about
future plans: e.g.

* that there will be a Python 2.7 that will incorporate what we learn from
  people trying to port,
* that 3.1 will rearrange the standard library in mostly-known ways, and 
* that we expect people to use 3.0 mostly for compatibility testing, 
  not going into serious production use until 3.1 or maybe even 3.2.

(The details are open to discussion, of course.)

--amk

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Georg Brandl

Nick Coghlan schrieb:
> Ondrej Certik wrote:
>> I tried to find the documentation here:
>> 
>> http://python.org/doc/
>> 
>> but clicking on the links:
>> 
>> http://docs.python.org/whatsnew/3.0.html
>> http://docs.python.org/3.0
> 
> These 404 for me as well. but the dev links have already rolled over to
> 3.1a0.
> 
> There are also no cross-links from the main 2.6 docs to the released
> py3k docs.
> 
> I was going to suggest there needs to be something in PEP 101 about
> checking the doc links, but it's already there :)

I can't find any docs built for Python 3.0 (not 3.1a0).  I would have
handled building and uploading the docs if somebody (or at least anybody)
had told me I was to do it.  Now we again have the situation that the
docs for the new release are wrecked.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Steve Holden

Georg Brandl wrote:
> Nick Coghlan schrieb:
>> Ondrej Certik wrote:
>>> I tried to find the documentation here:
>>>
>>> http://python.org/doc/
>>>
>>> but clicking on the links:
>>>
>>> http://docs.python.org/whatsnew/3.0.html
>>> http://docs.python.org/3.0
>> These 404 for me as well. but the dev links have already rolled over to
>> 3.1a0.
>>
>> There are also no cross-links from the main 2.6 docs to the released
>> py3k docs.
>>
>> I was going to suggest there needs to be something in PEP 101 about
>> checking the doc links, but it's already there :)
> 
> I can't find any docs built for Python 3.0 (not 3.1a0).  I would have
> handled building and uploading the docs if somebody (or at least anybody)
> had told me I was to do it.  Now we again have the situation that the
> docs for the new release are wrecked.
> 
Sounds like we need a bot to check the web each new release before the
release manager "presses the button" and makes the announcement.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Facundo Batista

2008/12/4 A.M. Kuchling <[EMAIL PROTECTED]>:

> * that there will be a Python 2.7 that will incorporate what we learn from
>  people trying to port,
> * that 3.1 will rearrange the standard library in mostly-known ways, and
> * that we expect people to use 3.0 mostly for compatibility testing,
>  not going into serious production use until 3.1 or maybe even 3.2.

I think that would be fantastic to have a small set of straightforward
sentences like these, to transmit the most important stuff.

For my part, when it's fixed, I will translate them to spanish and
propagate them.


> (The details are open to discussion, of course.)

I think those are fine. I would add something about the migration
path, something like "If you want to start testing your library/system
in 3.0, you should first use Python 2.6, see migration details [here]"

Regards,

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Merging mailing lists

2008-12-04 Thread Jeremy Hylton

On Thu, Dec 4, 2008 at 2:36 AM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> I would like to merge mailing lists, now that the design and first
> implementation of Python 3000 is complete. In particular, I would
> like to merge the python-3000 mailing list back into python-dev,
> and the python-3000-checkins mailing list back into python-checkins.
> The rationale is to simplify usage of the lists, and to avoid
> cross-postings.

+1

> To implement this, all subscribers of the 3000 mailing lists would
> be added to the trunk mailing lists (avoiding duplicates, of course),
> and all automated messages going to python-3000-checkins would then
> be directed to the trunk lists. The 3000 mailing lists would change
> into read-only mode (i.e. primarily leaving the archives behind).
>
> Any objections?

No

Jeremy


> Regards,
> Martin
> ___
> Python-3000 mailing list
> [EMAIL PROTECTED]
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-3000/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Merging flow

2008-12-04 Thread Christian Heimes

Several people have asked about the patch and merge flow. Now that 
Python 3.0 is out it's a bit more complicated.


Flow diagram


trunk ---> release26-maint
   \->  py3k   ---> release30-maint


Patches for all versions of Python should land in the trunk. They are 
then merged into release26-maint and py3k branches. Changes for Python 
3.0 are merged via the py3k branch.


Christian

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final


Georg Brandl wrote:

I can't find any docs built for Python 3.0 (not 3.1a0). 


The Windows installation has new 3.0 doc dated Dec 3, so it was built, 
just not posted correctly.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] RELEASED Python 3.0 final


Guido van Rossum wrote:


Python 3.0 (a.k.a. "Python 3000" or "Py3k") represents a major milestone in
Python's history, and was nearly three years in the making.  This is a new
version of the language that is incompatible with the 2.x line of releases,


I think this


while remaining true to BDFL Guido van Rossum's vision.  Some things you
will notice include:

* Fixes to many old language warts
* Removal of long deprecated features and redundant syntax
* Improvements in, and a reorganization of, the standard library
* Changes to the details of how built-in objects like strings and dicts work
* ...and many more new features

While these changes were made without concern for backward compatibility,


and this could give some people a mis-impression, most likely negative, 
as to the magnitude and nature of the change.  Most of the code I am now 
writing would, I believe, run with 2.5 except for print(..., file=xxx). 
 And I know that there was concern for backward compatibility to the 
point that some changes were rejected (renaming builtins) or delayed 
(deleting duplicate test asserts) for that reason.  So I would soften 
the statements to "... version of the language that is partially 
incompatible with... " and "were made without being bound by backward 
compatibility,"


tjr

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Mark Dickinson

On Thu, Dec 4, 2008 at 3:12 PM, Christian Heimes <[EMAIL PROTECTED]> wrote:
> Patches for all versions of Python should land in the trunk. They are then
> merged into release26-maint and py3k branches. Changes for Python 3.0 are
> merged via the py3k branch.

Thanks, Christian!

Questions:

(1) If I commit a change to the trunk that I don't want to go into
release26-maint, should I explicitly block it using svnmerge?

(2) Same question for trunk -> py3k

(3) Same question for py3k -> release30-maint.

I'm guessing that the answers are (1) No, (2) Yes, (3) No.

Mark
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Benjamin Peterson

On Thu, Dec 4, 2008 at 11:23 AM, Mark Dickinson <[EMAIL PROTECTED]> wrote:
> On Thu, Dec 4, 2008 at 3:12 PM, Christian Heimes <[EMAIL PROTECTED]> wrote:
>> Patches for all versions of Python should land in the trunk. They are then
>> merged into release26-maint and py3k branches. Changes for Python 3.0 are
>> merged via the py3k branch.
>
> Thanks, Christian!
>
> Questions:
>
> (1) If I commit a change to the trunk that I don't want to go into
> release26-maint, should I explicitly block it using svnmerge?
>
> (2) Same question for trunk -> py3k
>
> (3) Same question for py3k -> release30-maint.
>
> I'm guessing that the answers are (1) No, (2) Yes, (3) No.

That is correct. We don't care too much about blocking for the release branches.



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000-checkins] Merging mailing lists

On Wed, Dec 3, 2008 at 23:36, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> I would like to merge mailing lists, now that the design and first
> implementation of Python 3000 is complete. In particular, I would
> like to merge the python-3000 mailing list back into python-dev,
> and the python-3000-checkins mailing list back into python-checkins.
> The rationale is to simplify usage of the lists, and to avoid
> cross-postings.
>
> To implement this, all subscribers of the 3000 mailing lists would
> be added to the trunk mailing lists (avoiding duplicates, of course),
> and all automated messages going to python-3000-checkins would then
> be directed to the trunk lists. The 3000 mailing lists would change
> into read-only mode (i.e. primarily leaving the archives behind).
>
> Any objections?
>

Nope; +1.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Jeremy Hylton

On Thu, Dec 4, 2008 at 10:12 AM, Christian Heimes <[EMAIL PROTECTED]> wrote:
> Several people have asked about the patch and merge flow. Now that Python
> 3.0 is out it's a bit more complicated.
>
> Flow diagram
> 
>
> trunk ---> release26-maint
>   \->  py3k   ---> release30-maint
>
>
> Patches for all versions of Python should land in the trunk. They are then
> merged into release26-maint and py3k branches. Changes for Python 3.0 are
> merged via the py3k branch.

You say "they are then merged."  Does that mean if I commit something
on the trunk, someone else will merge it for me?  Or do I need to do
it?

The library is vastly different between 2.x and 3.x.  I'm personally
aware of the many changes related to httplib / urllib / xmlrpclib.
I'm worried that it will be hard to decide how to "merge" things
between the two versions.

Jeremy

> Christian
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Benjamin Peterson

On Thu, Dec 4, 2008 at 12:18 PM, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> On Thu, Dec 4, 2008 at 10:12 AM, Christian Heimes <[EMAIL PROTECTED]> wrote:
>> Several people have asked about the patch and merge flow. Now that Python
>> 3.0 is out it's a bit more complicated.
>>
>> Flow diagram
>> 
>>
>> trunk ---> release26-maint
>>   \->  py3k   ---> release30-maint
>>
>>
>> Patches for all versions of Python should land in the trunk. They are then
>> merged into release26-maint and py3k branches. Changes for Python 3.0 are
>> merged via the py3k branch.
>
> You say "they are then merged."  Does that mean if I commit something
> on the trunk, someone else will merge it for me?  Or do I need to do
> it?

Generally, somebody else will do it if it is on the trunk and bound
for py3k. (Bug fixes should be backported by the original committer.)
Of course, if the change required in py3k is complicated and vastly
different, I and the other mergers would appreciate it if you did it
yourself.

>
> The library is vastly different between 2.x and 3.x.  I'm personally
> aware of the many changes related to httplib / urllib / xmlrpclib.
> I'm worried that it will be hard to decide how to "merge" things
> between the two versions.

Feel free to do it yourself.

>
> Jeremy




-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Eric Smith


Christian Heimes wrote:
Several people have asked about the patch and merge flow. Now that 
Python 3.0 is out it's a bit more complicated.


Flow diagram


trunk ---> release26-maint
   \->  py3k   ---> release30-maint


Patches for all versions of Python should land in the trunk. They are 
then merged into release26-maint and py3k branches. Changes for Python 
3.0 are merged via the py3k branch.


Apologies if this has been discussed before. I looked but didn't see 
anything.


Given that at least 99% of the changes for the trunk will not get merged 
into release26-maint, doesn't it make more sense to merge the other way? 
That is, anything that gets checked in to release26-maint would 
potentially be merged into trunk. That would remove the huge number of 
merge blocks that will otherwise be required. Same fore py3k and 
release30-maint.


Eric.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Taint Mode in Python 3.0

2008-12-04 Thread Nicole King

Dear All,

I have published the diff for my implementation of tainted mode in Python for 
R3.0 (released version) at http://www.cats-muvva.net/software/. Look at the 
bottom the page. I apologise for past problems accessing this web site: I 
hope to have resolved all the issues with it.

Nicole
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Benjamin Peterson

On Thu, Dec 4, 2008 at 12:52 PM, Eric Smith <[EMAIL PROTECTED]> wrote:
> Christian Heimes wrote:
>>
>> Several people have asked about the patch and merge flow. Now that Python
>> 3.0 is out it's a bit more complicated.
>>
>> Flow diagram
>> 
>>
>> trunk ---> release26-maint
>>   \->  py3k   ---> release30-maint
>>
>>
>> Patches for all versions of Python should land in the trunk. They are then
>> merged into release26-maint and py3k branches. Changes for Python 3.0 are
>> merged via the py3k branch.
>
> Apologies if this has been discussed before. I looked but didn't see
> anything.
>
> Given that at least 99% of the changes for the trunk will not get merged
> into release26-maint, doesn't it make more sense to merge the other way?
> That is, anything that gets checked in to release26-maint would potentially
> be merged into trunk. That would remove the huge number of merge blocks that
> will otherwise be required. Same fore py3k and release30-maint.

I think the percentage is a bit lower than that. Also, we haven't been
using blocking with the maintenance branch so far; svnmerge.py is just
a convenience. (It generates commit messages and has a simpler
interface than a simple "svn merge" command.)



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Raymond Hettinger


From: "A.M. Kuchling" <[EMAIL PROTECTED]>

I think we should also have a statement upon on python.org about
future plans: e.g.

* that there will be a Python 2.7 that will incorporate what we learn from
 people trying to port,
* that 3.1 will rearrange the standard library in mostly-known ways, and 
* that we expect people to use 3.0 mostly for compatibility testing, 
 not going into serious production use until 3.1 or maybe even 3.2.


The latter statement worries me.  It seems to unnecessarily undermine
adoption of 3.0.  It essentially says, "don't use this".  Is that what we want?
ISTM, 3.0 is in pretty good shape.  There is nothing intrinsically wrong
with it.  The number one adoption issue is external, i.e. how quickly
key third-party modules get converted.


Raymond

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Taint Mode in Python 3.0

2008-12-04 Thread Maciej Fijalkowski

When I try to run this, I get:

Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "/home/fijal/lang/python/Python30/Lib/encodings/__init__.py",
line 31, in 
  File "/home/fijal/lang/python/Python30/Lib/codecs.py", line 1060, in 
TaintError: using tainted data
Aborted

Are there any tests what it should do? Didn't find it in a diff

On Thu, Dec 4, 2008 at 7:36 PM, Nicole King <[EMAIL PROTECTED]> wrote:
> Dear All,
>
> I have published the diff for my implementation of tainted mode in Python for
> R3.0 (released version) at http://www.cats-muvva.net/software/. Look at the
> bottom the page. I apologise for past problems accessing this web site: I
> hope to have resolved all the issues with it.
>
> Nicole
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Barry Warsaw


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Dec 4, 2008, at 2:12 PM, Raymond Hettinger wrote:


From: "A.M. Kuchling" <[EMAIL PROTECTED]>

I think we should also have a statement upon on python.org about
future plans: e.g.
* that there will be a Python 2.7 that will incorporate what we  
learn from

people trying to port,
* that 3.1 will rearrange the standard library in mostly-known  
ways, and * that we expect people to use 3.0 mostly for  
compatibility testing,  not going into serious production use until  
3.1 or maybe even 3.2.


The latter statement worries me.  It seems to unnecessarily undermine
adoption of 3.0.  It essentially says, "don't use this".  Is that  
what we want?
ISTM, 3.0 is in pretty good shape.  There is nothing intrinsically  
wrong

with it.  The number one adoption issue is external, i.e. how quickly
key third-party modules get converted.


I agree.  I tried to put a positive spin on the announcement, and the  
backward compatibility issue in particular.  I probably failed.


- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSTgybHEjvBPtnXfVAQJPjgP+NeyLY2ACryOmxeRV8qcotKrMJZYBwu6q
gtNjax3m0faRr2VrRwVLpiJqBoVkwpr+heKg7z2rR183MstsgQ9QsQpkZXBV+QnH
yK1yA18jaVZhLMR0VPT75GN1KPp5KCL+TbuT0cFtJ/SSt1LT5K356jdMYFi/ZbUP
t2YtaWoxB5o=
=4lo8
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow


On Dec 4, 2008, at 1:52 PM, Eric Smith wrote:
Apologies if this has been discussed before. I looked but didn't see  
anything.


Probably has, just 'cause everything has been discussed before.

Given that at least 99% of the changes for the trunk will not get  
merged into release26-maint, doesn't it make more sense to merge the  
other way? That is, anything that gets checked in to release26-maint  
would potentially be merged into trunk. That would remove the huge  
number of merge blocks that will otherwise be required. Same fore  
py3k and release30-maint.


The directions of merges were established in the past at some point.   
Though they feel wrong (at least to you and me), the direction is what  
it is.  I'd asked about the direction mostly because I can never  
remember after time away from working on the Python tree.


That said, don't let Python's decision on the direction keep you from  
managing your own projects the right way.  :-)


In fact, it's reasonable to fix bugs on the release26-maint branch,  
migrate the patch to the trunk, and then use svnmerge.py from there to  
propagate the changes.



  -Fred

--
Fred Drake   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python-3.0, unicode, and os.environ

I opened up bug http://bugs.python.org/issue4006 a while ago and it was
suggested in the report that it's not a bug but a feature and so I
should come here to see about getting the feature changed :-)

I have a specific problem with os.environ and a somewhat less important
architectural issue with the unicode/bytes handling in certain os.*
modules. I'll start with the important one:

Currently in python3 there's no way to get at environment variables that
are not encoded in the system default encoding. My understanding is
that this isn't a problem on Windows systems but on *nix this is a huge
problem. environment variables on *nix are a sequence of non-null
bytes. These bytes are almost always "characters" but they do not have
to be. Further, there is nothing that requires that the characters be
in the same encoding; some of the characters could be in the UTF-8
character set while others are in latin-1, shift-jis, or big-5.

These mixed encodings can occur for a variety of reasons. Here's an
example that isn't too contrived :-)

Swallow is a multi-user shell server hosted at a university in Japan.
The OS installed is Fedora 10 where the encoding of all filenames
provided by the OS are UTF-8. The administrator of the OS has kept this
convention and, among other things has created a directory to mount and
NFS directory from another computer. He calls that "ネットワーク"
("network" in Japanese). Since it's utf-8, that gets put on the
filesystem as
'\xe3\x83\x8d\xe3\x83\x83\xe3\x83\x88\xe3\x83\xaf\xe3\x83\xbc\xe3\x82\xaf'

Now the administrators of the fileserver have been maintaining it since
before Unicode was invented. Furthermore, they don't want to suffer
from the space loss of using utf-8 to encode Japanese so they use
shift-jis everywhere. They have a directory on the nfs share for
programs that are useful for people on the shell server to access. It's
called "プログラム" ("programs" in Japanese) Since they're using
shift-jis, the bytes on the filesystem are:
'\x83v\x83\x8d\x83O\x83\x89\x83\x80'

The system administrator of the shell server adds the directory of
programs to all his user's default PATH variables so then they have this:

PATH=/bin:/usr/bin:/usr/local/bin:/mnt/\xe3\x83\x8d\xe3\x83\x83\xe3\x83\x88\xe3\x83\xaf\xe3\x83\xbc\xe3\x82\xaf/\x83v\x83\x8d\x83O\x83\x89\x83\x80

(Note: python syntax, In the unix shell you'd likely have octal instead
of hex)

Now comes the problematic part. One of the user's on the system wants
to write a python3 program that needs to determine if a needed program
is in the user's PATH. He tries to code it like this::

#!/usr/bin/python3.0

import os

for directory in os.environ['PATH']:
programs = os.listdir(directory)

That code raises a KeyError because python3 has silently discarded the
PATH due to the shift-jis encoded path elements. Much more importantly,
there's no way the programmer can handle the KeyError and actually get
the PATH from within python.

In the bug report I opened, I listed four ways to fix this along with
the pros and cons:

1) return mixed unicode and byte types in os.environ and os.getenv
- I think this one is a bad idea. It's the easiest for simple code
to deal with but it's repeating the major problem with python2's Unicode
handling: mixing unicode and byte types unpredictably.

2) return only byte types in os.environ
- This is conceptually correct but the most annoying option.
Technically we're receiving bytes from the C libraries and the C
libraries expect bytes in return. But in the common case we will be
dealing with things in one encoding so this causes needless effort to
the application programmer in the common case.

3) silently ignore non-decodable value when accessing os.environ['PATH']
as we do now but allow access to the full information via
os.environ[b'PATH'] and os.getenvb().
- This mirrors the practice of os.listdir('.') vs os.listdir(b'.') and
os.getcwd() vs os.getcwdb().

4) raise an exception when non-decodable values are *accessed* and
continue as in #3. This means that os.environ wouldn't be a simple dict
as it would need to decode the values when keys are accessed (although
it could cache the values).
- This mirrors the practice of open() which is to decode the value for
the common case but throw an exception and allow the programmer to
decide what to do if all values are not decodable.

Either #3 or #4 will solve the major problem and both have precedent in
python3's current implementation. The difference between them is
whether to throw an exception when a non-decodable value is encountered.
Here's why I think that's appropriate:

One of the things I enjoy about python is the informative tracebacks
that make debugging easy. I think that the ease of debugging is lost
when we silently ignore an error. If we look at the difference in
coding and debugging for problems with files that aren't encoded in the
default encoding (where a traceback is issued) and os.listdir() when
filenames aren't in the d

Re: [Python-Dev] Holding a Python Language Summit at PyCon

2008-12-04 Thread Frank Wierzbicki

On Wed, Dec 3, 2008 at 10:31 AM, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
> 14:00 - 15:30
> =
>
> Two tracks:
>
> Cross-implementation issues:
>
>  What do the various VMs want/need from CPython to help with their
>  implementations?
>
>  * Marking CPython-specific tests in the test suite?
>  * Getting an implementation agnostic test suite for the Python language?
>  * Separating the language tests and the pure Python part of the stdlib into
>a separate project?  (Or publish them as a separate package.)
>  * Transition plans for 3.0?
>
>  Champion needed.
I would like to champion this one.

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Holding a Python Language Summit at PyCon

On Thu, Dec 4, 2008 at 12:05, Frank Wierzbicki <[EMAIL PROTECTED]> wrote:
> On Wed, Dec 3, 2008 at 10:31 AM, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
>> 14:00 - 15:30
>> =
>>
>> Two tracks:
>>
>> Cross-implementation issues:
>>
>>  What do the various VMs want/need from CPython to help with their
>>  implementations?
>>
>>  * Marking CPython-specific tests in the test suite?
>>  * Getting an implementation agnostic test suite for the Python language?
>>  * Separating the language tests and the pure Python part of the stdlib into
>>a separate project?  (Or publish them as a separate package.)
>>  * Transition plans for 3.0?
>>
>>  Champion needed.
> I would like to champion this one.
>

I told AMK this a while back, but might as well make it more public; I
am up for chairing as well.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Holding a Python Language Summit at PyCon

On Thu, Dec 04, 2008 at 03:05:51PM -0500, Frank Wierzbicki wrote:
> > Cross-implementation issues:
>
> I would like to champion this one.

Thanks!  You're now listed as the champion for it.

--amk
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Paul Moore

2008/12/4 Barry Warsaw <[EMAIL PROTECTED]>:
>>> * that 3.1 will rearrange the standard library in mostly-known ways, and
>>> * that we expect people to use 3.0 mostly for compatibility testing,  not 
>>> going into serious production
>>>   use until 3.1 or maybe even 3.2.
>> The latter statement worries me.  It seems to unnecessarily undermine
>> adoption of 3.0.  It essentially says, "don't use this".  Is that what we
>> want?
>> ISTM, 3.0 is in pretty good shape.  There is nothing intrinsically wrong
>> with it.  The number one adoption issue is external, i.e. how quickly
>> key third-party modules get converted.
>
> I agree.  I tried to put a positive spin on the announcement, and the
> backward compatibility issue in particular.  I probably failed.

Hmm, looking back, the quote Raymond is referring to is just a
suggestion for additional text on the 3.0 page. I agree with him that
it's a bit too negative.

The announcement itself hits just the right note in my view. You
(Barry) seem to have got it pretty well on target.

One thing I'd like to see more clearly stated is that there's no
reason NOT to use Python 3.0 for new code. I don't think that message
has really come across yet - in spite of the warnings being all about
compatibility issues, no-one has stressed the simple point that if
your code is new, it doesn't have compatibility concerns!

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Merging mailing lists

2008-12-04 Thread Paul Moore

2008/12/4 "Martin v. Löwis" <[EMAIL PROTECTED]>:
> Any objections?

The timing is right, go for it.
Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 1:02 PM, Toshio Kuratomi <[EMAIL PROTECTED]> wrote:
> I opened up bug http://bugs.python.org/issue4006 a while ago and it was
> suggested in the report that it's not a bug but a feature and so I
> should come here to see about getting the feature changed :-)
>
> I have a specific problem with os.environ and a somewhat less important
> architectural issue with the unicode/bytes handling in certain os.*
> modules.  I'll start with the important one:
>
> Currently in python3 there's no way to get at environment variables that
> are not encoded in the system default encoding.  My understanding is
> that this isn't a problem on Windows systems but on *nix this is a huge
> problem.  environment variables on *nix are a sequence of non-null
> bytes.  These bytes are almost always "characters" but they do not have
> to be.  Further, there is nothing that requires that the characters be
> in the same encoding; some of the characters could be in the UTF-8
> character set while others are in latin-1, shift-jis, or big-5.

Multiple encoding environments are best described as "batshit insane".
 It's impossible to handle any of it correctly *as text*, which is why
UTF-8 is becoming a universal standard.  For everybody's sanity python
should continue to push it.

However, some pragmatism is also possible.  Many uses of PATH may
allow it to be treated as black-box bytes, rather than text.  The
minimal solution I see is to make os.getenv() and os.putenv() switch
to byte modes when given byte arguments, as os.listdir() does.  This
use case doesn't require the ability to iterate over all environment
variables, as os.environb would allow.

I do wonder if controlling the environment given to a subprocess
requires os.environb, but it may be too obscure to really matter.

-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Jean-Paul Calderone


On Thu, 4 Dec 2008 20:20:34 +, Paul Moore <[EMAIL PROTECTED]> wrote:

2008/12/4 Barry Warsaw <[EMAIL PROTECTED]>:
[snip]

One thing I'd like to see more clearly stated is that there's no
reason NOT to use Python 3.0 for new code. I don't think that message
has really come across yet - in spite of the warnings being all about
compatibility issues, no-one has stressed the simple point that if
your code is new, it doesn't have compatibility concerns!


New code that wouldn't be more easily written with a dependency on a
library that hasn't been ported, you mean.

Although beyond that, there may be reasons (for example, the significant
performance degradation in the I/O library currently being discussed on
python-list).

Jean-Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] RELEASED Python 3.0 final

Terry Reedy wrote:
> and this could give some people a mis-impression, most likely negative,
> as to the magnitude and nature of the change.  Most of the code I am now
> writing would, I believe, run with 2.5 except for print(..., file=xxx).
>  And I know that there was concern for backward compatibility to the
> point that some changes were rejected (renaming builtins) or delayed
> (deleting duplicate test asserts) for that reason.  So I would soften
> the statements to "... version of the language that is partially
> incompatible with... " and "were made without being bound by backward
> compatibility,"

I would agree with Terry - while there are backwards incompatibilities,
they aren't gratuitous.

Then again, Guido does seem to want to discourage people from trying to
target the common subset of the two languages instead of using 2to3 as a
compilation step from the python3 version.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-04 Thread André Malo

* Adam Olsen wrote: 


> On Thu, Dec 4, 2008 at 1:02 PM, Toshio Kuratomi <[EMAIL PROTECTED]> 
wrote:
> > I opened up bug http://bugs.python.org/issue4006 a while ago and it was
> > suggested in the report that it's not a bug but a feature and so I
> > should come here to see about getting the feature changed :-)
> >
> > I have a specific problem with os.environ and a somewhat less important
> > architectural issue with the unicode/bytes handling in certain os.*
> > modules.  I'll start with the important one:
> >
> > Currently in python3 there's no way to get at environment variables
> > that are not encoded in the system default encoding.  My understanding
> > is that this isn't a problem on Windows systems but on *nix this is a
> > huge problem.  environment variables on *nix are a sequence of non-null
> > bytes.  These bytes are almost always "characters" but they do not have
> > to be.  Further, there is nothing that requires that the characters be
> > in the same encoding; some of the characters could be in the UTF-8
> > character set while others are in latin-1, shift-jis, or big-5.
>
> Multiple encoding environments are best described as "batshit insane".
>  It's impossible to handle any of it correctly *as text*, which is why
> UTF-8 is becoming a universal standard.  For everybody's sanity python
> should continue to push it.

Here's an example which will become popular soon, I guess: CGI scripts and, 
of course WSGI applications. All those get their environment in an unknown 
encoding. In the worst case one can blow up the application by simply 
sending strange header lines over the wire. But there's more: consider 
running the server in C locale, then probably even a single 8 bit char 
might break something (?).

> However, some pragmatism is also possible.  Many uses of PATH may
> allow it to be treated as black-box bytes, rather than text.  The
> minimal solution I see is to make os.getenv() and os.putenv() switch
> to byte modes when given byte arguments, as os.listdir() does.  This
> use case doesn't require the ability to iterate over all environment
> variables, as os.environb would allow.
>
> I do wonder if controlling the environment given to a subprocess
> requires os.environb, but it may be too obscure to really matter.

IMHO, environment variables are no text. They are bytes by definition and 
should be treated as such.
I know, there's windows having unicode enabled env vars on demand, but 
there's only trouble with those over there in apache's httpd (when passing 
them to CGI scripts, oh well...).

nd
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

A.M. Kuchling wrote:
> * that 3.1 will rearrange the standard library in mostly-known ways, and 
> * that we expect people to use 3.0 mostly for compatibility testing, 
>   not going into serious production use until 3.1 or maybe even 3.2.

As Raymond notes, this is probably too negative: for new projects, 3.0
should be fine so long as they don't need too many external libraries in
the short term.

For projects migrating from Python 2.x, the 3rd party library support
problem is likely to hold a lot of projects back for several months at
least, possibly to the point where it makes more sense to just wait for
2.7/3.1 to finalise any migration plans.

Such projects are still well-advised to start their porting efforts as
soon as possible though so they can identify *which* of their external
dependencies don't have python 3.0 compatible versions available yet.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

Adam Olsen wrote:
> On Thu, Dec 4, 2008 at 1:02 PM, Toshio Kuratomi <[EMAIL PROTECTED]> wrote:
>> I opened up bug http://bugs.python.org/issue4006 a while ago and it was
>> suggested in the report that it's not a bug but a feature and so I
>> should come here to see about getting the feature changed :-)
>>
>> I have a specific problem with os.environ and a somewhat less important
>> architectural issue with the unicode/bytes handling in certain os.*
>> modules.  I'll start with the important one:
>>
>> Currently in python3 there's no way to get at environment variables that
>> are not encoded in the system default encoding.  My understanding is
>> that this isn't a problem on Windows systems but on *nix this is a huge
>> problem.  environment variables on *nix are a sequence of non-null
>> bytes.  These bytes are almost always "characters" but they do not have
>> to be.  Further, there is nothing that requires that the characters be
>> in the same encoding; some of the characters could be in the UTF-8
>> character set while others are in latin-1, shift-jis, or big-5.
> 
> Multiple encoding environments are best described as "batshit insane".
>  It's impossible to handle any of it correctly *as text*, which is why
> UTF-8 is becoming a universal standard.  For everybody's sanity python
> should continue to push it.
> 
Amen brother!

> However, some pragmatism is also possible.

Unfortunately, this is exactly what I'm talking about :-)

>  Many uses of PATH may
> allow it to be treated as black-box bytes, rather than text.  The
> minimal solution I see is to make os.getenv() and os.putenv() switch
> to byte modes when given byte arguments, as os.listdir() does.  This
> use case doesn't require the ability to iterate over all environment
> variables, as os.environb would allow.
> 
This would be a partial implementation of my option #3.  It allows the
programmer to workaround problems but does allow subtle bugs to creep in
unawares.  For instance::

> I do wonder if controlling the environment given to a subprocess
> requires os.environb, but it may be too obscure to really matter.
> 
If you wanted to change one variable before passing it on to the
subprocess this could lead to head-scratcher bugs.  Here's a contrived
example:  Say I have an app that talks to multiple cvs repositories.  It
copies os.environ and modifies CVSROOT and CVS_RSH then calls subprocess
with env=temp_env.  If the PATH variable contains non-decodable elements
on some machines, this could lead to mysterious failures.  This is
particularly bad because we aren't directly modifying PATH anywhere in
our code so there won't be an obvious reason in the code that this is
failing.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

Toshio Kuratomi wrote:
> The bug report I opened suggests creating a PEP to address this issue.
> I think that's a good idea for whether os.listdir() and friends should
> be changed to raise an exception but not having any way to get at some
> environment variables seems like it's just a bug that needs to be
> addressed.  What do other people think on both these issues?

I'm pretty sure the discussion on this topic a while back decided that
where necessary Python 3 would grow parallel bytes versions of APIs
affected by environmental encoding issues (such as os.environb,
os.listdirb, os.getcwdb), but that we were OK with the idea of deferring
addition of those APIs until 3.1.

That is, this was an acknowledged limitation with a fairly
straightforward agreed solution, but it wasn't considered a common
enough issue to delay the release of 3.0 until all of those parallel
APIs had been implemented

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Georg Brandl

Benjamin Peterson schrieb:
> On Thu, Dec 4, 2008 at 12:52 PM, Eric Smith <[EMAIL PROTECTED]> wrote:
>> Christian Heimes wrote:
>>>
>>> Several people have asked about the patch and merge flow. Now that Python
>>> 3.0 is out it's a bit more complicated.
>>>
>>> Flow diagram
>>> 
>>>
>>> trunk ---> release26-maint
>>>   \->  py3k   ---> release30-maint
>>>
>>>
>>> Patches for all versions of Python should land in the trunk. They are then
>>> merged into release26-maint and py3k branches. Changes for Python 3.0 are
>>> merged via the py3k branch.
>>
>> Apologies if this has been discussed before. I looked but didn't see
>> anything.
>>
>> Given that at least 99% of the changes for the trunk will not get merged
>> into release26-maint, doesn't it make more sense to merge the other way?
>> That is, anything that gets checked in to release26-maint would potentially
>> be merged into trunk. That would remove the huge number of merge blocks that
>> will otherwise be required. Same fore py3k and release30-maint.

I've suggested that too; the counter-argument was that "most people don't
want to care in which branch to commit something".  I'm not too comfortable
with this argument as it implies a certain ignorance on the part of our
committers.  As Fred says, it wasn't discussed anyway.

Also, with svnmerge, it is not too late to change merging direction.

> I think the percentage is a bit lower than that. Also, we haven't been
> using blocking with the maintenance branch so far; svnmerge.py is just
> a convenience. (It generates commit messages and has a simpler
> interface than a simple "svn merge" command.)

I *did* use blocking with the 2.6 branch when I last merged a whole batch
of commits.  As you say, by using svnmerge without blocking we only get a
tool that can generate commit messages.  However, with blocking we get
something more valuable: we don't overlook backportable fixes anymore.

So: yes, blocking is more work, but it gives something important in return.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

2008-12-04 Thread Georg Brandl

Christian Heimes schrieb:
> Several people have asked about the patch and merge flow. Now that 
> Python 3.0 is out it's a bit more complicated.
> 
> Flow diagram
> 
> 
> trunk ---> release26-maint
> \->  py3k   ---> release30-maint
> 
> 
> Patches for all versions of Python should land in the trunk. They are 
> then merged into release26-maint and py3k branches. Changes for Python 
> 3.0 are merged via the py3k branch.

As a side-note: this merging flow means that bugfix and feature commits
may never be merged from trunk to py3k in one svnmerge batch.  Else,
they cannot be separated when merging from py3k to 30-maint.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

On Thu, Dec 04, 2008 at 08:20:34PM +, Paul Moore wrote:
> Hmm, looking back, the quote Raymond is referring to is just a
> suggestion for additional text on the 3.0 page. I agree with him that
> it's a bit too negative.

Actually I want it to be an entirely separate page so that we can
point people to it.

> has really come across yet - in spite of the warnings being all about
> compatibility issues, no-one has stressed the simple point that if
> your code is new, it doesn't have compatibility concerns!

Well, at least not until you decide you need some particular external
library that hasn't been ported to 3.0 yet.

For example, if you go to discussion threads such as
,
you can see people making statements like "I've been holding off
learning it until 3000 went gold."

But I think starting with Python 3.0 is a bad idea for a newbie,
because they'll be limited in what they can do until the libraries
have been ported.  They can do some tasks (command-line tools,
Fibonacci functions, Tk GUIs), but can they use the fancy new web
framework they've just read about?  Write a game?  Draw graphs with
matplotlib?  Use and extend an application such as Roundup?  Bzzt, no,
not yet!

Starting with 3.0 is starting out on an island.  While I expect the
island will grow in territory over time, I'm worried that new learners
will automatically go for the highest version number, find their
available tools are highly restricted, and get frustrated.

Perhaps the statement could say something like "we do not expect
most Python packages will be ported to the 3.x series until 
around the time 3.1 is released in X months."  (where X=12?  6?)

--amk
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Merging mailing lists

2008-12-04 Thread Dmitry Vasiliev

Martin v. Löwis wrote:
> I would like to merge mailing lists, now that the design and first
> implementation of Python 3000 is complete. In particular, I would

+1

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
  http://hlabs.spb.ru
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 2:09 PM, André Malo <[EMAIL PROTECTED]> wrote:
> * Adam Olsen wrote:
>> On Thu, Dec 4, 2008 at 1:02 PM, Toshio Kuratomi <[EMAIL PROTECTED]>
> wrote:
>> > I opened up bug http://bugs.python.org/issue4006 a while ago and it was
>> > suggested in the report that it's not a bug but a feature and so I
>> > should come here to see about getting the feature changed :-)
>> >
>> > I have a specific problem with os.environ and a somewhat less important
>> > architectural issue with the unicode/bytes handling in certain os.*
>> > modules.  I'll start with the important one:
>> >
>> > Currently in python3 there's no way to get at environment variables
>> > that are not encoded in the system default encoding.  My understanding
>> > is that this isn't a problem on Windows systems but on *nix this is a
>> > huge problem.  environment variables on *nix are a sequence of non-null
>> > bytes.  These bytes are almost always "characters" but they do not have
>> > to be.  Further, there is nothing that requires that the characters be
>> > in the same encoding; some of the characters could be in the UTF-8
>> > character set while others are in latin-1, shift-jis, or big-5.
>>
>> Multiple encoding environments are best described as "batshit insane".
>>  It's impossible to handle any of it correctly *as text*, which is why
>> UTF-8 is becoming a universal standard.  For everybody's sanity python
>> should continue to push it.
>
> Here's an example which will become popular soon, I guess: CGI scripts and,
> of course WSGI applications. All those get their environment in an unknown
> encoding. In the worst case one can blow up the application by simply
> sending strange header lines over the wire. But there's more: consider
> running the server in C locale, then probably even a single 8 bit char
> might break something (?).

I think that's an argument that the framework should reencode all
input text into the correct system encoding before passing it on to
the CGI script or WSGI app.  If the framework doesn't have a clear way
to determine the client's encoding then it's all just gibberish
anyway.  A HTTP 400 or 500 range error code is appropriate here.


>> However, some pragmatism is also possible.  Many uses of PATH may
>> allow it to be treated as black-box bytes, rather than text.  The
>> minimal solution I see is to make os.getenv() and os.putenv() switch
>> to byte modes when given byte arguments, as os.listdir() does.  This
>> use case doesn't require the ability to iterate over all environment
>> variables, as os.environb would allow.
>>
>> I do wonder if controlling the environment given to a subprocess
>> requires os.environb, but it may be too obscure to really matter.
>
> IMHO, environment variables are no text. They are bytes by definition and
> should be treated as such.
> I know, there's windows having unicode enabled env vars on demand, but
> there's only trouble with those over there in apache's httpd (when passing
> them to CGI scripts, oh well...).

Environment variables have textual names, are set via text, frequently
contain textual file names or paths, and my shell (bash in
gnome-terminal on ubuntu) lets me put unicode text in just fine.  The
underlying APIs may use bytes, but they're *intended* to be encoded
text.


-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Raymond Hettinger


From: "A.M. Kuchling" <[EMAIL PROTECTED]>

Perhaps the statement could say something like "we do not expect
most Python packages will be ported to the 3.x series until 
around the time 3.1 is released in X months."  (where X=12?  6?)


I would leave out any discussion of 3.1.  Its content and release date
have nothing to do with when third party modules get updated.

Also, we don't know the timing of the third-party updates.
Some may never get converted.  Some may convert quickly
and easily.  Someone (perhaps me) may organize a series of
funded sprints to get many of the major packages converted.

It would be better to simply say that at the present time,
most important third-party modules do not yet support 3.0.

FWIW, my father is Python newbie and I'm pointing him
to 3.0 because it will be easier to learn than 2.6's hodgepodge
of new and old features.  The 3.0 environment is much cleaner.



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

On Thu, Dec 4, 2008 at 13:21, Georg Brandl <[EMAIL PROTECTED]> wrote:
> Benjamin Peterson schrieb:
>> On Thu, Dec 4, 2008 at 12:52 PM, Eric Smith <[EMAIL PROTECTED]> wrote:
>>> Christian Heimes wrote:

 Several people have asked about the patch and merge flow. Now that Python
 3.0 is out it's a bit more complicated.

 Flow diagram
 

 trunk ---> release26-maint
   \->  py3k   ---> release30-maint


 Patches for all versions of Python should land in the trunk. They are then
 merged into release26-maint and py3k branches. Changes for Python 3.0 are
 merged via the py3k branch.
>>>
>>> Apologies if this has been discussed before. I looked but didn't see
>>> anything.
>>>
>>> Given that at least 99% of the changes for the trunk will not get merged
>>> into release26-maint, doesn't it make more sense to merge the other way?
>>> That is, anything that gets checked in to release26-maint would potentially
>>> be merged into trunk. That would remove the huge number of merge blocks that
>>> will otherwise be required. Same fore py3k and release30-maint.
>
> I've suggested that too; the counter-argument was that "most people don't
> want to care in which branch to commit something".  I'm not too comfortable
> with this argument as it implies a certain ignorance on the part of our
> committers.  As Fred says, it wasn't discussed anyway.
>

That would make the rule for choosing which branch to first commit to
be the one with the smallest version:

2.6 -> trunk -> 3.0 -> py3k

That seems reasonable to me since that is really what the code
branching is and how I suspect we will do things with a DVCS.

> Also, with svnmerge, it is not too late to change merging direction.
>

If changing it to be like above is not an issue then I vote for the change.

>> I think the percentage is a bit lower than that. Also, we haven't been
>> using blocking with the maintenance branch so far; svnmerge.py is just
>> a convenience. (It generates commit messages and has a simpler
>> interface than a simple "svn merge" command.)
>
> I *did* use blocking with the 2.6 branch when I last merged a whole batch
> of commits.  As you say, by using svnmerge without blocking we only get a
> tool that can generate commit messages.  However, with blocking we get
> something more valuable: we don't overlook backportable fixes anymore.
>
> So: yes, blocking is more work, but it gives something important in return.

The other perk of this ordering is you should be able to place a
single block along the chain where the patch should stop and
potentially be done with the merges if you are in a rush. That way
people who do mass merges can just sequentially merge and not worry
about where a patch should stop.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

Toshio Kuratomi wrote:

It does you no good and (and will irritate others) to conflate 'design
decision I do not agree with' with 'mistaken documentation or
implementation of a design decision'. The former is opinion, the latter
is usually fact (with occasional border cases). The latter is what core
developers mean by 'bug'.

To me, mixing encodings within a string is at least slightly insane. If
by design, maybe even a 'design bug' ;-).

These mixed encodings can occur for a variety of reasons. Here's an
example that isn't too contrived :-)

The system administrator of the shell server adds the directory of
programs to all his user's default PATH variables so then they have this:

PATH=/bin:/usr/bin:/usr/local/bin:/mnt/\xe3\x83\x8d\xe3\x83\x83\xe3\x83\x88\xe3\x83\xaf\xe3\x83\xbc\xe3\x82\xaf/\x83v\x83\x8d\x83O\x83\x89\x83\x80

I would think life would be ultimately easier if either the file server
or the shell server automatically translated file names from jis and
utf8 and back, so that the PATH on the *nix shell server is entirely
utf8. How would you ever display a mixture to users? What if there
were an ambiguous component that could be legally decoded more than one way?

Now comes the problematic part. One of the user's on the system wants
to write a python3 program that needs to determine if a needed program
is in the user's PATH. He tries to code it like this::

#!/usr/bin/python3.0

import os

for directory in os.environ['PATH']:
programs = os.listdir(directory)

Have you tried os.system or os.popen or the subprocess module to use and
get a response from a native *nix command? On Windows

>>> import subprocess as sp
>>> s=sp.Popen('path', shell=True, stdout=sp.PIPE)
>>> s.stdout.read()
b'PATH=C:\\temp\\WatconPermanent\\binnt;C:\\temp\\WatconPermanent\\binw;C:\\WINDOWS\\System32;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\Program
Files\\PC-Doctor for Windows\\services;C:\\Program Files\\ATI
Technologies\\ATI.ACE\\Core-Static;C:\\Program
Files\\Python25;C:\\Program Files\\QuickTime\\QTSystem\\\r\n'

There are the bytes. This took me 10 minutes and a few mistakes as a
first time subprocess user.

Another 10 minutes and I figured out how to get the entire environment
as bytes *and* convert them to a dict. This is a bit trickier

s=sp.Popen('set', shell=True, stdout=sp.PIPE) #null set (env) cmd gets
e1= s.stdout.read()
e2=e1.split(b'\r\n')
e2.pop() # get rid of trailing b'' from trailing '\r\n'
e3=[i.split(b'=') for i in e2]
env = dict(e3)

Whether either of these should be wrapped in os, I'll leave for others
to discuss and decide, but if you can do the same in *nix, you should be
able to do what you need to for now.

Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 2:19 PM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Toshio Kuratomi wrote:
>> The bug report I opened suggests creating a PEP to address this issue.
>> I think that's a good idea for whether os.listdir() and friends should
>> be changed to raise an exception but not having any way to get at some
>> environment variables seems like it's just a bug that needs to be
>> addressed.  What do other people think on both these issues?
>
> I'm pretty sure the discussion on this topic a while back decided that
> where necessary Python 3 would grow parallel bytes versions of APIs
> affected by environmental encoding issues (such as os.environb,
> os.listdirb, os.getcwdb), but that we were OK with the idea of deferring
> addition of those APIs until 3.1.

It looks like most of them got into 3.0.
http://docs.python.org/3.0/library/os.html says "All functions
accepting path or file names accept both bytes and string objects, and
result in an object of the same type, if a path or file name is
returned."


> That is, this was an acknowledged limitation with a fairly
> straightforward agreed solution, but it wasn't considered a common
> enough issue to delay the release of 3.0 until all of those parallel
> APIs had been implemented

Aye.  IMO it's fairly clear that os.getenv()/os.putenv() should follow
suit in 3.1.  I'm not so sure about adding os.environb (and making
subprocess use it), unless the OP can demonstrate they really need it.

-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] RELEASED Python 3.0 final

On Thu, Dec 4, 2008 at 13:07, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Terry Reedy wrote:
>> and this could give some people a mis-impression, most likely negative,
>> as to the magnitude and nature of the change.  Most of the code I am now
>> writing would, I believe, run with 2.5 except for print(..., file=xxx).
>>  And I know that there was concern for backward compatibility to the
>> point that some changes were rejected (renaming builtins) or delayed
>> (deleting duplicate test asserts) for that reason.  So I would soften
>> the statements to "... version of the language that is partially
>> incompatible with... " and "were made without being bound by backward
>> compatibility,"
>
> I would agree with Terry - while there are backwards incompatibilities,
> they aren't gratuitous.
>
> Then again, Guido does seem to want to discourage people from trying to
> target the common subset of the two languages instead of using 2to3 as a
> compilation step from the python3 version.
>

It makes sense if your code would have required jumping through hoops
to keep the base use-case. But if the only major difference is
something easily covered by a __future__ statement (think
print_function or unicode_literals, I believe although that __future__
statement is not documented anywhere according to Google), then I
honestly think it's okay to try to target the subset.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

Adam Olsen wrote:
> On Thu, Dec 4, 2008 at 2:09 PM, André Malo <[EMAIL PROTECTED]> wrote:
>> * Adam Olsen wrote:
>>> On Thu, Dec 4, 2008 at 1:02 PM, Toshio Kuratomi <[EMAIL PROTECTED]>
>> wrote:
 I opened up bug http://bugs.python.org/issue4006 a while ago and it was
 suggested in the report that it's not a bug but a feature and so I
 should come here to see about getting the feature changed :-)

 I have a specific problem with os.environ and a somewhat less important
 architectural issue with the unicode/bytes handling in certain os.*
 modules.  I'll start with the important one:

 Currently in python3 there's no way to get at environment variables
 that are not encoded in the system default encoding.  My understanding
 is that this isn't a problem on Windows systems but on *nix this is a
 huge problem.  environment variables on *nix are a sequence of non-null
 bytes.  These bytes are almost always "characters" but they do not have
 to be.  Further, there is nothing that requires that the characters be
 in the same encoding; some of the characters could be in the UTF-8
 character set while others are in latin-1, shift-jis, or big-5.
>>> Multiple encoding environments are best described as "batshit insane".
>>>  It's impossible to handle any of it correctly *as text*, which is why
>>> UTF-8 is becoming a universal standard.  For everybody's sanity python
>>> should continue to push it.
>> Here's an example which will become popular soon, I guess: CGI scripts and,
>> of course WSGI applications. All those get their environment in an unknown
>> encoding. In the worst case one can blow up the application by simply
>> sending strange header lines over the wire. But there's more: consider
>> running the server in C locale, then probably even a single 8 bit char
>> might break something (?).
> 
> I think that's an argument that the framework should reencode all
> input text into the correct system encoding before passing it on to
> the CGI script or WSGI app.  If the framework doesn't have a clear way
> to determine the client's encoding then it's all just gibberish
> anyway.  A HTTP 400 or 500 range error code is appropriate here.
> 
The framework can't always encode input bytes into the system encoding
for text.  Sometimes the framework can be dealing with actual bytes.
For instance, if the framework is being asked to reference an actual
file on a *NIX filesystem the bytes have to match up with the bytes in
the filename whether or not those bytes agree with the system encoding.

> 
>>> However, some pragmatism is also possible.  Many uses of PATH may
>>> allow it to be treated as black-box bytes, rather than text.  The
>>> minimal solution I see is to make os.getenv() and os.putenv() switch
>>> to byte modes when given byte arguments, as os.listdir() does.  This
>>> use case doesn't require the ability to iterate over all environment
>>> variables, as os.environb would allow.
>>>
>>> I do wonder if controlling the environment given to a subprocess
>>> requires os.environb, but it may be too obscure to really matter.
>> IMHO, environment variables are no text. They are bytes by definition and
>> should be treated as such.
>> I know, there's windows having unicode enabled env vars on demand, but
>> there's only trouble with those over there in apache's httpd (when passing
>> them to CGI scripts, oh well...).
> 
> Environment variables have textual names, are set via text, frequently
> contain textual file names or paths, and my shell (bash in
> gnome-terminal on ubuntu) lets me put unicode text in just fine.  The
> underlying APIs may use bytes, but they're *intended* to be encoded
> text.
> 
The example I've started using recently is this: text files on my system
contain character data and I expect them to be read into a string type
when I open them in python3.  However, if a text file contains text that
is not encoded in the system default encoding I should still be able to
get at the data and perform my own conversion.  So I agree with the
default of treating environment variables as text.  We just need to be
able to treat them as bytes when these corner cases come up.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-04 Thread André Malo

* Adam Olsen wrote: 

> On Thu, Dec 4, 2008 at 2:09 PM, André Malo <[EMAIL PROTECTED]> wrote:

> > Here's an example which will become popular soon, I guess: CGI scripts
> > and, of course WSGI applications. All those get their environment in an
> > unknown encoding. In the worst case one can blow up the application by
> > simply sending strange header lines over the wire. But there's more:
> > consider running the server in C locale, then probably even a single 8
> > bit char might break something (?).
>
> I think that's an argument that the framework should reencode all
> input text into the correct system encoding before passing it on to
> the CGI script or WSGI app.  If the framework doesn't have a clear way
> to determine the client's encoding then it's all just gibberish
> anyway.  A HTTP 400 or 500 range error code is appropriate here.

Duh.
See, you're already mixing different encodings and creating issues here! 
You're talking about client encoding (whatever that is) with correct system 
encoding (whatever that is, too) in the same paragraph and assume they are 
the same or compatible.

There are several points here:

- there is no clear way to get a single client encoding for the whole HTTP 
  transaction (headers + body), because *there is none*. If the whole 
  header set matches the same encoding, it's more or less luck.

- there is no correct system encoding either. As said, I prefer running my 
  servers in C locale, so it's all ascii. In fact, it shouldn't matter. The 
  locale should not have anything to do with an application called over the 
  network.

- A 400 or 500 response for a header containing something like my name is 
  not appropriate.

- Octets in HTTP headers are allowed. And they are what they are -
  octets. The interpretation has to be left to the application, not the 
  framework.


>
> >> However, some pragmatism is also possible.  Many uses of PATH may
> >> allow it to be treated as black-box bytes, rather than text.  The
> >> minimal solution I see is to make os.getenv() and os.putenv() switch
> >> to byte modes when given byte arguments, as os.listdir() does.  This
> >> use case doesn't require the ability to iterate over all environment
> >> variables, as os.environb would allow.
> >>
> >> I do wonder if controlling the environment given to a subprocess
> >> requires os.environb, but it may be too obscure to really matter.
> >
> > IMHO, environment variables are no text. They are bytes by definition
> > and should be treated as such.
> > I know, there's windows having unicode enabled env vars on demand, but
> > there's only trouble with those over there in apache's httpd (when
> > passing them to CGI scripts, oh well...).
>
> Environment variables have textual names, are set via text, frequently

Well, think about my example again. The friendly way to maintain them is not 
the issue. The problems arise at least when the variables are set by an 
attacker.

> contain textual file names or paths, and my shell (bash in
> gnome-terminal on ubuntu) lets me put unicode text in just fine.  The
> underlying APIs may use bytes, but they're *intended* to be encoded
> text.

Yes, encoded text == bytes. No, they're intended to be c-strings. And well,  
even if we assume that they should contain text (as in encoded unicode), 
their meaning is application specific and so is the encoding (even if it's 
mixed).

What I'm saying is: I don't see much use for unicode APIs for the 
environment at all, because I don't know what's in there before inspecting 
them. And apparently the only reliable way to inspect them is via a byte 
oriented API.

nd
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

Terry Reedy wrote:
> Toshio Kuratomi wrote:
>> I opened up bug http://bugs.python.org/issue4006 a while ago and it was
>> suggested in the report that it's not a bug but a feature and so I
>> should come here to see about getting the feature changed :-)
> 
> It does you no good and (and will irritate others) to conflate 'design
> decision I do not agree with' with 'mistaken documentation or
> implementation of a design decision'.  The former is opinion, the latter
> is usually fact (with occasional border cases).  The latter is what core
> developers mean by 'bug'.
> 
Noted.  However, there's also a difference between "Prevents us from
doing useful things" and "Allows doing a useful thing in a non-trivial
manner".  The latter I would call a difference in design decision and
the former I would call a bug in the design.

>> Currently in python3 there's no way to get at environment variables that
>> are not encoded in the system default encoding.  My understanding is
>> that this isn't a problem on Windows systems but on *nix this is a huge
>> problem.  environment variables on *nix are a sequence of non-null
>> bytes.  These bytes are almost always "characters" but they do not have
>> to be.  Further, there is nothing that requires that the characters be
>> in the same encoding; some of the characters could be in the UTF-8
>> character set while others are in latin-1, shift-jis, or big-5.
> 
> To me, mixing encodings within a string is at least slightly insane.  If
> by design, maybe even a 'design bug' ;-).
> 
As an application level developer I echo your sentiment :-)  I
recognize, though, that *nix filesystem semantics were designed many
years before unicode and the decision to treat filenames, environment
variables, and so much else as bytes follows naturally from the C
definition of a char.  It's up to a higher level than the OS to decide
how to displa6 the bytes.

[shell server and fileserver result in this insane PATH]
>> PATH=/bin:/usr/bin:/usr/local/bin:/mnt/\xe3\x83\x8d\xe3\x83\x83\xe3\x83\x88\xe3\x83\xaf\xe3\x83\xbc\xe3\x82\xaf/\x83v\x83\x8d\x83O\x83\x89\x83\x80
>>
> 
> I would think life would be ultimately easier if either the file server
> or the shell server automatically translated file names from jis and
> utf8 and back, so that the PATH on the *nix shell server is entirely
> utf8.

This is not possible because no part of the computer knows what the
encoding is.  To the computer, it's just a sequence of bytes.  Unlike
xml or the windows filesystem (winfs? ntfs?) where the encoding is
specified as part of the document/filesystem there's nothing to tell
what encoding the filenames are in.

>  How would you ever display a mixture to users?

This is up to the application.  My recomendation would be to keep the
raw bytes (to access the file on the filesystem) and display the results
of str(filename, errors='replace') to the user.

>  What if there
> were an ambiguous component that could be legally decoded more than one
> way?
> 
The ambiguity is the reason that the fileserver and shell server can't
automatically translate the filename (many encodings merely use all of
the 2^8 byte combinations available in a C char type.  This makes the
byte decodable in any one of those encodings).  In the application, only
using the raw bytes to access the file also prevents ambiguity because
the raw bytes only references one file.

>> Now comes the problematic part.  One of the user's on the system wants
>> to write a python3 program that needs to determine if a needed program
>> is in the user's PATH.  He tries to code it like this::
>>
>> #!/usr/bin/python3.0
>>
>> import os
>>
>> for directory in os.environ['PATH']:
>> programs = os.listdir(directory)
>>
>> That code raises a KeyError because python3 has silently discarded the
>> PATH due to the shift-jis encoded path elements.  Much more importantly,
>> there's no way the programmer can handle the KeyError and actually get
>> the PATH from within python.
> 
> Have you tried os.system or os.popen or the subprocess module to use and
> get a response from a native *nix command?  On Windows
> 
Sure, you can subprocess your way out of a lot of sticky situations
since you're essentially delegating the task to a C routine.  But there
are drawbacks:

* You become dependent on an external program being available.  What
happens if your code is run in a chroot, for instance?
* Do we want anyone writing programs that access the environment on *NIX
to have to discover this pattern themselves and implement it?

As for wrapping this up in os.*, that isn't necessary -- the python3
interpreter already knows about the byte-oriented environment; it just
isn't making it available to people programming in python.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Paul Moore

2008/12/4 Raymond Hettinger <[EMAIL PROTECTED]>:
> Also, we don't know the timing of the third-party updates.
> Some may never get converted.  Some may convert quickly
> and easily.  Someone (perhaps me) may organize a series of
> funded sprints to get many of the major packages converted.

One piece of encouraging news I heard today is that mod_wsgi
apparently works with 3.0 already - which may well mean that more web
software than I'd originally anticipated will work sooner rather than
later.

But it's certainly true that Python (all versions, not just 3.0) is
more of an ecosystem than just the CPython core. "Batteries included"
notwithstanding. And it'll take longer for the 3.0 ecosystem to grow
than the 2.6 one.

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 3:47 PM, André Malo <[EMAIL PROTECTED]> wrote:
> * Adam Olsen wrote:
>
>> On Thu, Dec 4, 2008 at 2:09 PM, André Malo <[EMAIL PROTECTED]> wrote:
>
>> > Here's an example which will become popular soon, I guess: CGI scripts
>> > and, of course WSGI applications. All those get their environment in an
>> > unknown encoding. In the worst case one can blow up the application by
>> > simply sending strange header lines over the wire. But there's more:
>> > consider running the server in C locale, then probably even a single 8
>> > bit char might break something (?).
>>
>> I think that's an argument that the framework should reencode all
>> input text into the correct system encoding before passing it on to
>> the CGI script or WSGI app.  If the framework doesn't have a clear way
>> to determine the client's encoding then it's all just gibberish
>> anyway.  A HTTP 400 or 500 range error code is appropriate here.
>
> Duh.
> See, you're already mixing different encodings and creating issues here!
> You're talking about client encoding (whatever that is) with correct system
> encoding (whatever that is, too) in the same paragraph and assume they are
> the same or compatible.

Mixing can work so long as the encoding is clearly specified and
unambiguous.  It limits your character set to a common subset of both
encodings, but that's a lesser problem.


> There are several points here:
>
> - there is no clear way to get a single client encoding for the whole HTTP
>  transaction (headers + body), because *there is none*. If the whole
>  header set matches the same encoding, it's more or less luck.

If there is no way, via official standards or defacto standards, you
should assume ascii and blow up if anything else is given.  At that
point it's meaningless garbage anyway.


> - there is no correct system encoding either. As said, I prefer running my
>  servers in C locale, so it's all ascii. In fact, it shouldn't matter. The
>  locale should not have anything to do with an application called over the
>  network.

I half agree: the network should be unaffected by the C locale.
However, using a C locale should limit you to ascii file names and
environment variables.


> - A 400 or 500 response for a header containing something like my name is
>  not appropriate.
>
> - Octets in HTTP headers are allowed. And they are what they are -
>  octets. The interpretation has to be left to the application, not the
>  framework.

If there is no clear interpretation then they're garbage.  If there is
a clear interpretation it could be done just as well in the framework,
which also lets all the apps benefit from a single implementation,
rather than trying to reimplement it for each one.


>> >> However, some pragmatism is also possible.  Many uses of PATH may
>> >> allow it to be treated as black-box bytes, rather than text.  The
>> >> minimal solution I see is to make os.getenv() and os.putenv() switch
>> >> to byte modes when given byte arguments, as os.listdir() does.  This
>> >> use case doesn't require the ability to iterate over all environment
>> >> variables, as os.environb would allow.
>> >>
>> >> I do wonder if controlling the environment given to a subprocess
>> >> requires os.environb, but it may be too obscure to really matter.
>> >
>> > IMHO, environment variables are no text. They are bytes by definition
>> > and should be treated as such.
>> > I know, there's windows having unicode enabled env vars on demand, but
>> > there's only trouble with those over there in apache's httpd (when
>> > passing them to CGI scripts, oh well...).
>>
>> Environment variables have textual names, are set via text, frequently
>
> Well, think about my example again. The friendly way to maintain them is not
> the issue. The problems arise at least when the variables are set by an
> attacker.

Maintaining them *IS* the issue.  The whole reason they're text in the
first place is to display them to and receive them back from the user.
 Otherwise we'd use meaningless serial numbers for directories or
something.

It may not seem to matter in this use case, but that's only because
they're communicated to/from the user on another system.


>> contain textual file names or paths, and my shell (bash in
>> gnome-terminal on ubuntu) lets me put unicode text in just fine.  The
>> underlying APIs may use bytes, but they're *intended* to be encoded
>> text.
>
> Yes, encoded text == bytes. No, they're intended to be c-strings. And well,
> even if we assume that they should contain text (as in encoded unicode),
> their meaning is application specific and so is the encoding (even if it's
> mixed).
>
> What I'm saying is: I don't see much use for unicode APIs for the
> environment at all, because I don't know what's in there before inspecting
> them. And apparently the only reliable way to inspect them is via a byte
> oriented API.

If you don't think your paths should contain text then please alter
your other systems to stop using japanese

Re: [Python-Dev] Python-3.0, unicode, and os.environ

Adam Olsen wrote:
> On Thu, Dec 4, 2008 at 2:19 PM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
>> Toshio Kuratomi wrote:
>>> The bug report I opened suggests creating a PEP to address this issue.
>>> I think that's a good idea for whether os.listdir() and friends should
>>> be changed to raise an exception but not having any way to get at some
>>> environment variables seems like it's just a bug that needs to be
>>> addressed.  What do other people think on both these issues?
>> I'm pretty sure the discussion on this topic a while back decided that
>> where necessary Python 3 would grow parallel bytes versions of APIs
>> affected by environmental encoding issues (such as os.environb,
>> os.listdirb, os.getcwdb), but that we were OK with the idea of deferring
>> addition of those APIs until 3.1.
> 
> It looks like most of them got into 3.0.
> http://docs.python.org/3.0/library/os.html says "All functions
> accepting path or file names accept both bytes and string objects, and
> result in an object of the same type, if a path or file name is
> returned."
> 
  I'm very glad this is coming along.  Just want to make sure the
environment is also handled in 3.1.
> 
>> That is, this was an acknowledged limitation with a fairly
>> straightforward agreed solution, but it wasn't considered a common
>> enough issue to delay the release of 3.0 until all of those parallel
>> APIs had been implemented
> 
> Aye.  IMO it's fairly clear that os.getenv()/os.putenv() should follow
> suit in 3.1.  I'm not so sure about adding os.environb (and making
> subprocess use it), unless the OP can demonstrate they really need it.
> 
Note: subprocess currently uses the "real" environment (the raw
environment as given to the python interpreter) when it is started
without the `env` parameter.  So the question would be what people
overriding the env parameter on their own need to do.

To be non-surprising I'd think they'd want to have a way to override
just a few variables from the raw environment.  Otherwise you have to
know which variables the program you're calling relies on and make sure
that those are set or call os.getenvb() to retrieve the byte version and
add it to your copy of os.environ before passing that to subprocess.

One example of something that would be even harder to implement without
access to the os.environb dictionary would be writing a program that
wraps make.  Since make takes all the variables from the environment and
transforms them into make variables you need to pass everything from the
environment that you are not modifying into the command.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

>> I can't find any docs built for Python 3.0 (not 3.1a0). 
> 
> The Windows installation has new 3.0 doc dated Dec 3, so it was built,
> just not posted correctly.

That doesn't mean very much. I built it on my local machine. Anybody
with subversion and python could do that; the documentation is in
subversion.

Whether or not it appears on the web site as part of the release
process is an entirely different matter. It used to be that the
doc maintainer (Fred Drake) was part of the release team and release
process. I think Georg is complaining that he is release maintainer,
but not part of the release process.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

> ISTM, 3.0 is in pretty good shape.  There is nothing intrinsically wrong
> with it.

I think it has many bugs, some known before the release, but many more
yet to show up. I agree that the design is good; the implementation will
certainly improve (I deliberately didn't say "could have been better",
because it could not have been better, given the time available to the
contributors).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Barry Warsaw


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Dec 4, 2008, at 6:21 PM, Martin v. Löwis wrote:


I can't find any docs built for Python 3.0 (not 3.1a0).


The Windows installation has new 3.0 doc dated Dec 3, so it was  
built,

just not posted correctly.


That doesn't mean very much. I built it on my local machine. Anybody
with subversion and python could do that; the documentation is in
subversion.

Whether or not it appears on the web site as part of the release
process is an entirely different matter. It used to be that the
doc maintainer (Fred Drake) was part of the release team and release
process. I think Georg is complaining that he is release maintainer,
but not part of the release process.


I've asked Georg to update PEP 101 to make his role as Documentation  
Expert explicit.  Unfortunately we only debug major releases once (or  
twice) every 18 months.  But next time, we'll get that part right for  
sure!


In the meantime, I'll make sure Georg is involved in point releases  
moving forward.


- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSThm8HEjvBPtnXfVAQJgGgP/eiAUroHbxvpJLT8JRpW5H+nmyU5yGGCY
NZYrU/Vm2vRPFyfDevOFErQX9Jr1LqO0x4Qgxm4PpIj3OVwM16INz98as6nONEhC
MfTjf8Pp7f5BrF7HYh1XfPqTy5qpVhAkzKrCcjUk2VT/JHzJ4wrAl+29VhDTjvrz
3SXphnxWi6w=
=dfm7
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Taint Mode in Python 3.0

2008-12-04 Thread Amaury Forgeot d'Arc

Hello,

On Thu, Dec 4, 2008 at 19:36, Nicole King <[EMAIL PROTECTED]> wrote:
> Dear All,
>
> I have published the diff for my implementation of tainted mode in Python for
> R3.0 (released version) at http://www.cats-muvva.net/software/. Look at the
> bottom the page. I apologise for past problems accessing this web site: I
> hope to have resolved all the issues with it.

The patch is indeed huge! it seems that every function that returns a
PyObject must be modified.
And it seems very difficult to check for its correctness.

Did you look at the Pypy project? The C code of the interpreter is
generated, and it already proposes a "Taint" option at translation
time.
http://codespeak.net/pypy/dist/pypy/doc/objspace-proxies.html#taint
With only 300 lines of elegant python code...

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow


>> trunk ---> release26-maint
>> \->  py3k   ---> release30-maint
>>
>>
> 
> As a side-note: this merging flow means that bugfix and feature commits
> may never be merged from trunk to py3k in one svnmerge batch.  Else,
> they cannot be separated when merging from py3k to 30-maint.

True. However, the same would be true for the merge flow

26 -> trunk -> 3.0 -> 3k

In fact, that merge flow wouldn't support merging features *at all*:
a feature added to trunk would need to flow through 3.0, which can't
accept new features.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Taint Mode in Python 3.0

2008-12-04 Thread Maciej Fijalkowski

Hello,

The thing is pypy's taint code is broken. Basically you don't only
need to patch all places that return pyobject, but also all places
that might modify anything. (All side effects) For example innocently
looking call to addition might end up calling arbitrary python code
(and have arbitrary side effects). There is a question how do you
approach such things?

Cheers,
fijal

On Fri, Dec 5, 2008 at 12:29 AM, Amaury Forgeot d'Arc
<[EMAIL PROTECTED]> wrote:
> Hello,
>
> On Thu, Dec 4, 2008 at 19:36, Nicole King <[EMAIL PROTECTED]> wrote:
>> Dear All,
>>
>> I have published the diff for my implementation of tainted mode in Python for
>> R3.0 (released version) at http://www.cats-muvva.net/software/. Look at the
>> bottom the page. I apologise for past problems accessing this web site: I
>> hope to have resolved all the issues with it.
>
> The patch is indeed huge! it seems that every function that returns a
> PyObject must be modified.
> And it seems very difficult to check for its correctness.
>
> Did you look at the Pypy project? The C code of the interpreter is
> generated, and it already proposes a "Taint" option at translation
> time.
> http://codespeak.net/pypy/dist/pypy/doc/objspace-proxies.html#taint
> With only 300 lines of elegant python code...
>
> --
> Amaury Forgeot d'Arc
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

> In the bug report I opened, I listed four ways to fix this along with
> the pros and cons:

I'm in favour of a different, fifth solution:

5) represent all environment variables in Unicode strings,
   including the ones that currently fail to decode.
   (then do the same to file names, then drop the byte-oriented
file operations again)

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging mailing lists

2008-12-04 Thread Janzert

Martin v. Löwis wrote:
> I would like to merge mailing lists, now that the design and first
> implementation of Python 3000 is complete. In particular, I would
> like to merge the python-3000 mailing list back into python-dev,
> and the python-3000-checkins mailing list back into python-checkins.
> The rationale is to simplify usage of the lists, and to avoid
> cross-postings.
> 
> To implement this, all subscribers of the 3000 mailing lists would
> be added to the trunk mailing lists (avoiding duplicates, of course),
> and all automated messages going to python-3000-checkins would then
> be directed to the trunk lists. The 3000 mailing lists would change
> into read-only mode (i.e. primarily leaving the archives behind).
> 
> Any objections?
> 
> Regards,
> Martin

I like the general sentiment, but I think it may be a bad idea to
automatically bring all the subscribers from the -3000 lists over to the
more general lists. For instance if someone has an address subscribed
specifically to archive the -3000 list suddenly it will begin archiving
the other. I would rather just see a final announcement to switch to the
other list and then close the list to further submissions. Let people
join the new appropriate list manually if needed.

Otherwise +1 on getting the discussion and checkins back into combined
lists.

Janzert

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Holding a Python Language Summit at PyCon

2008-12-04 Thread Frank Wierzbicki

On Thu, Dec 4, 2008 at 3:16 PM, Brett Cannon <[EMAIL PROTECTED]> wrote:
> On Thu, Dec 4, 2008 at 12:05, Frank Wierzbicki <[EMAIL PROTECTED]> wrote:
>> On Wed, Dec 3, 2008 at 10:31 AM, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
>>> 14:00 - 15:30
>>> =
>>>
>>> Two tracks:
>>>
>>> Cross-implementation issues:
>>>
>>>  What do the various VMs want/need from CPython to help with their
>>>  implementations?
>>>
>>>  * Marking CPython-specific tests in the test suite?
>>>  * Getting an implementation agnostic test suite for the Python language?
>>>  * Separating the language tests and the pure Python part of the stdlib into
>>>a separate project?  (Or publish them as a separate package.)
>>>  * Transition plans for 3.0?
>>>
>>>  Champion needed.
>> I would like to champion this one.
>>
>
> I told AMK this a while back, but might as well make it more public; I
> am up for chairing as well.
Brett,

Are you saying you've already called the cross-implementation champion
role?  If so I'm happy to defer or co-chair.

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ


Toshio Kuratomi wrote:



I would think life would be ultimately easier if either the file server
or the shell server automatically translated file names from jis and
utf8 and back, so that the PATH on the *nix shell server is entirely
utf8.


This is not possible because no part of the computer knows what the
encoding is.  To the computer, it's just a sequence of bytes.  Unlike
xml or the windows filesystem (winfs? ntfs?) where the encoding is
specified as part of the document/filesystem there's nothing to tell
what encoding the filenames are in.


I thought you said that the file server keep all filenames in shift-jis, 
and the shell server all in utf-8.  If so, then the shell server could 
know if it were told so.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

2008-12-04 Thread Raymond Hettinger


2008/12/4 Raymond Hettinger <[EMAIL PROTECTED]>:

Also, we don't know the timing of the third-party updates.
Some may never get converted.  Some may convert quickly
and easily.  Someone (perhaps me) may organize a series of
funded sprints to get many of the major packages converted.


From: "Paul Moore" <[EMAIL PROTECTED]>

One piece of encouraging news I heard today is that mod_wsgi
apparently works with 3.0 already - which may well mean that more web
software than I'd originally anticipated will work sooner rather than
later.

But it's certainly true that Python (all versions, not just 3.0) is
more of an ecosystem than just the CPython core. "Batteries included"
notwithstanding. And it'll take longer for the 3.0 ecosystem to grow
than the 2.6 one.


Here's a bright idea.  On the 3.0 release page, include a box listing
which major third-party apps have been converted.  Update it
once every couple of weeks.  That way, we're not explicitly
discouraging adoption of 3.0, we're just listing what support is
then currently available (if you need twisted and its not on the list,
then that would be your guide).


Raymond

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final


Raymond Hettinger wrote:

From: "A.M. Kuchling" <[EMAIL PROTECTED]>

Perhaps the statement could say something like "we do not expect
most Python packages will be ported to the 3.x series until around the 
time 3.1 is released in X months."  (where X=12?  6?)


I would leave out any discussion of 3.1.  Its content and release date
have nothing to do with when third party modules get updated.

Also, we don't know the timing of the third-party updates.
Some may never get converted.  Some may convert quickly
and easily.  Someone (perhaps me) may organize a series of
funded sprints to get many of the major packages converted.

It would be better to simply say that at the present time,
most important third-party modules do not yet support 3.0.

FWIW, my father is Python newbie and I'm pointing him
to 3.0 because it will be easier to learn than 2.6's hodgepodge
of new and old features.  The 3.0 environment is much cleaner.


I agree with all 4 points, especially the last. I think newcomers should 
be informed of the +/- of different versions and then choose for 
themselves.  For full battery availability, 2.5 is it and will be for 
some months.  For a fresh start without need of extras, 3.0 wins in my 
experience so far.


tjr

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] RELEASED Python 3.0 final


Nick Coghlan wrote:

Terry Reedy wrote:

and this could give some people a mis-impression, most likely negative,
as to the magnitude and nature of the change.  Most of the code I am now
writing would, I believe, run with 2.5 except for print(..., file=xxx).
 And I know that there was concern for backward compatibility to the
point that some changes were rejected (renaming builtins) or delayed
(deleting duplicate test asserts) for that reason.  So I would soften
the statements to "... version of the language that is partially
incompatible with... " and "were made without being bound by backward
compatibility,"


I would agree with Terry - while there are backwards incompatibilities,
they aren't gratuitous.

Then again, Guido does seem to want to discourage people from trying to
target the common subset of the two languages instead of using 2to3 as a
compilation step from the python3 version.


I am not restricting myself to that subset.  There simply is an 
unchanged core that happens to include what I currently need (except 
print, which is isolated in one module).  I might need 'except x as y:' 
someday and will use it if I do but so far 'except x:' is enough and 
back compatible).


tjr

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-04 Thread James Y Knight


On Dec 4, 2008, at 6:39 PM, Martin v. Löwis wrote:

I'm in favour of a different, fifth solution:

5) represent all environment variables in Unicode strings,
  including the ones that currently fail to decode.
  (then do the same to file names, then drop the byte-oriented
   file operations again)


Yay, maybe we can have this whole discussion all over again!

Let's bring out all the same arguments, come to no conclusion, and let  
it taper off unresolved, yet again! :)


FWIW, I still agree with Martin that that's the most reasonable  
solution.


James
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ


James Y Knight wrote:

On Dec 4, 2008, at 6:39 PM, Martin v. Löwis wrote:

I'm in favour of a different, fifth solution:

5) represent all environment variables in Unicode strings,
  including the ones that currently fail to decode.
  (then do the same to file names, then drop the byte-oriented
   file operations again)


Yay, maybe we can have this whole discussion all over again!

Let's bring out all the same arguments, come to no conclusion, and let 
it taper off unresolved, yet again! :)


My impression was that there was not enough time to do something like 
that for the soon-to-be-released 3.0, so it was deferred.  Now or soon 
is the time to reconsider.



FWIW, I still agree with Martin that that's the most reasonable solution.


FWIW2, I have much the same feeling.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 6:14 PM, James Y Knight <[EMAIL PROTECTED]> wrote:
> On Dec 4, 2008, at 6:39 PM, Martin v. Löwis wrote:
>>
>> I'm in favour of a different, fifth solution:
>>
>> 5) represent all environment variables in Unicode strings,
>>  including the ones that currently fail to decode.
>>  (then do the same to file names, then drop the byte-oriented
>>   file operations again)
>
> Yay, maybe we can have this whole discussion all over again!
>
> Let's bring out all the same arguments, come to no conclusion, and let it
> taper off unresolved, yet again! :)
>
> FWIW, I still agree with Martin that that's the most reasonable solution.

It died because nobody presented a viable solution, and I maintain no
solution is possible.  All suggestions involve arbitrary
transformations that fail to round trip correctly at some point or
another.  They're simply about shuffling the failure around to
somewhere the poster happens to like.

Please, if you have a *new* idea that doesn't have a failure mode, by
all means post it.  But don't resurrect a pointless bikeshed.

-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

On Thu, Dec 04, 2008 at 05:29:31PM -0800, Raymond Hettinger wrote:
> Here's a bright idea.  On the 3.0 release page, include a box listing
> which major third-party apps have been converted.  Update it
> once every couple of weeks.  That way, we're not explicitly

That's an excellent idea.  We could have a webpage, or start a
topic-specific weblog for posting announcements.

I've started a draft of a 3.0 FAQ in the wiki at
.  Once it's finished we
can move it into the 3.0 release pages.  Everyone please edit and
improve it!

--amk
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-04 Thread Dino Viehland

Does anyone know what Mono does here?  Presumably they have the exact same
problem as all strings in .NET are Unicode, and filenames/env vars/etc...
are always strings.

Maybe if it's gotta be broken at least it can be broken in a manner
that's consistent with others :)

> -Original Message-
> From: [EMAIL PROTECTED] [mailto:python-
> [EMAIL PROTECTED] On Behalf Of Adam Olsen
> Sent: Thursday, December 04, 2008 6:32 PM
> To: James Y Knight
> Cc: "Martin v. Löwis"; python-dev List
> Subject: Re: [Python-Dev] Python-3.0, unicode, and os.environ
>
> On Thu, Dec 4, 2008 at 6:14 PM, James Y Knight <[EMAIL PROTECTED]> wrote:
> > On Dec 4, 2008, at 6:39 PM, Martin v. Löwis wrote:
> >>
> >> I'm in favour of a different, fifth solution:
> >>
> >> 5) represent all environment variables in Unicode strings,
> >>  including the ones that currently fail to decode.
> >>  (then do the same to file names, then drop the byte-oriented
> >>   file operations again)
> >
> > Yay, maybe we can have this whole discussion all over again!
> >
> > Let's bring out all the same arguments, come to no conclusion, and
> let it
> > taper off unresolved, yet again! :)
> >
> > FWIW, I still agree with Martin that that's the most reasonable
> solution.
>
> It died because nobody presented a viable solution, and I maintain no
> solution is possible.  All suggestions involve arbitrary
> transformations that fail to round trip correctly at some point or
> another.  They're simply about shuffling the failure around to
> somewhere the poster happens to like.
>
> Please, if you have a *new* idea that doesn't have a failure mode, by
> all means post it.  But don't resurrect a pointless bikeshed.
>
>
> --
> Adam Olsen, aka Rhamphoryncus
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-
> dev/dinov%40microsoft.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 8:24 PM, Dino Viehland <[EMAIL PROTECTED]> wrote:
> Does anyone know what Mono does here?  Presumably they have the exact same
> problem as all strings in .NET are Unicode, and filenames/env vars/etc...
> are always strings.
>
> Maybe if it's gotta be broken at least it can be broken in a manner
> that's consistent with others :)

Many of the windows APIs use UTF-16 without validating it.  They'll
pass through invalid strings until they hit something that does
validate, at which point it'll blow up.

I suspect that it doesn't happen very often in practice, as having
only one encoding makes it quite clear that it's a broken file name,
not a mixed encoding environment.

-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ


On 02:08 am, [EMAIL PROTECTED] wrote:

James Y Knight wrote:

On Dec 4, 2008, at 6:39 PM, Martin v. L�wis wrote:

I'm in favour of a different, fifth solution:

5) represent all environment variables in Unicode strings,
  including the ones that currently fail to decode.
  (then do the same to file names, then drop the byte-oriented
   file operations again)


FWIW, I still agree with Martin that that's the most reasonable 
solution.


FWIW2, I have much the same feeling.


And I still disagree, but I re-read the old thread and didn't see much 
of a clear argument there, so at least I'm not re-treading old ground 
:).


The only strategy that would allow us to encode all inputs as unicode 
(including the invalid ones) is to abuse NUL to mean "ha ha, this isn't 
actually a unicode string, it's something I couldn't decode".  This is 
nice because it allows the type of the returned value to be the same, so 
a Python program that expects a unicode object will be able to 
manipulate this object (as long as it doesn't split it up too close to a 
NUL).


It seems to me that this convenient, but clever-clever type distinction 
will inevitably be a bug magnet.  For the most basic example, see the 
caveat above.  But more realistically - not too much code splits 
filenames on anything but "." or os.sep, after all - if you pass this to 
an extension module which then wants to invoke a C library function 
which passes the file name to open() and friends, what is the right 
thing for the extension module to do?  There would need to be a new API 
which could get the "right" bytes out of a unicode string which 
potentially has NULs in it.  This can't just be an encoding, either, 
because you might need to get the Shift-JIS bytes (some foreign system's 
encoding) for some got-NULs-in-it filename even though your locale says 
the encoding is UTF-8.  And what if those bytes happen to be valid 
Shift-JIS?  Decoding bytes makes a lot more sense to me than transcoding 
strings.


Filenames and environment variables would all need to be encoded or 
decoded according to this magic encoding.  And what happens if you get 
some garbage data from elsewhere and pass it to a function that 
*generates* a filename?  Now, you get a pleasant error message, 
"TypeError: file() argument 1 must be (encoded string without NULL 
bytes), not str".  In the future, I can only assume (if you're lucky) 
that you'll get some weird thing out of the guts of an encoding module; 
or, more likely, some crazy mojibake filename containing PUA code points 
or whatever will silently get opened.  You can make this less likely 
(and harder to debug in the odd cases where it does happen) by making 
the encoding more clever, but eventually your luck will run out: most 
likely on somebody's computer who doesn't speak english well enough to 
report the problem clearly.


The scenario gets progressively more nightmarish as you start putting 
more libraries into the mix.  You pass some environment variable into 
some library which knows all about unicode and happily handles it 
correctly, but a second library which doesn't know about this proposed 
tricky NUL convention gets the unicode filename and transcodes it 
literally, causing an error return from open().  This puts the apparent 
error very far away from the responsible code.


Ultimately it makes sense to expose the underlying bytes as bytes 
without forcing everyone to pretend that they make sense as anything but 
bytes, and allow different applications to make appropriately educated 
guesses about their character format.  In any case, programmers who 
don't know about these kinds of issues are going to make mistakes in 
handling invalid filenames on UNIXy systems, and some users won't be 
able to open some files.  If there is an easy and straightforward way to 
get the bytes out, it's more likely that programmers who know what they 
are doing will be able to get the correct behavior.


Of course, the NUL-encoding trick will make it *possible* to do the 
right thing, but our hypothetically savvy programmer now needs to learn 
about the bytes/unicode distinction between 
windows/mac+linux+everythingelse, and Python's special convention for 
invalid data, and how to mix it with encoding/decoding/transcoding, 
rather than just Python's distinct API for the distinct types that may 
represent a filename.  I think this is significantly harder to document 
than just having two parallel APIs (environ, environb, open(str), 
open(bytes), listdir(str), listdir(bytes)) to reflect the very subtle, 
but nevertheless very real, distinction between the Windows and UNIX 
worlds.


This distinct API can still provide the same illusion of "it usually 
works" portability that the encoding convention can: for Windows, 
environb can be the representation of the environment in a particular 
encoding; for UNIX, environ(u) can be all of the variables which 
correctly decode.  And so on for each other API.


At least this time I thi

Re: [Python-Dev] Python-3.0, unicode, and os.environ



On 02:32 am, [EMAIL PROTECTED] wrote:

On Thu, Dec 4, 2008 at 6:14 PM, James Y Knight <[EMAIL PROTECTED]> wrote:


FWIW, I still agree with Martin that that's the most reasonable 
solution.


It died because nobody presented a viable solution, and I maintain no
solution is possible.  All suggestions involve arbitrary
transformations that fail to round trip correctly at some point or
another.  They're simply about shuffling the failure around to
somewhere the poster happens to like.

Please, if you have a *new* idea that doesn't have a failure mode, by
all means post it.  But don't resurrect a pointless bikeshed.


Despite my objection to the funny-encoding strategy (which I've 
documented thoroughly in my other message to this thread) this isn't 
accurate.  The PUA solution doesn't work, but using NUL does.  This was 
proposed last time, as a copy of what Mono does.  You can't get a NUL in 
os.environ or a filename; it's not valid.  So, it works fine as an 
escape character.  It can round-trip perfectly.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final



On 02:35 am, [EMAIL PROTECTED] wrote:

On Thu, Dec 04, 2008 at 05:29:31PM -0800, Raymond Hettinger wrote:

Here's a bright idea.  On the 3.0 release page, include a box listing
which major third-party apps have been converted.  Update it
once every couple of weeks.  That way, we're not explicitly


That's an excellent idea.  We could have a webpage, or start a
topic-specific weblog for posting announcements.

I've started a draft of a 3.0 FAQ in the wiki at
.  Once it's finished we
can move it into the 3.0 release pages.  Everyone please edit and
improve it!


It occurs to me that this specific idea (the box with the list of 
supported applications / libraries) should be implementable as a simple 
query against PyPI.  I don't know if it actually is :), but it should 
be.  In general it would be nice to know whether one's favorite tools 
were available for *any* new Python version.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final


On Dec 4, 2008, at 10:59 PM, [EMAIL PROTECTED] wrote:
It occurs to me that this specific idea (the box with the list of  
supported applications / libraries) should be implementable as a  
simple query against PyPI.  I don't know if it actually is :), but  
it should be.  In general it would be nice to know whether one's  
favorite tools were available for *any* new Python version.



I agree, this would be ideal.  I'm not sure the metadata is there to  
support it, though.


Each (version of each) package would need to register metadata  
recording which versions of Python it's known to be compatible with  
("has been tested with").  I'd love for this to be available, and  
would be more proactive about testing software I've been involved in  
releasing against more Python versions.



  -Fred

--
Fred Drake   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

I hear some folks are considering advertising 3.0 as experimental or
not ready for serious use yet.

I think that's too negative -- we should encourage people to use it,
period. They'll have to decide for themselves whether they can live
with the lack of ported 3rd party libraries -- which may resolve
itself soon enough. We should make it clear that it's perfectly fine
to stick with 2.6, but at the same time encourage people to try 3.0
and see for themselves -- IMO it's as solid as 2.6. (2.6.1 being more
solid, of course, as will be 3.0.1).

Especially from the education front I've heard a lot of positive
noises about 3.0. See e.g. an early review, posted 8 months ago:
http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html.

I also want to remind folks that I've promised left and right that
post-3.0 we'll stick to the same backwards compatibility strategy that
we used for the 2.x series. No new incompatibilities. No new features
in 3.0.1 etc.; those go in 3.1, 3.2, etc.

The only compromise I'd be willing to make is that 3.1 can be rather
sooner than the typical 18-24 months cycle. But any API that exists in
3.0 will have to take the regular deprecation route, and if we start
having releases close together we should be careful to measure the
deprecation time in years, not releases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 8:55 PM,  <[EMAIL PROTECTED]> wrote:
>
> On 02:32 am, [EMAIL PROTECTED] wrote:
>>
>> On Thu, Dec 4, 2008 at 6:14 PM, James Y Knight <[EMAIL PROTECTED]> wrote:
>
>>> FWIW, I still agree with Martin that that's the most reasonable solution.
>>
>> It died because nobody presented a viable solution, and I maintain no
>> solution is possible.  All suggestions involve arbitrary
>> transformations that fail to round trip correctly at some point or
>> another.  They're simply about shuffling the failure around to
>> somewhere the poster happens to like.
>>
>> Please, if you have a *new* idea that doesn't have a failure mode, by
>> all means post it.  But don't resurrect a pointless bikeshed.
>
> Despite my objection to the funny-encoding strategy (which I've documented
> thoroughly in my other message to this thread) this isn't accurate.  The PUA
> solution doesn't work, but using NUL does.  This was proposed last time, as
> a copy of what Mono does.  You can't get a NUL in os.environ or a filename;
> it's not valid.  So, it works fine as an escape character.  It can
> round-trip perfectly.

The failure is more subtle, in that a path from the filesystem cannot
round trip via a different return path.  i.e. list the dir via python,
pass it to an external lib to open.

If you don't need that to work it's quite easy to explicitly encode
byte strings into text for storage or whatever.


-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] RELEASED Python 2.6.1

2008-12-04 Thread Barry Warsaw


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hot on the heals of Python 3.0 comes the Python 2.6.1 bug-fix  
release.  This is the latest production-ready version in the Python  
2.6 family.  Dozens of issues have fixed since Python 2.6 final was  
released in October.  Please see the NEWS file for details:


http://www.python.org/download/releases/2.6.1/NEWS.txt

For more information on Python 2.6 please see

http://docs.python.org/dev/whatsnew/2.6.html

Source tarballs and Windows installers can be downloaded from the  
Python 2.6.1 page:


   http://www.python.org/download/releases/2.6.1/

Bugs can be reported in the Python bug tracker:

   http://bugs.python.org

Enjoy,
- -Barry

Barry Warsaw
[EMAIL PROTECTED]
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSTi3KnEjvBPtnXfVAQLhQAP7BR8eqlVLDlu/bp2tGaRRQS8GW5X8KQQk
h0RwCcAKK19WH6YS6zH+VoIpD8LnD37YqZL3m5MQZ/rDf0o3e6152CZ6GJvWE+0i
6w0cSvDqdWuOpfUfpYR21eQnoFuC6x/yfI//yWCnu8bZCypjmJCLKZAvu4pMjYgD
ceChg4lLE68=
=u/iW
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

>> On Dec 4, 2008, at 6:39 PM, Martin v. Löwis wrote:
>>> I'm in favour of a different, fifth solution:
>>>
>>> 5) represent all environment variables in Unicode strings,
>>>  including the ones that currently fail to decode.
>>>  (then do the same to file names, then drop the byte-oriented
>>>   file operations again)

> On Thu, Dec 4, 2008 at 6:14 PM, James Y Knight <[EMAIL PROTECTED]> wrote:
[...]
>> FWIW, I still agree with Martin that that's the most reasonable solution.

On Thu, Dec 4, 2008 at 6:32 PM, Adam Olsen <[EMAIL PROTECTED]> wrote:
> It died because nobody presented a viable solution, and I maintain no
> solution is possible.  All suggestions involve arbitrary
> transformations that fail to round trip correctly at some point or
> another.  They're simply about shuffling the failure around to
> somewhere the poster happens to like.
>
> Please, if you have a *new* idea that doesn't have a failure mode, by
> all means post it.  But don't resurrect a pointless bikeshed.

I don't like Martin's solution at all. Glyph's message nails the
problem -- the "funny encoding" solution breaks as soon as filenames
get passed to other components, and as that's what Python is often all
about, it's likely to happen all the time.

The simplest example I can think of is a program that prints a
directory listing to stdout -- printing the "funny" encoding to stdout
isn't going to be what users expect. So the program has to be aware of
the possibility of "funny" encoded filenames, and the roundtripping
isn't useful at all.

At the risk of bringing up something that was already rejected, let me
propose something that follows the path taken in 3.0 for filenames,
rather than doubling back:

For os.environ, os.getenv() and os.putenv(), I think a similar
approach as used for os.listdir() and os.getcwd() makes sense: let
os.environ skip variables whose name or value is undecodable, and have
a separate os.environb() which contains bytes; let os.getenv() and
os.putenv() do the right thing when the arguments passed in are bytes.

For sys.argv, because it's positional, you can't skip undecodable
values, so I propose to use error=replace for the decoding; again, we
can add sys.argvb that contains the raw bytes values. The various
os.exec*() and os.spawn*() calls (as well as os.system(), os.popen()
and the subprocess module) should all accept bytes as well as strings.

On Windows, the bytes APIs should probably not exist.

I predict that most developers can get away with not using the bytes
APIs at all. The small minority that needs to be robust if not all
filenames use the system encoding can use the bytes APIs. This would
be developers on various Unix systems except OSX (which uses UTF8 for
its filesystems), and perhaps the occasional developer on OSX whose
app needs to work with files on mounted filesystems that use a
different encoding.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final



On 4 Dec, 07:12 pm, [EMAIL PROTECTED] wrote:

The latter statement worries me.  It seems to unnecessarily undermine
adoption of 3.0.  It essentially says, "don't use this".  Is that what 
we want?


I think so.  The default case, the case of the user without the 
wherewithal to understand the nuances of the distinction between 2.x and 
3.x, is a user who should use 2.x.  If the user understands what's going 
on, they're not going to pay attention to such a notice anyway.  I think 
Barry did a great job phrasing this; the language in this comment has to 
be strong enough to counter the prevailing wisdom that "higher version 
number = better".  I think it did that without being overly negative.


For most users, especially new users who have yet to be impressed with 
Python's power, 2.x is much better.  It's not like "library support" is 
one small check-box on the language's feature sheet: most of the 
attractive things about Python are libraries.  Of course I am not free 
from bias, being the author of many libraries myself, but it was other 
libraries that drew me to Python in the first place.


If you're writing an application with 2.x, you get GTK, Qt, PyGame, PIL, 
NumPy, and of course the wonderful Twisted.  With 3.0, you get... 
Tkinter, and ... pywin32, I guess, although I can't find the download on 
sourceforge?  A fork of django that "just barely works"?  A "half 
broken" email module in the stdlib?  All things which you can *also* get 
on 2.x, modulo the "barely works" and "half broken".


If you're writing a library, even if you intend to support py3 as a 
platform on day one, you could reach a much wider audience by simply 
writing in 2to3-friendly style and releasing 2.x source.  Writing a 
3.x-only library will artificially limit your audience and make it much 
harder to combine your library with *other* useful Python libraries 
which have not yet been ported.  There's no 3to2 yet, and maybe there 
never will be.  ("py3to2" looks like an interesting project, but seems 
to be misleadingly named, since I don't think it will help you run your 
3.x-source programs on a stock 2.x VM).


The third (albeit much less likely) option is that you're learning 
Python to learn to interact with a system that's scriptable in embedded 
Python, like Blender or Gimp.  I don't think there's a single system of 
that variety which uses 3.0 yet, and these will likely be even slower to 
move than libraries.  So if the user downloads Python 3 and the 
accompanying tutorial they're likely to be confused when they try to use 
their newly-acquired knowledge to script the tool in question.


Of course, in the long term, maintenance for 2.x is going away and we 
are all being gently herded to 3.x.  Aren't the things I just talked 
about the reason for the continued maintenance of 2.x, though?


It makes sense to talk about 3.1 and beyond, because that points to some 
continuity with 3.0.  It doesn't make sense to say "don't use it", but 
it does make sense to say "use it to get ready for the eventual 
direction of the language".  For example, my experience so far suggests 
that the only motion on Twisted towards 3.x during the 3.0.x/2.6.x cycle 
will be us reporting bugs in 2to3 and in the new version of the stdlib. 
3.1 is likely to be the first version we could realistically target.  I 
am sure that many other libraries are in a similar situation, since 2to3 
has not yet been exposed to a wide variety of ugly, real-world code, and 
nobody's maintaining an #ifdef'd up C extension module yet.  By the time 
3.1 rolls around, we will all know how this migration strategy is really 
working out, and will be able to predict the likely migration timetable 
for various libraries with some degree of accuracy.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 10:14 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> At the risk of bringing up something that was already rejected, let me
> propose something that follows the path taken in 3.0 for filenames,
> rather than doubling back:
>
> For os.environ, os.getenv() and os.putenv(), I think a similar
> approach as used for os.listdir() and os.getcwd() makes sense: let
> os.environ skip variables whose name or value is undecodable, and have
> a separate os.environb() which contains bytes; let os.getenv() and
> os.putenv() do the right thing when the arguments passed in are bytes.

+1 (as that's what I suggested)


> For sys.argv, because it's positional, you can't skip undecodable
> values, so I propose to use error=replace for the decoding; again, we
> can add sys.argvb that contains the raw bytes values. The various
> os.exec*() and os.spawn*() calls (as well as os.system(), os.popen()
> and the subprocess module) should all accept bytes as well as strings.

+1.  I wish there was a better solution to sys.argv.


> On Windows, the bytes APIs should probably not exist.

-0.  I'd prefer byte APIs return UTF-16 bytes and the unicode APIs
become validating.


> I predict that most developers can get away with not using the bytes
> APIs at all. The small minority that needs to be robust if not all
> filenames use the system encoding can use the bytes APIs. This would
> be developers on various Unix systems except OSX (which uses UTF8 for
> its filesystems), and perhaps the occasional developer on OSX whose
> app needs to work with files on mounted filesystems that use a
> different encoding.


-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

On Thu, Dec 4, 2008 at 9:40 PM,  <[EMAIL PROTECTED]> wrote:
> The default case, the case of the user without the wherewithal
> to understand the nuances of the distinction between 2.x and 3.x, is a user
> who should use 2.x.

Not at all clear. If they're not sensitive to those nuances it's just
as likely that they're a casual developer (e.g. a student just
learning to program). Such users are unlikely to start using major 3rd
party packages like Twisted or Django, which would be completely
overwhelming to someone just learning. As shown in
http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html, Python 3.0
removes quite a few warts that are likely to trip up learners.

Once they are ready (probably under the wings of some guru) to dive
deeper, they may have to learn about 2.6 and how it differs -- that's
a useful exercise by itself, but if I'm right, most learners won't
have to go there because by the time they get to that point, the 3.0
ecosystem has matured enough to support their needs.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

On Thu, Dec 4, 2008 at 9:46 PM, Adam Olsen <[EMAIL PROTECTED]> wrote:
> On Thu, Dec 4, 2008 at 10:14 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>> On Windows, the bytes APIs should probably not exist.
>
> -0.  I'd prefer byte APIs return UTF-16 bytes and the unicode APIs
> become validating.

-1 on UTF-16 bytes, as this seems extremely useless and confusing to me.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

> Let's bring out all the same arguments, come to no conclusion, and let
> it taper off unresolved, yet again! :)

This time, it will be different. I will write a PEP, and will request
that anybody proposing an alternative solution also write a PEP (and
no change is made to the code before the PEPs have been fully specified,
discussed, and a BDFL pronouncement has been made).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

> Please, if you have a *new* idea that doesn't have a failure mode, by
> all means post it.  But don't resurrect a pointless bikeshed.

While I completely agree that it is pointless to reiterate the same
arguments over and over, I disagree that the bikeshed metapher applies.
This metapher (IIUC) describes a trivial design issue that is merely
a matter of taste, rather than having deep technical implications.
Using Unicode or bytes for strings is not of that kind.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final


On 04:16 am, [EMAIL PROTECTED] wrote:

I hear some folks are considering advertising 3.0 as experimental or
not ready for serious use yet.


With all due respect, for me, "library support" and "serious use" are 
synonymous.  When prompted I would say that 2.5 is probably the version 
that a new Python user should use.  It's what's already installed on 
their Mac or their Ubuntu box, and it's easiest to get libraries for. 
I've already said in my other note why I think the python website should 
say the same.


Speaking of respect, out of respect for all of you folks I have 
refrained from shouting this opinion from the rooftops.  I have avoided 
blogging about it, I've kept all my public feedback on this list, and I 
plan to continue saying nothing (elsewhere) until I have something nice 
to say.  (The occasional snide comment on IRC notwithstanding.)


That doesn't mean I'm going to tell people who have real problems to 
solve to mess around trying out 3.0, just to see if it has the library 
support that they need, when I already know that it doesn't.  Sorry, but 
community spirit only goes so far: when people ask for my 
recommendation, I'm going to tell the truth.


For example, I recently helped my sister do some work that involved 
running a Fourier transform over a large amount of data.  Doing this 
with python 2.5 took only a few minutes (numpy apparently preinstalled 
on leopard!); much faster than trying to debug the obscure errors she 
was getting out of Fortran.  Doing it with Python 3.0 would have been an 
exercise in frustration (no numpy yet at all), and even 2.6 would have 
been a pain (download, compile, install, get numpy, compile, install, 
etc etc).  If python 3.0 had for some reason *been* the preinstalled 
version, we would have needed to download 2.6 or 2.5.  For this reason I 
don't want to encourage the upstream, in this case Apple, to consider 
3.0 "ready" yet either.  2.x is still a necessity, even if they want to 
start shipping 3.0 soon.


In my experience this is an entirely typical usage of Python.  I know 
very few people who have learned the language for its own sake (and in 
fact, the two I can think of right now have long since switched to 
Haskell); it's almost always for this or that library.  In the cases 
where it is for the language itself, the conversation almost always 
begins, "Hey, I've been thinking about learning Python.  Can it do 
$TASK?".  If the answer is (as it often is) "Sure, just use Py$TASK" 
then they're immediately sold.  If not, "learn python" remains one of 
their never-done back-burner projects like "clean out the garage".  Even 
in my own case, I learned Python because it was easier to write GTK+ 
programs in than C; Java's GUI libraries having been demonstrated 
deficient, I wanted something better.  The networking stuff was a side- 
effect.


Given that this is my typical experience of Python introductions (of 
which I have done quite a few), until a majority of Py$TASK for $TASKs 
that I'm interested in have been ported to py3, then even in the 
abstract, py3 remains "experimental" and "not ready for serious use".


That's not the same thing as "bad":
IMO it's as solid as 2.6. (2.6.1 being more solid, of course, as will 
be 3.0.1).


I have not heard anyone saying that 3.0 is flaky, broken, or "beta".  I 
certainly haven't said that, or even thought it.  Library support is 
_the_ problem.

Especially from the education front I've heard a lot of positive
noises about 3.0. See e.g. an early review, posted 8 months ago:
http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html.


To be fair, if someone asked me specifically about educating non- 
programmer adults about programming, I would probably at least *mention* 
py3, if not recommend it outright.  The improved consistency is worth a 
lot in an educational setting.  (But, if one is educating children and 
interested in soliciting their genuine enthusiasm, whiz-bang graphics 
are really a must-have, not a negotiable extra.)


Note, however, that even this paper specifically mentions several 
libraries which must be available, or they will have to "abandon these 
examples entirely or (reluctantly) delay adoption of version 3.0".  I 
hope for Mr. Efford's sake that these libraries will all become 
available shortly.  They have all taken steps to produce 3.0-compatible 
versions.  However, none are available today, making it still a 
difficult choice to use 3 rather than 2.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

> Here's a bright idea.  On the 3.0 release page, include a box listing
> which major third-party apps have been converted.  Update it
> once every couple of weeks.  That way, we're not explicitly
> discouraging adoption of 3.0, we're just listing what support is
> then currently available (if you need twisted and its not on the list,
> then that would be your guide).

As a slight variation: that should be a wiki page (or, as AMK suggests,
a weblog). The release page should link to it.

If maintenance of this list was in the hands of a single person (the
release manager), or a few (the pydotorg editors), it would always
be outdated.

FWIW, there is also the py3 category in PyPI:

http://pypi.python.org/pypi?:action=browse&c=533

Regads,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final

> I agree, this would be ideal.  I'm not sure the metadata is there to
> support it, though.

There is. There have been the following trove classifiers defined for
a few weeks now:

Programming Language :: Python :: 2
Programming Language :: Python :: 2.3
Programming Language :: Python :: 2.4
Programming Language :: Python :: 2.5
Programming Language :: Python :: 2.6
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.0
Programming Language :: Python :: 3.1

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final


On Dec 5, 2008, at 2:27 AM, Martin v. Löwis wrote:

There is. There have been the following trove classifiers defined for
a few weeks now:



Wonderful!  Thanks for clueing me in.  I'll update my projects to use  
those in future releases.



  -Fred

--
Fred Drake   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RELEASED Python 3.0 final