Re: [Python-Dev] PEP 393: Flexible String Representation

[Florian Weimer]

* Stefan Behnel:

> "Martin v. Löwis", 24.01.2011 21:17:
>> The Py_UNICODE type is still supported but deprecated. It is always
>> defined as a typedef for wchar_t, so the wstr representation can double
>> as Py_UNICODE representation.
>
> It's too bad this isn't initialised by default, though. Py_UNICODE is
> the only representation that can be used efficiently from C code

Is this really true?  I don't think I've seen any C API which actually
uses wchar_t, beyond that what is provided by libc.  UTF-8 and even
UTF-16 are much, much more common.

-- 
Florian Weimer
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Stefan Behnel]

Florian Weimer, 28.01.2011 10:35:

* Stefan Behnel:

"Martin v. Löwis", 24.01.2011 21:17:

The Py_UNICODE type is still supported but deprecated. It is always
defined as a typedef for wchar_t, so the wstr representation can double
as Py_UNICODE representation.


It's too bad this isn't initialised by default, though. Py_UNICODE is
the only representation that can be used efficiently from C code


Is this really true?  I don't think I've seen any C API which actually
uses wchar_t, beyond that what is provided by libc.  UTF-8 and even
UTF-16 are much, much more common.


They are also much harder to use, unless you are really only interested in 
7-bit ASCII data - which is the case for most C libraries, so I believe 
that's what you meant here. However, this is the CPython runtime with 
built-in Unicode support, not the C runtime where it comes as an add-on at 
best, and where Unicode processing without being Unicode aware is common.


The nice thing about Py_UNICODE is that is basically gives you native 
Unicode code points directly, without needing to decode UTF-8 byte runs and 
the like. In Cython, it allows you to do things like this:


def test_for_those_characters(unicode s):
for c in s:
# warning: randomly chosen Unicode escapes ahead
if c in u"\u0356\u1012\u3359\u4567":
return True
else:
return False

The loop runs in plain C, using the somewhat obvious implementation with a 
loop over Py_UNICODE characters and a switch statement for the comparison. 
This would look a *lot* more ugly with UTF-8 encoded byte strings.


Regarding Cython specifically, the above will still be *possible* under the 
proposal, given that the memory layout of the strings will still represent 
the Unicode code points. It will just be trickier to implement in Cython's 
type system as there is no longer a (user visible) C type representation 
for those code units. It can be any of uchar, ushort16 or uint32, neither 
of which is necessarily a 'native' representation of a Unicode character in 
CPython. While I'm somewhat confident that I'll find a way to fix this in 
Cython, my point is just that this adds a certain level of complexity to C 
code using the new memory layout that simply wasn't there before.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] getting stable URLs for major.minor versions

[skip]

Brett> I don't get what you are worried about: http://www.python.org/2
Brett> would refer to 2.7.1 while http://www.python.org/3 would refer to
Brett> 3.1.3.

In my world, 2 == major, 7 == minor, 1 == micro.  I interpreted your
reference to "major" as implying .../2 would refer to .../3.  I thought the
smiley was because you didn't relly expect people to do that.

S
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Florian Weimer]

* Stefan Behnel:

> The nice thing about Py_UNICODE is that is basically gives you native
> Unicode code points directly, without needing to decode UTF-8 byte
> runs and the like. In Cython, it allows you to do things like this:
>
> def test_for_those_characters(unicode s):
> for c in s:
> # warning: randomly chosen Unicode escapes ahead
> if c in u"\u0356\u1012\u3359\u4567":
> return True
> else:
> return False
>
> The loop runs in plain C, using the somewhat obvious implementation
> with a loop over Py_UNICODE characters and a switch statement for the
> comparison. This would look a *lot* more ugly with UTF-8 encoded byte
> strings.

Not really, because UTF-8 is quite search-friendly.  (The if would
have to invoke a memmem()-like primitive.)  Random subscrips are
problematic.

However, why would one want to write loops like the above?  Don't you
have to take combining characters (comprising multiple codepoints)
into account most of the time when you look at individual characters?
Then UTF-32 does not offer much of a simplification.

-- 
Florian Weimer
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Stefan Behnel]

Florian Weimer, 28.01.2011 15:27:

* Stefan Behnel:


The nice thing about Py_UNICODE is that is basically gives you native
Unicode code points directly, without needing to decode UTF-8 byte
runs and the like. In Cython, it allows you to do things like this:

 def test_for_those_characters(unicode s):
 for c in s:
 # warning: randomly chosen Unicode escapes ahead
 if c in u"\u0356\u1012\u3359\u4567":
 return True
 else:
 return False

The loop runs in plain C, using the somewhat obvious implementation
with a loop over Py_UNICODE characters and a switch statement for the
comparison. This would look a *lot* more ugly with UTF-8 encoded byte
strings.


Not really, because UTF-8 is quite search-friendly.  (The if would
have to invoke a memmem()-like primitive.)  Random subscrips are
problematic.

However, why would one want to write loops like the above?  Don't you
have to take combining characters (comprising multiple codepoints)
into account most of the time when you look at individual characters?
Then UTF-32 does not offer much of a simplification.


Hmm, I think this discussion is pointless. Regardless of the memory layout, 
you can always go down to the byte level and use an efficient 
(multi-)substring search algorithm. (which is obviously helped if you know 
the layout at compile time *wink*)


Bad example, I guess.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[anatoly techtonik]

Hi, I'd like to

You probably know that after installation on Windows system it is
possible to call Python from Explorer's Run dialog (Win-R). It is
because Python path is added to App Paths registry key and Windows
Explorer shell checks this key when looking for executable.

But Python doesn't work from cmd session and, more importantly,
*Python doesn't work from .bat files*. It is because cmd shell doesn't
know about App Paths and relies on system PATH to find executable. As
far as I remember, there is no function in Python stdlib either, that
executes processes and does lookups in App Paths.

I never paid much attention to this fact, because I put several .bat
files for every 25, 26, 27, 31 and 32 version of Python into PATH
manually. But when bootstrap script for build environment of Native
Client (NaCl) said that I have no Python available and started to
install its own, I've asked myself - "How come? There are 5! possible
versions of Python on my system." It appeared that the following .bat
file doesn't work:

---cut mypy.bat--
python.exe
---cut mypy.bat--

C:\>mypy.bat

C:\>python.exe
'python.exe' is not recognized as an internal or external command,
operable program or batch file.


I've seen about 7 requests to add Python into %PATH% in installer. All
closed with no result, but with some fear and uncertainty. Martin
feared that MSI installers are not able to remove entry from PATH and
even if they can, they may kill the whole PATH instead of removing
just one entry.

To prove or dispel these fears, I've just installed/uninstalled
Mercurial from mercurial-1.7.3-1-x86.msi and App Engine from
GoogleAppEngine-1.4.1.msi several times. Both add entries to PATH and
both remove them without any further problems. Should we finally add
this to 3.2 installer for Python?

-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Brian Curtin]

On Fri, Jan 28, 2011 at 10:12, anatoly techtonik wrote:

> Hi, I'd like to
>
> You probably know that after installation on Windows system it is
> possible to call Python from Explorer's Run dialog (Win-R). It is
> because Python path is added to App Paths registry key and Windows
> Explorer shell checks this key when looking for executable.
>
> But Python doesn't work from cmd session and, more importantly,
> *Python doesn't work from .bat files*. It is because cmd shell doesn't
> know about App Paths and relies on system PATH to find executable. As
> far as I remember, there is no function in Python stdlib either, that
> executes processes and does lookups in App Paths.
>
> I never paid much attention to this fact, because I put several .bat
> files for every 25, 26, 27, 31 and 32 version of Python into PATH
> manually. But when bootstrap script for build environment of Native
> Client (NaCl) said that I have no Python available and started to
> install its own, I've asked myself - "How come? There are 5! possible
> versions of Python on my system." It appeared that the following .bat
> file doesn't work:
>
> ---cut mypy.bat--
> python.exe
> ---cut mypy.bat--
>
> C:\>mypy.bat
>
> C:\>python.exe
> 'python.exe' is not recognized as an internal or external command,
> operable program or batch file.
>
>
> I've seen about 7 requests to add Python into %PATH% in installer. All
> closed with no result, but with some fear and uncertainty. Martin
> feared that MSI installers are not able to remove entry from PATH and
> even if they can, they may kill the whole PATH instead of removing
> just one entry.
>
> To prove or dispel these fears, I've just installed/uninstalled
> Mercurial from mercurial-1.7.3-1-x86.msi and App Engine from
> GoogleAppEngine-1.4.1.msi several times. Both add entries to PATH and
> both remove them without any further problems. Should we finally add
> this to 3.2 installer for Python?
>
> --
> anatoly t.


Definitely not for 3.2, but this is something I'd like to look into for 3.3.

Recently I've talked to two Python trainers/educators and the major gripe
their attendees see is that you can't just sit down and type "python" and
have something work. For multi-Python installs, we'll have to define what
that "something" is, but I think it should be possible for the installer to
optionally put Python into the path, and to also remove itself on uninstall.

One of said trainers is running a course inside my company right now and the
training room VMs they are running on do not have the path setup. Some users
were puzzled as to why "python foo.py" doesn't work, but executing "foo.py"
does (via file association).

One quick-and-dirty solution was to create a "Command Shell" shortcut in the
start menu which would just be a batch file that adds Python to the path for
that cmd session. It would be kind of similar to the "Python (command line)"
shortcut, which uses pythonw.exe. I think we can do better than this,
though.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Tim Golden]

On 28/01/2011 16:29, Brian Curtin wrote:

On Fri, Jan 28, 2011 at 10:12, anatoly techtonikwrote:

I've seen about 7 requests to add Python into %PATH% in installer. All
closed with no result, but with some fear and uncertainty. Martin
feared that MSI installers are not able to remove entry from PATH and
even if they can, they may kill the whole PATH instead of removing
just one entry.

To prove or dispel these fears, I've just installed/uninstalled
Mercurial from mercurial-1.7.3-1-x86.msi and App Engine from
GoogleAppEngine-1.4.1.msi several times. Both add entries to PATH and
both remove them without any further problems. Should we finally add
this to 3.2 installer for Python?

--
anatoly t.



Definitely not for 3.2, but this is something I'd like to look into for 3.3.

Recently I've talked to two Python trainers/educators and the major gripe
their attendees see is that you can't just sit down and type "python" and
have something work. For multi-Python installs, we'll have to define what
that "something" is, but I think it should be possible for the installer to
optionally put Python into the path, and to also remove itself on uninstall.


I don't think, ultimately, that there is any insurmountable technical
objection. There are misgivings but they could undoubtedly be overcome
or overridden. But it would require someone to patch the MSI builder
so it added the functionality and -- I think -- offered it as an option
which could be enabled or disabled.

TJG
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2011-01-21 - 2011-01-28)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open2567 (+40)
  closed 20262 (+34)
  total  22829 (+74)

Open issues with patches: 1085 


Issues opened (54)
==

#10042: total_ordering stack overflow
http://bugs.python.org/issue10042  reopened by eric.araujo

#10501: make_buildinfo regression with unquoted path
http://bugs.python.org/issue10501  reopened by eli.bendersky

#10708: Misc/porting should be folded into the development FAQ or the 
http://bugs.python.org/issue10708  reopened by pitrou

#10976: json.loads() throws TypeError on bytes object
http://bugs.python.org/issue10976  opened by hhas

#10977: Concrete object C API needs abstract path for subclasses of bu
http://bugs.python.org/issue10977  opened by rhettinger

#10978: Add optional argument to Semaphore.release for releasing multi
http://bugs.python.org/issue10978  opened by rhettinger

#10979: setUpClass exception causes explosion with "-b"
http://bugs.python.org/issue10979  opened by brandon-rhodes

#10980: http.server Header Unicode Bug
http://bugs.python.org/issue10980  opened by aronacher

#10983: Errors in http.client.HTTPConnection class (python3)
http://bugs.python.org/issue10983  opened by nooB

#10984: argparse add_mutually_exclusive_group should accept existing a
http://bugs.python.org/issue10984  opened by gotgenes

#10988: Descriptor protocol documentation for super bindings is incorr
http://bugs.python.org/issue10988  opened by Joshua.Arnold

#10989: ssl.SSLContext(True).load_verify_locations(None, True) segfaul
http://bugs.python.org/issue10989  opened by haypo

#10990: tests mutating sys.gettrace() w/o re-instating previous state
http://bugs.python.org/issue10990  opened by brett.cannon

#10991: trace fails when test imported a temporary file
http://bugs.python.org/issue10991  opened by brett.cannon

#10992: tests failing when run under coverage
http://bugs.python.org/issue10992  opened by brett.cannon

#10994: implementation details in sys module
http://bugs.python.org/issue10994  opened by fijall

#10998: Remove last traces of -Q / sys.flags.division_warning / Py_Div
http://bugs.python.org/issue10998  opened by eric.araujo

#10999: os.chflags refers to stat constants, but the constants are not
http://bugs.python.org/issue10999  opened by r.david.murray

#11001: Various obvious errors in cookies documentation
http://bugs.python.org/issue11001  opened by spookylukey

#11003: os.system should be deprecated in favour of subprocess module
http://bugs.python.org/issue11003  opened by Jakob.Bowyer

#11005: Assertion error on RLock._acquire_restore
http://bugs.python.org/issue11005  opened by haypo

#11006: warnings with subprocess and pipe2
http://bugs.python.org/issue11006  opened by pitrou

#11007: stack tracebacks should give the relevant class name
http://bugs.python.org/issue11007  opened by stickwithjosh

#11009: urllib.splituser is not documented
http://bugs.python.org/issue11009  opened by techtonik

#11011: More functools functions
http://bugs.python.org/issue11011  opened by Jason.Baker

#11012: Add log1p(), exp1m(), gamma(), and lgamma() to cmath
http://bugs.python.org/issue11012  opened by rhettinger

#11015: Bring test.support docs up to date
http://bugs.python.org/issue11015  opened by ncoghlan

#11021: email MIME-Version headers for each part in multipart message
http://bugs.python.org/issue11021  opened by david.caro

#11022: locale.setlocale() doesn't change I/O codec, os.environ does
http://bugs.python.org/issue11022  opened by sdaoden

#11023: pep 227 missing text
http://bugs.python.org/issue11023  opened by aisaac

#11025: Distutils2 install command without setup.py or setup.cfg creat
http://bugs.python.org/issue11025  opened by Boris.FELD

#11027: Implement sectionxform in configparser
http://bugs.python.org/issue11027  opened by Kunjesh.Kaushik

#11028: Implement the setup.py -> setup.cfg in mkcfg
http://bugs.python.org/issue11028  opened by alaintty

#11029: Crash, 2.7.1, Tkinter and threads and line drawing
http://bugs.python.org/issue11029  opened by PythonInTheGrass

#11030: regrtest - allow for relative path with --coverdir
http://bugs.python.org/issue11030  opened by sandro.tosi

#11031: regrtest - --testdir, new command-line option to specify alter
http://bugs.python.org/issue11031  opened by sandro.tosi

#11032: _string: formatter_field_name_split() and formatter_parser doe
http://bugs.python.org/issue11032  opened by haypo

#11033: ElementTree.fromstring doesn't work with Unicode
http://bugs.python.org/issue11033  opened by Peter.Cai

#11034: Build problem on Windows with MSVC++ Express 2008
http://bugs.python.org/issue11034  opened by eli.bendersky

#11035: Segmentation fault
http://bugs.python.org/issue11035  opened by Dmitry.Groshev

#11037: How distutils2 handle namespaces
http://bugs.python.org/issue11037 

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Éric Araujo]

Hello

See http://bugs.python.org/issue3561 (rejected by Martin).

Regards
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Beta version of the new devguide

[Brett Cannon]

On Thu, Jan 27, 2011 at 22:52, Eli Bendersky  wrote:
> On Sun, Jan 23, 2011 at 03:08, Brett Cannon  wrote:
>> http://docs.python.org/devguide/
>>
>> If you are a core developer and have a correction you want to make you
>> can simply check out the devguide yourself (link is in the Resources
>> section of the devguide) and make the corrections yourself. Otherwise
>> reply here (you can email me directly but I already have instances of
>> multiple people telling me about the same spelling mistake so it's
>> nice to have it public so people know when I have been informed).
>
> Brett,
> A couple of concerns regarding the "Getting Set Up" page:
>
> 1)
>
> "Do note that CPython will notice that it is being run from a source
> checkout. This means that it if you edit Python source code in your
> checkout the changes will be picked up by the interpreter for
> immediate testing. "
>
> I'm not sure what this means. Does CPython really know it's being run
> from a source checkout as opposed to a source tarball?

Technically yes because of sys.subversion, but otherwise not really.
But then again the distinction is so minimal I'm not going to bother
rephrasing it to make it clear.

> By editing
> "Python source code" you mean the standard libraries/tests?

I'll make it "Python's".

> To be
> "picked up by the interpreter" you then need to run it from the root
> of the checkout (after build) but this is also true for source
> tarballs.

Once again, not an important distinction.

>
> 2)
>
> "The core CPython interpreter only needs a C compiler to build itself;"
>
> I find this confusing since the CPython interpreter doesn't build
> itself. A developer builds it with a C compiler / makefile. Some tools
> indeed "build themselves" in some kind of a bootstrap process (i.e.
> gcc, AFAIK).

True. I'll rephrase.

>
>
> I apologize in advance if this is too nit-picky ;-)

Sure, but at least you said it nicely. =)

-Brett

> Eli
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] fcmp() in test.support

[Brett Cannon]

On Thu, Jan 27, 2011 at 20:55, Eli Bendersky  wrote:
> I'm working on improving the .rst documentation of test.support (Issue
> 11015), and came upon the undocumented "fcmp" function that's being
> exported from test.support, along with a "FUZZ"constant.
>
> As I search through the tests (py3k trunk), I see fcmp() is being used
> only in two places in a fairly trivial way:
> 1. test_float: where it can be directly replaced by assertAlmostEqual
> from unittest
> 2. test_builtin: where the assertion can also be easily rewritten in
> terms of assertAlmostEqual
>
> Although fcmp seems to provide extra functionality over
> assertAlmostEqual, the above makes me think it should probably be
> removed altogether, or added to unittest if it's still deemed
> important.
>
> +/- ?

I say drop it if it can be done so safely.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] fcmp() in test.support

[Raymond Hettinger]

On Jan 28, 2011, at 10:09 AM, Brett Cannon wrote:

> On Thu, Jan 27, 2011 at 20:55, Eli Bendersky  wrote:
>> I'm working on improving the .rst documentation of test.support (Issue
>> 11015), and came upon the undocumented "fcmp" function that's being
>> exported from test.support, along with a "FUZZ"constant.
>> 
>> As I search through the tests (py3k trunk), I see fcmp() is being used
>> only in two places in a fairly trivial way:
>> 1. test_float: where it can be directly replaced by assertAlmostEqual
>> from unittest
>> 2. test_builtin: where the assertion can also be easily rewritten in
>> terms of assertAlmostEqual
>> 
>> Although fcmp seems to provide extra functionality over
>> assertAlmostEqual, the above makes me think it should probably be
>> removed altogether, or added to unittest if it's still deemed
>> important.
>> 
>> +/- ?
> 
> I say drop it if it can be done so safely.

Yes, please remove fcmp() altogether.
Like you said, the usage is trivial.

If you're feeling bold, replace them with assertEqual(),
the tests look like they produce exact values even
in floating point.


Raymond


--


~/py32 $ ack "fcmp" --python
Doc/tools/pygments/lexers/asm.py
261: r'|lshr|ashr|and|or|xor|icmp|fcmp'

Lib/test/support.py
36:"fcmp", "is_jython", "TESTFN", "HOST", "FUZZ", "SAVEDCWD", "temp_cwd",
354:def fcmp(x, y): # fuzzy comparison function
364:outcome = fcmp(x[i], y[i])

Lib/test/test_builtin.py
13:from test.support import fcmp, TESTFN, unlink,  run_unittest, check_warnings
397:self.assertTrue(not fcmp(divmod(3.25, 1.0), (3.0, 0.25)))
398:self.assertTrue(not fcmp(divmod(-3.25, 1.0), (-4.0, 0.75)))
399:self.assertTrue(not fcmp(divmod(3.25, -1.0), (-4.0, -0.75)))
400:self.assertTrue(not fcmp(divmod(-3.25, -1.0), (3.0, -0.25)))

Lib/test/test_float.py
91:self.assertEqual(support.fcmp(float("  .25e-1  "), .025), 0)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Michael Foord]

On 28/01/2011 16:29, Brian Curtin wrote:
On Fri, Jan 28, 2011 at 10:12, anatoly techtonik > wrote:


Hi, I'd like to

You probably know that after installation on Windows system it is
possible to call Python from Explorer's Run dialog (Win-R). It is
because Python path is added to App Paths registry key and Windows
Explorer shell checks this key when looking for executable.

But Python doesn't work from cmd session and, more importantly,
*Python doesn't work from .bat files*. It is because cmd shell doesn't
know about App Paths and relies on system PATH to find executable. As
far as I remember, there is no function in Python stdlib either, that
executes processes and does lookups in App Paths.

I never paid much attention to this fact, because I put several .bat
files for every 25, 26, 27, 31 and 32 version of Python into PATH
manually. But when bootstrap script for build environment of Native
Client (NaCl) said that I have no Python available and started to
install its own, I've asked myself - "How come? There are 5! possible
versions of Python on my system." It appeared that the following .bat
file doesn't work:

---cut mypy.bat--
python.exe
---cut mypy.bat--

C:\>mypy.bat

C:\>python.exe
'python.exe' is not recognized as an internal or external command,
operable program or batch file.


I've seen about 7 requests to add Python into %PATH% in installer. All
closed with no result, but with some fear and uncertainty. Martin
feared that MSI installers are not able to remove entry from PATH and
even if they can, they may kill the whole PATH instead of removing
just one entry.

To prove or dispel these fears, I've just installed/uninstalled
Mercurial from mercurial-1.7.3-1-x86.msi and App Engine from
GoogleAppEngine-1.4.1.msi several times. Both add entries to PATH and
both remove them without any further problems. Should we finally add
this to 3.2 installer for Python?

--
anatoly t.


Definitely not for 3.2, but this is something I'd like to look into 
for 3.3.


Recently I've talked to two Python trainers/educators and the major 
gripe their attendees see is that you can't just sit down and type 
"python" and have something work. For multi-Python installs, we'll 
have to define what that "something" is, but I think it should be 
possible for the installer to optionally put Python into the path, and 
to also remove itself on uninstall.




I've helped quite a few "python newbies" on Windows who are also 
surprised / frustrated on learning that "python" on the command line 
doesn't work after installing python.


All the best,

Michael Foord

One of said trainers is running a course inside my company right now 
and the training room VMs they are running on do not have the path 
setup. Some users were puzzled as to why "python foo.py" doesn't work, 
but executing "foo.py" does (via file association).


One quick-and-dirty solution was to create a "Command Shell" shortcut 
in the start menu which would just be a batch file that adds Python to 
the path for that cmd session. It would be kind of similar to the 
"Python (command line)" shortcut, which uses pythonw.exe. I think we 
can do better than this, though.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Raymond Hettinger]

On Jan 28, 2011, at 11:21 AM, Michael Foord wrote:

> On 28/01/2011 16:29, Brian Curtin wrote:
>> 
>> Recently I've talked to two Python trainers/educators and the major gripe 
>> their attendees see is that you can't just sit down and type "python" and 
>> have something work. For multi-Python installs, we'll have to define what 
>> that "something" is, but I think it should be possible for the installer to 
>> optionally put Python into the path, and to also remove itself on uninstall.
>> 
> 
> I've helped quite a few "python newbies" on Windows who are also surprised / 
> frustrated on learning that "python" on the command line doesn't work after 
> installing python.

At the very least, we should add some prominent instructions for getting the 
command line version up and running.


Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] fcmp() in test.support

[Michael Foord]

On 28/01/2011 04:55, Eli Bendersky wrote:

I'm working on improving the .rst documentation of test.support (Issue
11015), and came upon the undocumented "fcmp" function that's being
exported from test.support, along with a "FUZZ"constant.



This module shouldn't really be documented at all. It exists to support 
the Python test framework and we don't want to have to support users or 
make API stability guarantees. Plus most of it is rather old. Please 
don't document more stuff in this module.



As I search through the tests (py3k trunk), I see fcmp() is being used
only in two places in a fairly trivial way:
1. test_float: where it can be directly replaced by assertAlmostEqual
from unittest
2. test_builtin: where the assertion can also be easily rewritten in
terms of assertAlmostEqual

Although fcmp seems to provide extra functionality over
assertAlmostEqual, the above makes me think it should probably be
removed altogether, or added to unittest if it's still deemed
important.


Yes, get rid of it.

Michael Foord


+/- ?
Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] fcmp() in test.support

[Eli Bendersky]

>> I'm working on improving the .rst documentation of test.support (Issue
>> 11015), and came upon the undocumented "fcmp" function that's being
>> exported from test.support, along with a "FUZZ"constant.
>>

The documentation of the test module clearly states right at the top:

""
Note

The test package is meant for internal use by Python only. It is
documented for the benefit of the core developers of Python. Any use
of this package outside of Python’s standard library is discouraged as
code mentioned here can change or be removed without notice between
releases of Python.
""

Given that disclaimer, I don't think it's a bad idea to document more
parts of test.support. People adding new tests should be aware of some
of the tools that already exist there, and only some of which are
documented. Just my 2c here.

Maybe Nick will want to chip in here since he opened issue 11015.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Christian Heimes]

Am 28.01.2011 20:29, schrieb Raymond Hettinger:
> At the very least, we should add some prominent instructions for getting the 
> command line version up and running.

/me pops out of Guido's time machine and says: "execute
Tools/scripts/win_add2path.py"

I'm -1 on adding Python to %PATH%. The private MSVCRT DLLs may lead to
unexpected side effects and it doesn't scale at all. What about people
with more than one Python installation? I suggest that we add a single
user specific directory or a global directory to %PATH% for all
installations. Then the Python installer or 3rd party modules can drop
executables like python3.3.exe or plip-3.3.exe into this directory. A
.bat file won't do good because .bat files must be called with "call
python33.bat" from another .bat file or the first one gets terminated.

We can even use a single and simple executable as template for all tasks:

 * get registry key from resource section of the executable
 * use the registry key to lookup the location and name of pythonXX.dll
 * load DLL
 * get optional dotted module name for resource section
 * either fire up interpreter as shell, with **argv or -m
dotted.module.name **argv

Done ;)

Christian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Brian Curtin]

On Fri, Jan 28, 2011 at 14:34, Christian Heimes  wrote:

> Am 28.01.2011 20:29, schrieb Raymond Hettinger:
> > At the very least, we should add some prominent instructions for getting
> the command line version up and running.
>
> /me pops out of Guido's time machine and says: "execute
> Tools/scripts/win_add2path.py"
>
> I'm -1 on adding Python to %PATH%. The private MSVCRT DLLs may lead to
> unexpected side effects and it doesn't scale at all. What about people
> with more than one Python installation?


The same "problem" exists when it comes to file associations. The last
installer you've run wins the battle. Since setting file associations is
optional, and only one association can exist, I don't see why we can't do
the same for the PATH.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Martin v. Löwis]

> The nice thing about Py_UNICODE is that is basically gives you native
> Unicode code points directly, without needing to decode UTF-8 byte runs
> and the like. In Cython, it allows you to do things like this:
> 
> def test_for_those_characters(unicode s):
> for c in s:
> # warning: randomly chosen Unicode escapes ahead
> if c in u"\u0356\u1012\u3359\u4567":
> return True
> else:
> return False
> 
> The loop runs in plain C, using the somewhat obvious implementation with
> a loop over Py_UNICODE characters and a switch statement for the
> comparison. This would look a *lot* more ugly with UTF-8 encoded byte
> strings.

And indeed, when Cython is updated to 3.3, it shouldn't access the UTF-8
representation for such a loop. Instead, it should access the str
representation, and might compile this to code like

#define Cython_CharAt(data, kind, pos) kind==LATIN1 ? \
 ((unsigned char*)data)[pos] : kind==UCS2 ? \
 ((unsigned short*)data)[pos] : \
 ((Py_UCS4*)data)[pos]

 void *data = PyUnicode_Data(s);
 int kind = PyUnicode_Kind(s);
 for(int pos=0; pos < PyUnicode_Size(s); pos++){
   Py_UCS4 c = Cython_CharAt(data, kind, pos);
   Py_UCS4 tmp = {0x356, 0x1012, 0x3359, 0x4567};
   for (int k=0; k<4; k++)
 if (c == tmp[k])
  return 1;
 }
 return 0;

> Regarding Cython specifically, the above will still be *possible* under
> the proposal, given that the memory layout of the strings will still
> represent the Unicode code points. It will just be trickier to implement
> in Cython's type system as there is no longer a (user visible) C type
> representation for those code units.

There is: Py_UCS4 remains available.

> It can be any of uchar, ushort16 or
> uint32, neither of which is necessarily a 'native' representation of a
> Unicode character in CPython.

There won't be a "native" representation anymore - that's the whole
point of the PEP.

> While I'm somewhat confident that I'll
> find a way to fix this in Cython, my point is just that this adds a
> certain level of complexity to C code using the new memory layout that
> simply wasn't there before.

Understood. However, I think it is easier than you think it is.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Josiah Carlson]

Pardon me for this drive-by posting, but this thread smells a lot like this
old thread (don't be afraid to read it all, there are some good points in
there; not directed at you Martin, but at all readers/posters in this
thread)...

http://mail.python.org/pipermail/python-3000/2006-September/003795.html

I'm
not averse to faster and/or more memory efficient unicode representations (I
would be quite happy with them, actually). I do see the usefulness of having
non-utf-8 representations, and caching them is a good idea, though I wonder
if that is a "good for Python itself to cache", or "good for the application
to cache".

The evil side of me says that we should just provide an API available in
Python/C for "give me the representation of unicode string X using the
2byte/4byte code points", and have it just return the appropriate
array.array() value (useful for passing to other APIs, or for those who need
to do manual manipulation of code-points), or whatever structure is deemed
to be appropriate.

The less evil side of me says that going with what the PEP offers isn't a
bad idea, and might just be a good idea.

I'll defer my vote to Martin.

Regards,
 - Josiah

On Mon, Jan 24, 2011 at 12:17 PM, "Martin v. Löwis" wrote:

> I have been thinking about Unicode representation for some time now.
> This was triggered, on the one hand, by discussions with Glyph Lefkowitz
> (who complained that his server app consumes too much memory), and Carl
> Friedrich Bolz (who profiled Python applications to determine that
> Unicode strings are among the top consumers of memory in Python).
> On the other hand, this was triggered by the discussion on supporting
> surrogates in the library better.
>
> I'd like to propose PEP 393, which takes a different approach,
> addressing both problems simultaneously: by getting a flexible
> representation (one that can be either 1, 2, or 4 bytes), we can
> support the full range of Unicode on all systems, but still use
> only one byte per character for strings that are pure ASCII (which
> will be the majority of strings for the majority of users).
>
> You'll find the PEP at
>
> http://www.python.org/dev/peps/pep-0393/
>
> For convenience, I include it below.
>
> Regards,
> Martin
>
> PEP: 393
> Title: Flexible String Representation
> Version: $Revision: 88168 $
> Last-Modified: $Date: 2011-01-24 21:14:21 +0100 (Mo, 24. Jan 2011) $
> Author: Martin v. Löwis 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 24-Jan-2010
> Python-Version: 3.3
> Post-History:
>
> Abstract
> 
>
> The Unicode string type is changed to support multiple internal
> representations, depending on the character with the largest Unicode
> ordinal (1, 2, or 4 bytes). This will allow a space-efficient
> representation in common cases, but give access to full UCS-4 on all
> systems. For compatibility with existing APIs, several representations
> may exist in parallel; over time, this compatibility should be phased
> out.
>
> Rationale
> =
>
> There are two classes of complaints about the current implementation
> of the unicode type: on systems only supporting UTF-16, users complain
> that non-BMP characters are not properly supported. On systems using
> UCS-4 internally (and also sometimes on systems using UCS-2), there is
> a complaint that Unicode strings take up too much memory - especially
> compared to Python 2.x, where the same code would often use ASCII
> strings (i.e. ASCII-encoded byte strings). With the proposed approach,
> ASCII-only Unicode strings will again use only one byte per character;
> while still allowing efficient indexing of strings containing non-BMP
> characters (as strings containing them will use 4 bytes per
> character).
>
> One problem with the approach is support for existing applications
> (e.g. extension modules). For compatibility, redundant representations
> may be computed. Applications are encouraged to phase out reliance on
> a specific internal representation if possible. As interaction with
> other libraries will often require some sort of internal
> representation, the specification choses UTF-8 as the recommended way
> of exposing strings to C code.
>
> For many strings (e.g. ASCII), multiple representations may actually
> share memory (e.g. the shortest form may be shared with the UTF-8 form
> if all characters are ASCII). With such sharing, the overhead of
> compatibility representations is reduced.
>
> Specification
> =
>
> The Unicode object structure is changed to this definition::
>
>  typedef struct {
>PyObject_HEAD
>Py_ssize_t length;
>void *str;
>Py_hash_t hash;
>int state;
>Py_ssize_t utf8_length;
>void *utf8;
>Py_ssize_t wstr_length;
>void *wstr;
>  } PyUnicodeObject;
>
> These fields have the following interpretations:
>
> - length: number of code points in the string (result of sq_length)
> - str: shortest-for

Re: [Python-Dev] PEP 393: Flexible String Representation

[Stefan Behnel]

"Martin v. Löwis", 28.01.2011 22:49:

And indeed, when Cython is updated to 3.3, it shouldn't access the UTF-8
representation for such a loop. Instead, it should access the str
representation


Sure.



Regarding Cython specifically, the above will still be *possible* under
the proposal, given that the memory layout of the strings will still
represent the Unicode code points. It will just be trickier to implement
in Cython's type system as there is no longer a (user visible) C type
representation for those code units.


There is: Py_UCS4 remains available.


Thanks for that pointer. I had always thought that all "*UCS4*" names were 
platform specific and had completely missed that type. It's a lot nicer 
than Py_UNICODE because it allows users to fold surrogate pairs back into 
the character value.


It's completely missing from the docs, BTW. Google doesn't give me a single 
mention for all of docs.python.org, even though it existed at least since 
(and likely long before) Cython's oldest supported runtime Python 2.3.


If I had known about that type earlier, I could have ended up making that 
the native Unicode character type in Cython instead of bothering with 
Py_UNICODE. But this can still be changed I think. Since type inference was 
available before native Py_UNICODE support, it's unlikely that users will 
have Py_UNICODE written in their code explicitly. So I can make the switch 
under the hood.


Just to explain, a native CPython C type is much better than an arbitrary 
integer type, because it allows Cython to apply specific coercion rules 
from and to Python object types. As currently Py_UNICODE, Py_UCS4 would 
obviously coerce from and to a 1 character Unicode string, but it could 
additionally handle surrogate pair splitting and combining automatically on 
current 16-bit Unicode builds so that you'd get a Unicode string with two 
code points on coercion to Python.




While I'm somewhat confident that I'll
find a way to fix this in Cython, my point is just that this adds a
certain level of complexity to C code using the new memory layout that
simply wasn't there before.


Understood. However, I think it is easier than you think it is.


Let's see about the implications once there is an implementation.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Stefan Behnel]

"Martin v. Löwis", 24.01.2011 21:17:

I have been thinking about Unicode representation for some time now.
This was triggered, on the one hand, by discussions with Glyph Lefkowitz
(who complained that his server app consumes too much memory), and Carl
Friedrich Bolz (who profiled Python applications to determine that
Unicode strings are among the top consumers of memory in Python).
On the other hand, this was triggered by the discussion on supporting
surrogates in the library better.

I'd like to propose PEP 393, which takes a different approach,
addressing both problems simultaneously: by getting a flexible
representation (one that can be either 1, 2, or 4 bytes), we can
support the full range of Unicode on all systems, but still use
only one byte per character for strings that are pure ASCII (which
will be the majority of strings for the majority of users).

You'll find the PEP at

http://www.python.org/dev/peps/pep-0393/

[...]
Stable ABI
--

None of the functions in this PEP become part of the stable ABI.


I think that's only part of the truth. This PEP can potentially have an 
impact on the stable ABI in the sense that the build-time size of 
Py_UNICODE may no longer be important for extensions that work on unicode 
buffers in the future as long as they only use the 'str' pointer and not 
'wstr'.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393: Flexible String Representation

[Stefan Behnel]

"Martin v. Löwis", 24.01.2011 21:17:

I have been thinking about Unicode representation for some time now.
This was triggered, on the one hand, by discussions with Glyph Lefkowitz
(who complained that his server app consumes too much memory), and Carl
Friedrich Bolz (who profiled Python applications to determine that
Unicode strings are among the top consumers of memory in Python).
On the other hand, this was triggered by the discussion on supporting
surrogates in the library better.

I'd like to propose PEP 393, which takes a different approach,
addressing both problems simultaneously: by getting a flexible
representation (one that can be either 1, 2, or 4 bytes), we can
support the full range of Unicode on all systems, but still use
only one byte per character for strings that are pure ASCII (which
will be the majority of strings for the majority of users).

You'll find the PEP at

http://www.python.org/dev/peps/pep-0393/


After much discussion, I'm +1 for this PEP. Implementation and benchmarks 
are pending, but there are strong indicators that it will bring relief for 
the memory overhead of most applications without leading to a major 
degradation performance-wise. Not for Python code anyway, and I'll try to 
make sure Cython extensions won't notice much when switching to CPython 3.3.


Martin, this is a smart way of doing it.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com