[issue4966] Improving Lib Doc Sequence Types Section

2012-01-23 Thread Georg Brandl

Georg Brandl ge...@python.org added the comment:

+1 for splitting.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4966
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13838] In str.format {0:#.5g} for decimal.Decimal doesn't print trailing zeros

2012-01-23 Thread Mark Dickinson

Mark Dickinson dicki...@gmail.com added the comment:

IMO, the behaviour is fine;  it's the docs that are unclear.  The rules for 
Decimal are different mainly because trailing zeros have meaning for the 
Decimal type.  (Decimal('1.250') and Decimal('1.25') are two distinct Decimal 
objects, unlike float('1.250') and float('1.25').)

See also issue #7094.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13838
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13838] In str.format {0:#.5g} for decimal.Decimal doesn't print trailing zeros

2012-01-23 Thread Mark Dickinson

Mark Dickinson dicki...@gmail.com added the comment:

Ah no, I take it back.  I think (2) is fine---this is the usual preservation of 
trailing zeros where possible.  (1) needs to be fixed (and issue #7094 was left 
open waiting for this fix).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13838
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6210] Exception Chaining missing method for suppressing context

2012-01-23 Thread Catalin Iacob

Changes by Catalin Iacob iacobcata...@gmail.com:


--
nosy: +catalin.iacob

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4966] Improving Lib Doc Sequence Types Section

2012-01-23 Thread Raymond Hettinger

Raymond Hettinger raymond.hettin...@gmail.com added the comment:

+1 for Nick's suggested breakout:

4.6 Sequence Types - list, tuple, range
4.7 Text Sequence Type - str
4.8 Binary Data Sequence Types - bytes, bytearray, memoryview

--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4966
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13793] hasattr, delattr, getattr fail with unnormalized names

2012-01-23 Thread Raymond Hettinger

Raymond Hettinger raymond.hettin...@gmail.com added the comment:

I concur with Benjamin on all counts.

--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13793
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13797] Allow objects implemented in pure Python to export PEP 3118 buffers

2012-01-23 Thread Raymond Hettinger

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13797
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9625] argparse: Problem with defaults for variable nargs

2012-01-23 Thread Michał M .

Michał M. pyt...@michalski.im added the comment:

Of course I've made a mistake:

list for user provided or list for default

should be:

list for user provided or STRING for default

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9625
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13812] multiprocessing package doesn't flush stderr on child exception

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

 I also found that under Python 2.x, even a low-level exit like
 os._exit or multiprocessing.win32.ExitProcess, called from within a
 user-level function in the child, caused flushing.

The difference is the following:
- Python 2.x uses C stdio (write() calls C fwrite(), flush() calls C
fflush(), etc.); buffering is managed by the libc and what you see at
shutdown is the behaviour of your platform's libc
- Python 3.x uses its own buffering mechanism; it flushes automatically
when the object destructor is called, but os._exit() bypasses all
destructors

Now why os._exit() is used. The problem with sys.exit() is that it's too
high-level: it merely raises a SystemExit exception. That exception can
be caught by upper levels in the code stack. When you use fork() and you
are in the child process, you don't want to give back control to the
calling function, especially if that function isn't fork()-aware (think
about what happens if that function writes to a file, both in the child
and the parent).

This happens for example when running multiprocessing's own test suite:
if forking.py used sys.exit(), the child's SystemExit exception would be
caught by the unittest framework, be interpreted as a test failure, and
the rest of the test suite would proceed... in both processes!

It is less obvious, however, why ExitProcess is used under Windows.
Windows doesn't use fork(), it launches a separate process from scratch.
Perhaps for consistency with the Unix behaviour.

   2. Add an explicit stdout/stderr flush where appropriate in
 forking.py and process.py, to ensure tracebacks get written and to
 match the unix behavior. Leave it to the user to worry about flushing
 their own streams.

That's the simplest solution, and the least likely to break
compatibility with user code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13812
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13840] create_string_buffer rejects str init_or_size parameter

2012-01-23 Thread Vincent Pelletier

Vincent Pelletier plr.vinc...@gmail.com added the comment:

Thanks for the quick reply.

FWIW, in 2.7 doc ctype.create_string_buffer is said to accept unicode objects 
as parameter. I don't use this personally, so I don't mind 3.x only working on 
bytes - and already fixed my code accordingly. It's just that I noticed this 
after your answer. Also, I didn't try to confirm if it actually works.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13840
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

Georg, thanks for the tip. Is there any difference in reST between *i*\ 
th and *i*\th ?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Ezio Melotti

Ezio Melotti ezio.melo...@gmail.com added the comment:

You can check on the devguide the section about building the doc and see it 
yourself on the generated HTML.  While building you will also see all the 
warnings caused by invalid markup.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Marc-Andre Lemburg

Marc-Andre Lemburg m...@egenix.com added the comment:

Dave Malcolm wrote:
 
 Dave Malcolm dmalc...@redhat.com added the comment:
 
 On Fri, 2012-01-06 at 12:52 +, Marc-Andre Lemburg wrote:
 Marc-Andre Lemburg m...@egenix.com added the comment:

 Demo patch implementing the collision limit idea for Python 2.7.

 --
 Added file: http://bugs.python.org/file24151/hash-attack.patch

 
 Marc: is this the latest version of your patch?

Yes. As mentioned in the above message, it's just a demo of how
the collision limit idea can be implemented.

 Whether or not we go with collision counting and/or adding a random salt
 to hashes and/or something else, I've had a go at updating your patch
 
 Although debate on python-dev seems to have turned against the
 collision-counting idea, based on flaws reported by Frank Sievertsen
 http://mail.python.org/pipermail/python-dev/2012-January/115726.html
 it seemed to me to be worth at least adding some test cases to flesh out
 the approach.  Note that the test cases deliberately avoid containing
 hostile data.

Martin's example is really just a red herring: it doesn't matter
where the hostile data originates or how it gets into the application.
There are many ways an attacker can get the O(n^2) worst case
timing triggered.

Frank's example is an attack on the second possible way to
trigger the O(n^2) behavior. See msg150724 further above where I
listed the two possibilities:


An attack can be based on trying to find many objects with the same
hash value, or trying to find many objects that, as they get inserted
into a dictionary, very often cause collisions due to the collision
resolution algorithm not finding a free slot.


My demo patch only addresses the first variant. In order to cover
the second variant as well, you'd have to count and limit the
number of iterations in the perturb for-loop of the lookdict()
functions where the hash value of the slot does not match the
key's hash value.

Note that the second variant is both a lot less likely to trigger
(due to the dict getting resized on a regular basis) and the
code involved a lot faster than the code for the first
variant (which requires a costly object comparison), so the
limit for the second variant would have to be somewhat higher
than for the first.

BTW: The collision counting patch chunk for the string dicts in my
demo patch is wrong. I've attached a corrected version. In the
original patch it was counting both collision variants with the
same counter and limit.

--
Added file: http://bugs.python.org/file24295/hash-attack-2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___Index: Objects/dictobject.c
===
--- Objects/dictobject.c(revision 88933)
+++ Objects/dictobject.c(working copy)
@@ -9,6 +9,8 @@
 
 #include Python.h
 
+/* Maximum number of allowed hash collisions. */
+#define Py_MAX_DICT_COLLISIONS 1000
 
 /* Set a key error with the specified argument, wrapping it in a
  * tuple automatically so that tuple keys are not unpacked as the
@@ -327,6 +329,7 @@
 register PyDictEntry *ep;
 register int cmp;
 PyObject *startkey;
+size_t collisions;
 
 i = (size_t)hash  mask;
 ep = ep0[i];
@@ -361,6 +364,7 @@
 
 /* In the loop, me_key == dummy is by far (factor of 100s) the
least likely outcome, so test for that last. */
+collisions = 1;
 for (perturb = hash; ; perturb = PERTURB_SHIFT) {
 i = (i  2) + i + perturb + 1;
 ep = ep0[i  mask];
@@ -387,6 +391,11 @@
  */
 return lookdict(mp, key, hash);
 }
+   if (++collisions  Py_MAX_DICT_COLLISIONS) {
+   PyErr_SetString(PyExc_KeyError,
+   too many hash collisions);
+   return NULL;
+   }
 }
 else if (ep-me_key == dummy  freeslot == NULL)
 freeslot = ep;
@@ -413,6 +422,7 @@
 register size_t mask = (size_t)mp-ma_mask;
 PyDictEntry *ep0 = mp-ma_table;
 register PyDictEntry *ep;
+size_t collisions;
 
 /* Make sure this function doesn't have to handle non-string keys,
including subclasses of str; e.g., one reason to subclass
@@ -439,17 +449,24 @@
 
 /* In the loop, me_key == dummy is by far (factor of 100s) the
least likely outcome, so test for that last. */
+collisions = 1;
 for (perturb = hash; ; perturb = PERTURB_SHIFT) {
 i = (i  2) + i + perturb + 1;
 ep = ep0[i  mask];
 if (ep-me_key == NULL)
 return freeslot == NULL ? ep : freeslot;
-if (ep-me_key == key
-|| (ep-me_hash == hash
- ep-me_key != dummy
- _PyString_Eq(ep-me_key, key)))
+if (ep-me_key == key)
 return ep;
-if (ep-me_key == dummy  freeslot == NULL)
+

[issue13703] Hash collision security issue

2012-01-23 Thread Marc-Andre Lemburg

Marc-Andre Lemburg m...@egenix.com added the comment:

Alex Gaynor wrote:
 I'm able to put N pieces of data into the database on successive requests,
 but then *rendering* that data puts it in a dictionary, which renders that
 page unviewable by anyone.

I think you're asking a bit much here :-) A broken app is a broken
app, no matter how nice Python tries to work around it. If an
app puts too much trust into user data, it will be vulnerable
one way or another and regardless of how the user data enters
the app.

These are the collision counting possibilities we've discussed
so far:

With an collision counting exception you'd get a clear notice that
something in your data and your application is wrong and needs
fixing. The rest of your web app will continue to work fine and
you won't run into a DoS problem taking down all of your web
server.

With the proposed enhancement of collision counting + universal hash
function for Python 3.3, you'd get a warning printed to the logs, the
dict implementation would self-heal and your page is viewable nonetheless.
The admin would then see the log entry and get a chance to fix the
problem.

Note: Even if Python works around the problem successfully, there's no
guarantee that the data doesn't end up being processed by some other
tool in the chain with similar problems. All this is a work-around
for an application bug, nothing more. Silencing the problem
by e.g. using randomization in the string hash algorithm
doesn't really help in identifying the bug.

Overall, I don't think we should make Python's hash function
non-deterministic. Even with the universal hash function idea,
the dict implementation should use a predefined way of determining
the next hash parameter to use, so that running the application
twice against attack data will still result in the same data
output.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

 Frank's example is an attack on the second possible way to
 trigger the O(n^2) behavior. See msg150724 further above where I
 listed the two possibilities:
 
 
 An attack can be based on trying to find many objects with the same
 hash value, or trying to find many objects that, as they get inserted
 into a dictionary, very often cause collisions due to the collision
 resolution algorithm not finding a free slot.
 

No, Frank's examples attack both possible ways.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

 With an collision counting exception you'd get a clear notice that
 something in your data and your application is wrong and needs
 fixing. The rest of your web app will continue to work fine

Except when it doesn't, because you've also broken batch processing
functions and the like.

 Note: Even if Python works around the problem successfully, there's no
 guarantee that the data doesn't end up being processed by some other
 tool in the chain with similar problems.

Non-Python tools don't use Python's hash functions, they are therefore
not vulnerable to the same data.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

I hope the patch is now good. What do you think?

--
Added file: http://bugs.python.org/file24296/done deal.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13793] hasattr, delattr, getattr fail with unnormalized names

2012-01-23 Thread Benjamin Peterson

Changes by Benjamin Peterson benja...@python.org:


--
resolution:  - wont fix
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13793
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

Ah, darn it! My last patch is a total garbage. But then an interesting thing 
happened: I wanted to delete my last patch set, but I got a

ProgrammingError at /review/13816/patchset/4039/delete
schema datetime does not exist

and got all the Django nitty-gritty info about your server. This is not good 
because someone can misuse that information. Please fix this. You have DEBUG = 
True set and so all the details of your server is leaked. Better change that 
to DEBUG = False and let the user get a standard 500 error page instead.

It says at the bottom: You're seeing this error because you have DEBUG = True 
in your Django settings file. Change that to False, and Django will display a 
standard 500 page.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Changes by Boštjan Mejak bostjan.me...@gmail.com:


Removed file: http://bugs.python.org/file24296/done deal.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-01-23 Thread sbt

sbt shibt...@gmail.com added the comment:

Attached is an updated version of the mp_fork_exec.patch.  This one is able to 
reliably clean up any unlinked semaphores if the program exits abnormally.

--
Added file: http://bugs.python.org/file24297/mp_fork_exec.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8713
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Changes by Boštjan Mejak bostjan.me...@gmail.com:


Removed file: http://bugs.python.org/file24293/fixed patch final.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-01-23 Thread sbt

sbt shibt...@gmail.com added the comment:

mp_split_tests.patch splits up the test_multiprocessing.py:

test_multiprocessing_misc.py
  miscellaneous tests which need not be run with multiple configurations

mp_common.py
  testcases which should be run with multiple configurations

test_multiprocessing_fork.py
test_multiprocessing_nofork.py
test_multiprocessing_manager_fork.py
test_multiprocessing_manager_nofork.py
test_multiprocessing_threads.py
  run the testcases in mp_common.py with various configurations

--
Added file: http://bugs.python.org/file24298/mp_split_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8713
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

I wasn't able to remove the patch via Delete Patch Set but I did it by 
clicking on the Unlink button. So the Delete Patch Set option has a bug.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

Georg, I object to your comment about *i*th needing to be *i*\ th

because currently in the source code we have it written as *i*'th which, by 
your logic, would need to be written as *i*\ 'th

which is not so and is compiled/displayed/whatever perfectly as is, so *i*th 
would be compiled/displayed/whatever, too.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13838] In str.format {0:#.5g} for decimal.Decimal doesn't print trailing zeros

2012-01-23 Thread Eric V. Smith

Changes by Eric V. Smith e...@trueblade.com:


--
resolution:  - duplicate
status: open - closed
superseder:  - Add alternate float formatting styles to new-style formatting.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13838
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

This stupid HG is so cryptic. SVN was so easy, now I can't even create a patch. 
Can you just fix those two typos as key function and *i*th please? Thank 
you.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Stefan Krah

Changes by Stefan Krah stefan-use...@bytereef.org:


--
nosy:  -skrah

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4686] Exceptions in ConfigParser don't set .args

2012-01-23 Thread Roundup Robot

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 8e091e36fc80 by Łukasz Langa in branch '2.7':
Fixes #4686. Reverts redundant picklability code from r74544.
http://hg.python.org/cpython/rev/8e091e36fc80

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4686
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13760] ConfigParser exceptions are not pickleable

2012-01-23 Thread Łukasz Langa

Łukasz Langa luk...@langa.pl added the comment:

The reason why 3.2 and 3.3 work is that some time ago I fixed #4686 by 
explicitly settings .args in configparser exceptions. Unfortunately that patch 
was not backported at the time. I did that just now.

You're right, that is specific to configparser and doesn't affect #1692335 
which is still unsolved.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13760
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Marc-Andre Lemburg

Marc-Andre Lemburg m...@egenix.com added the comment:

Here's a version of the collision counting patch that takes both hash
and slot collisions into account.

I've also added a test script which demonstrates both types of
collisions using integer objects (since it's trivial to calculate
their hashes).

To see the collision counting, enable the DEBUG_DICT_COLLISIONS
macro variable.

--
Added file: http://bugs.python.org/file24299/hash-attack-3.patch
Added file: http://bugs.python.org/file24300/integercollision.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___Index: Objects/dictobject.c
===
--- Objects/dictobject.c(revision 88933)
+++ Objects/dictobject.c(working copy)
@@ -9,7 +9,13 @@
 
 #include Python.h
 
+/* Maximum number of allowed collisions. */
+#define Py_MAX_DICT_HASH_COLLISIONS 1000
+#define Py_MAX_DICT_SLOT_COLLISIONS 1000
 
+/* Debug collision detection */
+#define DEBUG_DICT_COLLISIONS 0
+
 /* Set a key error with the specified argument, wrapping it in a
  * tuple automatically so that tuple keys are not unpacked as the
  * exception arguments. */
@@ -327,6 +333,7 @@
 register PyDictEntry *ep;
 register int cmp;
 PyObject *startkey;
+size_t hash_collisions, slot_collisions;
 
 i = (size_t)hash  mask;
 ep = ep0[i];
@@ -361,6 +368,8 @@
 
 /* In the loop, me_key == dummy is by far (factor of 100s) the
least likely outcome, so test for that last. */
+hash_collisions = 1;
+slot_collisions = 1;
 for (perturb = hash; ; perturb = PERTURB_SHIFT) {
 i = (i  2) + i + perturb + 1;
 ep = ep0[i  mask];
@@ -387,9 +396,27 @@
  */
 return lookdict(mp, key, hash);
 }
+   #if DEBUG_DICT_COLLISIONS
+   printf(hash collisions = %zu (i=%zu)\n, hash_collisions, i);
+   #endif
+   if (++hash_collisions  Py_MAX_DICT_HASH_COLLISIONS) {
+   PyErr_SetString(PyExc_KeyError,
+   too many hash collisions);
+   return NULL;
+   }
 }
-else if (ep-me_key == dummy  freeslot == NULL)
-freeslot = ep;
+else {
+   if (ep-me_key == dummy  freeslot == NULL)
+   freeslot = ep;
+   #if DEBUG_DICT_COLLISIONS
+   printf(slot collisions = %zu (i=%zu)\n, slot_collisions, i);
+   #endif
+   if (++slot_collisions  Py_MAX_DICT_SLOT_COLLISIONS) {
+   PyErr_SetString(PyExc_KeyError,
+   too many slot collisions);
+   return NULL;
+   }
+   }
 }
 assert(0);  /* NOT REACHED */
 return 0;
@@ -413,6 +440,7 @@
 register size_t mask = (size_t)mp-ma_mask;
 PyDictEntry *ep0 = mp-ma_table;
 register PyDictEntry *ep;
+size_t hash_collisions, slot_collisions;
 
 /* Make sure this function doesn't have to handle non-string keys,
including subclasses of str; e.g., one reason to subclass
@@ -439,18 +467,39 @@
 
 /* In the loop, me_key == dummy is by far (factor of 100s) the
least likely outcome, so test for that last. */
+hash_collisions = 1;
+slot_collisions = 1;
 for (perturb = hash; ; perturb = PERTURB_SHIFT) {
 i = (i  2) + i + perturb + 1;
 ep = ep0[i  mask];
 if (ep-me_key == NULL)
 return freeslot == NULL ? ep : freeslot;
-if (ep-me_key == key
-|| (ep-me_hash == hash
- ep-me_key != dummy
- _PyString_Eq(ep-me_key, key)))
+if (ep-me_key == key)
 return ep;
-if (ep-me_key == dummy  freeslot == NULL)
-freeslot = ep;
+if (ep-me_hash == hash  ep-me_key != dummy) {
+   if (_PyString_Eq(ep-me_key, key))
+   return ep;
+   #if DEBUG_DICT_COLLISIONS
+   printf(hash collisions = %zu (i=%zu)\n, hash_collisions, i);
+   #endif
+   if (++hash_collisions  Py_MAX_DICT_HASH_COLLISIONS) {
+   PyErr_SetString(PyExc_KeyError,
+   too many hash collisions);
+   return NULL;
+   }
+   }
+else {
+   if (ep-me_key == dummy  freeslot == NULL)
+   freeslot = ep;
+   #if DEBUG_DICT_COLLISIONS
+   printf(slot collisions = %zu (i=%zu)\n, slot_collisions, i);
+   #endif
+   if (++slot_collisions  Py_MAX_DICT_SLOT_COLLISIONS) {
+   PyErr_SetString(PyExc_KeyError,
+   too many slot collisions);
+   return NULL;
+   }
+   }
 }
 assert(0);  /* NOT REACHED */
 return 0;
def integer_hash_collisions(max=1000):

print 'Hash collision attack:'

d = {}
for x in xrange(max):
i = 

[issue13703] Hash collision security issue

2012-01-23 Thread Marc-Andre Lemburg

Marc-Andre Lemburg m...@egenix.com added the comment:

 I've also added a test script which demonstrates both types of
 collisions using integer objects (since it's trivial to calculate
 their hashes).

I forgot to mention: the test script is for 64-bit platforms. It's
easy to adapt it to 32-bit if needed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6792] Distutils-based installer does not detect 64bit versions of Python

2012-01-23 Thread Brian Curtin

Brian Curtin br...@python.org added the comment:

Without a patch or a solution, the priority doesn't really matter (like Tarek 
said in msg127630). If anyone is actively working on this feel free to say 
otherwise, but I see no status to update.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6792
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13812] multiprocessing package doesn't flush stderr on child exception

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

I propose applying the following patch.
We could open a separate issue to suggest exit = sys.exit under Windows (it 
doesn't seem to break any tests).

--
keywords: +patch
nosy: +neologix
stage: test needed - patch review
Added file: http://bugs.python.org/file24301/mpstderrflush.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13812
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13190] ConfigParser uses wrong newline on Windows

2012-01-23 Thread Łukasz Langa

Łukasz Langa luk...@langa.pl added the comment:

That is indeed the behaviour. Citing the tutorial: Python on Windows makes a 
distinction between text and binary files; the end-of-line characters in text 
files are automatically altered slightly when data is read or written.

When you're opening the file in binary mode, \n characters are not altered. 
Opening a file that way specifies programmer intent and I don't think we should 
force os.linesep in that case.

--
resolution:  - works for me
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13190
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Justin Wehnes

Justin Wehnes jweh...@gmail.com added the comment:

Fixed.

--
Added file: http://bugs.python.org/file24302/issue13816.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-01-23 Thread Santoso Wijaya

Changes by Santoso Wijaya santoso.wij...@gmail.com:


--
nosy:  -santa4nt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8713
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13829] exception error in _scproxy.so

2012-01-23 Thread Ned Deily

Ned Deily n...@acm.org added the comment:

The problem was originally discussed in these threads:

http://mail.python.org/pipermail/pythonmac-sig/2011-December/023428.html
http://mail.python.org/pipermail/pythonmac-sig/2011-December/023430.html

Because _scproxy is used to get information about Internet proxies, a test case 
should be constructed on OS X 10.7.x with a proxy configured.

--
assignee:  - ronaldoussoren
components: +Macintosh
nosy: +ned.deily, ronaldoussoren
resolution: invalid - 
status: closed - open
title: exception error - exception error in _scproxy.so

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13829
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13760] ConfigParser exceptions are not pickleable

2012-01-23 Thread Faheem Mitha

Faheem Mitha fah...@faheem.info added the comment:

I see. Thanks for the pointer to the earlier (2008) bug report. I notice you 
fixed the bug differently for 2.7 (define __reduce__) and 3.2 (set args). Is 
there some reason for that?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13760
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13812] multiprocessing package doesn't flush stderr on child exception

2012-01-23 Thread Jon Brandvein

Jon Brandvein jon.brandv...@gmail.com added the comment:

Regarding the patch: I'd also like to see sys.stdout.flush() and 
sys.stderr.flush() between exitcode = self._boostrap() and exit(exitcode) 
in /Lib/multiprocessing/forking.py :: main(). (The extra stderr flush would be 
for symmetry with Popen.__init__() for unix.)

The remainder of this post is what I wrote before seeing your patch.


Thank you for the explanation concerning libc and buffering.

As for os._exit, I can see the need under unix, where the child process has a 
stack inherited from the parent. Also, allowing the child to cleanup shared 
resources would be disastrous for the parent. (I was going to propose the idea 
of making a sys.exit()-like function that did not rely on the exception 
mechanism to unwind the stack. But it wouldn't be usable here for this reason.)

Under Windows, the child process's multiprocessing.forking.main() is launched 
either by directly calling main() on the command line, or from within 
freeze_support() in the user program. In the latter case, the user can be 
advised not to catch SystemExit around freeze_support(), just as they are 
already advised to make calling freeze_support() the first statement within if 
__name__ == '__main__':. So I don't see any harm in switching to sys.exit() 
there.


I think that in general, the differences between multiprocessing's behavior 
under unix and windows are a weakness, as is the lack of a precise 
specification for this behavior.

At the same time, multiprocessing is a lot more convenient to use than the 
subprocess module when the child process is a Python program. In particular, I 
use multiprocessing for its support of passing pickled objects between parent 
and child. With subprocess, I don't think it's even possible to share a file 
descriptor between parent and child under Windows.

I'm wondering whether it would be desirable to make an option for the unix side 
of multiprocessing to behave more like the windows one, and invoke the Python 
interpreter from the beginning with no trace of the parent's stack. That is, 
it'd follow a more traditional fork()-then-exec*() pattern. Then you'd be able 
to have the interpreter destruct naturally under both platforms, close 
resources, etc.

Are there use cases that require just forking under unix? Is the performance 
significantly better without an exec*()?

Perhaps it would be better If I took this discussion to python-ideas.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13812
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13812] multiprocessing package doesn't flush stderr on child exception

2012-01-23 Thread Jesse Noller

Jesse Noller jnol...@gmail.com added the comment:

There's already a bug / pending patch for this behavior here:

http://bugs.python.org/issue8713

No need to take it to -ideas.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13812
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13841] multiprocessing should use sys.exit() where possible

2012-01-23 Thread Jon Brandvein

New submission from Jon Brandvein jon.brandv...@gmail.com:

Currently the multiprocessing library calls a hard exit function (os._exit 
under unix, ExitProcess under Windows) to terminate the child process.

Under unix, this is necessary because the child is forked without exec-ing. 
Calling sys.exit() would make it possible for the child to run code on the part 
of the stack inherited from the parent, via the exception handling mechanism. 
It might also allow the child to accidentally destroy shared state through an 
object destructor, even when that object was not explicitly relied on by the 
child.

Under Windows, I do not see any benefit besides symmetry. Processes are not 
forked, and the only way control can pass to user code after executing the 
target function, is if the process is frozen and the user puts the call to 
freeze_support() inside a try block. This special case can be taken care of by 
advising the user not to do that. (We already tell the user where 
freeze_support() should be located.)

Changing the multiprocessing exit routine from ExitProcess to sys.exit on 
Windows would ensure that all objects holding resources get properly destroyed. 
In particular, it would ensure that all file streams (including standard output 
and standard error) are flushed. This is especially important under Python 3, 
since the new IO system uses its own buffering which cannot be flushed by 
ExitProcess -- from the user's point of view, a potential regression from 
Python 2.x.

Related issues:
  - #13812 would not have been a problem under windows.
  - If #8713 gets adopted, unix can use sys.exit() as well.

--
components: Library (Lib)
messages: 151835
nosy: brandj, jnoller, neologix, pitrou
priority: normal
severity: normal
status: open
title: multiprocessing should use sys.exit() where possible
type: enhancement
versions: Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13841
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Georg Brandl

Georg Brandl ge...@python.org added the comment:

1. Please report issues with the tracker to the meta-tracker at 
http://psf.upfronthosting.co.za/roundup/meta/.

2. You may object all you want, *i*th is invalid reST and a very quick tryout 
with rst2html would have shown you that.

3. Lastly, it is usual to test patches before submitting. Had you tried to 
build your changed docs instead of complaining here, you might have been 
enlightened.

Justin: thanks for the patch.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Roundup Robot

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset a72ca8b23cdf by Georg Brandl in branch '3.2':
#13816: fix two minor style issues. Thanks to Justin Wehnes for the patch.
http://hg.python.org/cpython/rev/a72ca8b23cdf

New changeset f4f9ab2fd51b by Georg Brandl in branch '2.7':
#13816: fix two minor style issues. Thanks to Justin Wehnes for the patch.
http://hg.python.org/cpython/rev/f4f9ab2fd51b

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13842] Cannot pickle Ellipsis or NotImplemented

2012-01-23 Thread James Sanders

New submission from James Sanders bistromath...@gmail.com:

At present, the built-in constants Ellipsis (...) and NotImplemented cannot be 
pickled.  Perhaps there is a good reason for this, but the only discussion I 
can find is at msg108957, where it is stated that these values (along with 
their types, and type(None)) cannot be pickled.

I ran across this in a class that keeps track of numpy-style slicing 
operations, and so sometimes stores references to Ellipsis.  While this is easy 
to work around, it does seem a little surprising that ... cannot be pickled, 
when slice objects can be.  I don't know if there is a likely use for pickling 
NotImplemented.

If this is not changed, perhaps it could be explicitly stated in the 
documentation that these objects cannot be pickled?

--
messages: 151838
nosy: James.Sanders, alexandre.vassalotti, pitrou
priority: normal
severity: normal
status: open
title: Cannot pickle Ellipsis or NotImplemented
type: enhancement
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13842
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10580] Installer sentence in bold

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

Is this fixed or what?

--
nosy: +jwehnes
type:  - enhancement

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10580
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10580] Installer sentence in bold

2012-01-23 Thread Brian Curtin

Brian Curtin br...@python.org added the comment:

No.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10580
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13772] listdir() doesn't work with non-trivial symlinks

2012-01-23 Thread Brian Curtin

Brian Curtin br...@python.org added the comment:

Looks good to me.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13772
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13737] bugs.python.org/review's Django settings file DEBUG=True

2012-01-23 Thread Ezio Melotti

Ezio Melotti ezio.melo...@gmail.com added the comment:

I now set DEBUG = False in gae2django/settings.py and rietveld/settings.py.

--
nosy: +ezio.melotti
resolution:  - fixed
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13737
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13842] Cannot pickle Ellipsis or NotImplemented

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

I think it's a reasonable feature request. Now someone has to write a patch for 
it.

--
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13842
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13816] Two typos in the docs

2012-01-23 Thread Georg Brandl

Changes by Georg Brandl ge...@python.org:


--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13842] Cannot pickle Ellipsis or NotImplemented

2012-01-23 Thread Alexandre Vassalotti

Alexandre Vassalotti alexan...@peadrop.com added the comment:

We will need to bump the protocol number to add support for None, Ellipsis, and 
NotImplemented. Antoine, can you add this to PEP 3154?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13842
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6721] Locks in python standard library should be sanitized on fork

2012-01-23 Thread sbt

sbt shibt...@gmail.com added the comment:

Attached is a patch (without documentation) which creates an atfork module for 
Unix.

Apart from the atfork() function modelled on pthread_atfork() there is also a 
get_fork_lock() function.  This returns a recursive lock which is held whenever 
a child process is created using os.fork(), subprocess.Popen() or 
multiprocessing.Process().  It can be used like

with atfork.get_fork_lock():
r, w = os.pipe()
pid = os.fork()
if pid == 0:
try:
os.close(r)
# do something with w
finally:
os._exit(0)
else:
os.close(w)

# do something with r

This prevents processes forked by other threads from accidentally inheriting 
the writable end (which would potentially cause EOF to be delayed when reading 
from the pipe).  It can also be used to eliminate the potential race where you 
create an fd and then set the CLOEXEC flag on it.

The patch modifies Popen() and Process.start() to acquire the lock when they 
create their pipes.  (A race condition previously made Process.sentinel and 
Process.join() potentially unreliable in a multithreaded program.)

Note that using the deprecated os.popen?() and os.spawn?() functions can still 
cause accidental inheritance of fds.

(I have also done a hopefully complete patch to multiprocessing to optionally 
allow fork+exec on Unix -- see Issue 8713.)

--
Added file: http://bugs.python.org/file24303/atfork.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6721
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6721] Locks in python standard library should be sanitized on fork

2012-01-23 Thread sbt

sbt shibt...@gmail.com added the comment:

Is there any particular reason not to merge Charles-François's 
reinit_locks.diff?

Reinitialising all locks to unlocked after a fork seems the only sane option.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6721
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Dave Malcolm

Dave Malcolm dmalc...@redhat.com added the comment:

I'm attaching an attempt at backporting haypo's random-8.patch to 2.7

Changes relative to random-8.patch:

   * The randomization is off by default, and must be enabled by setting
 a new environment variable PYTHONHASHRANDOMIZATION to a non-empty string.
 (if so then, PYTHONHASHSEED also still works, if provided, in the same
 way as in haypo's patch)

   * All of the various Py_hash_t become long again (Py_hash_t was
 added in 3.2: issue9778)

   * I expanded the randomization from just PyUnicodeObject to also cover
 these types:

 * PyStringObject

 * PyBufferObject

 The randomization does not cover numeric types: if we change the hash of
 int so that hash(i) no longer equals i, we also have to change it
 consistently for long, float, complex, decimal.Decimal and
 fractions.Fraction; however, there are 3rd-party numeric types that
 have their own __hash__ implementation that mimics int.__hash__ (see
 e.g. numpy)

 As noted in http://bugs.python.org/issue13703#msg151063 and
 http://bugs.python.org/issue13703#msg151064, it's not possible
 to directly create a dict with integer keys via JSON or XML-RPC.

 This seems like a tradeoff between the risk of attack via other means
 vs breakage induced by not having hash() == hash() for the various
 equivalent numerical representations in pre-existing code.

   * To support my expanded usage of the random secret, I moved:
   
   PyAPI_DATA(_Py_unicode_hash_secret_t) _Py_unicode_hash_secret

 from unicodeobject.h to object.h and renamed it to:

   PyAPI_DATA(_Py_HashSecret_t) _Py_HashSecret;

 This also exposes it for usage by C extension modules, just in case
 they need it (Murphy's Law suggests we will need if we don't expose
 it).   This is an extension of the API, but warranted, I feel.  My
 plan for downstream RHEL is to add this explicitly to the RPM metadata
 as a Provides of the RPM providing libpython.so so that if something
 needs to use it, it can express a Requires on it; I assume that
 something similar is possible with .deb)

   * generalized test_unicode.HashTest to support the new env var and the
 additional types.  In my version, get_hash takes a _repr string rather
 than an object, so that I can test it with a buffer().  Arguably the
 tests should thus be moved from test_unicode to somewhere else, but this
 location keeps things consistent with haypo's patch.

 haypo: in random-8.patch, within test_unicode.HashTest.test_null_hash,
 hash_empty seems to be misnamed

   * dropped various selftest fixes where the corresponding selftests don't
 exist in 2.7

   * adds a description of the new environment variables to the manpage;
 arguably this should be done for the patch for the default branch also

Caveats:

   * only tested on Linux (Fedora 15 x86_64); not tested on Windows.  Tested
 via make test both with and without PYTHONHASHRANDOMIZATION=1

   * not yet benchmarked

 Doc/using/cmdline.rst  |   28 ++
 Include/object.h   |7 
 Include/pythonrun.h|2 
 Lib/lib-tk/test/test_ttk/test_functions.py |2 
 Lib/os.py  |   19 -
 Lib/test/mapping_tests.py  |2 
 Lib/test/regrtest.py   |5 
 Lib/test/test_gdb.py   |   15 +
 Lib/test/test_inspect.py   |1 
 Lib/test/test_os.py|   47 +++-
 Lib/test/test_unicode.py   |   55 +
 Makefile.pre.in|1 
 Misc/python.man|   22 ++
 Modules/posixmodule.c  |  126 ++--
 Objects/bufferobject.c |8 
 Objects/object.c   |2 
 Objects/stringobject.c |8 
 Objects/unicodeobject.c|   17 +
 PCbuild/pythoncore.vcproj  |4 
 Python/pythonrun.c |2 
 b/Python/random.c  |  284 +
 21 files changed, 510 insertions(+), 147 deletions(-)

--
Added file: 
http://bugs.python.org/file24304/backport-of-hash-randomization-to-2.7-dmalcolm-2012-01-23-001.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13843] Python doesn't compile anymore on our Solaris buildbot: undefined libintl_* symbols

2012-01-23 Thread STINNER Victor

New submission from STINNER Victor victor.stin...@haypocalc.com:

gcc-o python Modules/python.o libpython3.3dm.a -lsocket -lnsl -lintl -lrt 
-ldl -lsendfile   -lm  
Undefined   first referenced
 symbol in file
libintl_bind_textdomain_codeset libpython3.3dm.a(_localemodule.o)
libintl_gettext libpython3.3dm.a(_localemodule.o)
libintl_textdomain  libpython3.3dm.a(_localemodule.o)
libintl_dcgettext   libpython3.3dm.a(_localemodule.o)
libintl_bindtextdomain  libpython3.3dm.a(_localemodule.o)
libintl_dgettextlibpython3.3dm.a(_localemodule.o)
ld: fatal: Symbol referencing errors. No output written to python
collect2: ld returned 1 exit status
*** Error code 1
make: Fatal error: Command failed for target `python'
program finished with exit code 1

http://www.python.org/dev/buildbot/all/builders/sparc%20solaris10%20gcc%203.x/builds/4131/steps/compile/logs/stdio

--
components: Build
messages: 151848
nosy: haypo, pitrou
priority: normal
severity: normal
status: open
title: Python doesn't compile anymore on our Solaris buildbot: undefined 
libintl_* symbols
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13843
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13843] Python doesn't compile anymore on our Solaris buildbot: undefined libintl_* symbols

2012-01-23 Thread STINNER Victor

STINNER Victor victor.stin...@haypocalc.com added the comment:

The issue doesn't occur on Python 2.7.

--
versions: +Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13843
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Marc-Andre Lemburg

Marc-Andre Lemburg m...@egenix.com added the comment:

 To see the collision counting, enable the DEBUG_DICT_COLLISIONS
 macro variable.

Running (part of (*)) the test suite with debugging enabled on a 64-bit
machine shows that slot collisions are much more frequent than
hash collisions, which only account for less than 0.01% of all
collisions.

It also shows that slot collisions in the low 1-10 range are
most frequent, with very few instances of a dict lookup
reaching 20 slot collisions (less than 0.0002% of all
collisions).

The great number of cases with 1 or 2 slot collisions surprised
me. It seems that there's potential for improvement of
the perturbation formula left.

Due to the large number of 1 or 2 slot collisions, the patch
is going to cause a minor hit to dict lookup performance.
It may make sense to unroll the slot search loop and only
start counting after the third round of misses.

(*) I stopped the run after several hours run-time, producing
some 148GB log data.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13842] Cannot pickle Ellipsis or NotImplemented

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

 We will need to bump the protocol number to add support for None,
 Ellipsis, and NotImplemented. Antoine, can you add this to PEP 3154?

I don't think this needs a protocol bump. These are global singletons,
they can be pickled as such, and they will be unpickleable on any
Python.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13842
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6721] Locks in python standard library should be sanitized on fork

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

 Is there any particular reason not to merge Charles-François's 
 reinit_locks.diff?
 
 Reinitialising all locks to unlocked after a fork seems the only sane option.

I agree with this. 
I haven't looked at the patch very closely. I think perhaps each lock
could have an optional callback for specific code to be run after
forking, but that may come in another patch.
(this would allow to make e.g. the C RLock fork-safe)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6721
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6792] Distutils-based installer does not detect 64bit versions of Python

2012-01-23 Thread Martin v . Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 If priority escalation is out of the question

It's not out of the question - it's just that setting the priority on
the issue is not a proper way to escalate.

Instead, there are two forms of escalation available:
1. submit a patch that fixes the issue
2. pay somebody to fix the issue if you can't fix it yourself

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6792
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13843] Python doesn't compile anymore on our Solaris buildbot: undefined libintl_* symbols

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

The buildbot is Martin's.

--
nosy: +loewis

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13843
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10580] Installer sentence in bold

2012-01-23 Thread Brian Curtin

Brian Curtin br...@python.org added the comment:

Yes.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10580
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10580] Installer sentence in bold

2012-01-23 Thread Boštjan Mejak

Boštjan Mejak bostjan.me...@gmail.com added the comment:

Can you please fix this?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10580
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9625] argparse: Problem with defaults for variable nargs when using choices

2012-01-23 Thread Martin Pengelly-Phillips

Martin Pengelly-Phillips d...@thesociable.net added the comment:

The real issue is that the choices flag does not work with a default flag and * 
nargs.

The following works as expected:
 parser.add_argument('chosen', nargs='*', default=['a'])
 print(parser.parse_args())
Namespace(chosen=['a'])
 print(parser.parse_args(['a', 'b']))
Namespace(chosen=['a', 'b'])

Introducing a choices constraint breaks down when using the defaults:
 parser.add_argument('chosen', nargs='*', default=['a'], choices=['a', 'b'])
 print(parser.parse_args(['a']))
Namespace(chosen=['a'])
 print(parser.parse_args())
error: argument chosen: invalid choice: ['a'] (choose from 'a', 'b')

I would expect instead to have Namespace.chosen populated with the default list 
as before, but the choices constraint check does not validate correctly.

I think that changing the choices constraint logic to iterate over the default 
values if nargs results in a list would be a possible solution.

--
title: argparse: Problem with defaults for variable nargs - argparse: Problem 
with defaults for variable nargs when using choices

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9625
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10278] add time.wallclock() method

2012-01-23 Thread Roundup Robot

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset fb0f4fe8123e by Victor Stinner in branch 'default':
Issue #10278: wallclock() cannot go backward, but two consecutive calls
http://hg.python.org/cpython/rev/fb0f4fe8123e

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10278
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5305] imaplib should support international mailbox names

2012-01-23 Thread C Fraire

C Fraire cfra...@yahoo.com added the comment:

I've used the PloneMailList implementation in another project. It works well to 
add 'imap4-utf-7' as codec.

The twisted imap implementation seems to have been updated to properly support 
non-printable ASCII, but the twisted imap API is problematic for imaplib 
because twisted seems to expect its arguments to already be Python unicode.

So can we be specific about what kind of API change would satisfy this issue:

1) a number of API methods take one or more mailbox arguments. Of course, 
imaplib currently expects these to be ASCII, but what kind of argument should 
the methods take? UTF? Unicode? So would the library need a class property to 
describe an optional specified input encoding? Would it be expected to take 
Python unicode?

2) some methods, such as list and lsub, return mailbox names UTF-7 encoded and 
embedded in larger ASCII strings. Would imaplib be expected to alter the 
contents of these large strings and transform them into another other encoding 
(when a switch as described in 1) is active)?

--
nosy: +cfraire

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5305
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13844] hg.python.org doesn't escape title attributes in annotate view

2012-01-23 Thread Robert Xiao

New submission from Robert Xiao nneon...@gmail.com:

On hg.python.org, the annotate view doesn't properly escape the title 
attribute of the a elements, resulting in breakage on the left column:

http://hg.python.org/cpython/annotate/728cfc671d15/Modules/Setup.config.in

--
components: None
messages: 151860
nosy: nneonneo
priority: normal
severity: normal
status: open
title: hg.python.org doesn't escape title attributes in annotate view
type: behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13844
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13844] hg.python.org doesn't escape title attributes in annotate view

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

This is more of a Mercurial issue than a Python issue, so I suggest you report 
it at http://mercurial.selenic.com/bts/ instead. Thanks!

--
nosy: +georg.brandl, pitrou
resolution:  - invalid
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13844
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13760] ConfigParser exceptions are not pickleable

2012-01-23 Thread Łukasz Langa

Łukasz Langa luk...@langa.pl added the comment:

Currently both branches use the same solution to the problem (e.g. setting 
`args` in `__init__()`). I simply forgot about the old ticket when I prepared a 
fix for this one.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13760
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13844] hg.python.org doesn't escape title attributes in annotate view

2012-01-23 Thread Georg Brandl

Georg Brandl ge...@python.org added the comment:

Quickfixed it locally now.  Let's hope hg itself fixes it more comprehensively.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13844
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13844] hg.python.org doesn't escape title attributes in annotate view

2012-01-23 Thread Robert Xiao

Robert Xiao nneon...@gmail.com added the comment:

My testing suggests that this issue is already fixed in Mercurial itself, since 
using hg serve on a local copy gives the expected result. Thus, the problem 
is probably with hg.python.org's local installation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13844
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13844] hg.python.org doesn't escape title attributes in annotate view

2012-01-23 Thread Ezio Melotti

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
nosy: +ezio.melotti
stage:  - committed/rejected

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13844
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13845] Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows

2012-01-23 Thread STINNER Victor

New submission from STINNER Victor victor.stin...@haypocalc.com:

Python implements time.time() using gettimeofday() which has only a resolution 
of 1 microsecond because it uses the timeval structure which is only able to 
store microseconds.

Attached patch changes _PyTime_gettimeofday() to make it uses the timespec 
structure (which has a resolution has 1 nanosecond) and use 
GetSystemTimeAsFileTime() on Windows. So time.time() has a theorical resolution 
10 times better than currently.

--
files: timespec.patch
keywords: patch
messages: 151865
nosy: belopolsky, haypo
priority: normal
severity: normal
status: open
title: Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows
versions: Python 3.3
Added file: http://bugs.python.org/file24305/timespec.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13845
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13845] Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows

2012-01-23 Thread STINNER Victor

STINNER Victor victor.stin...@haypocalc.com added the comment:

Oops, my first patch contains an unrelated change for Windows.

New patch fixes this bug, and change time_clock() to reuse time_time() if 
time_clock() fails to get the CPU frequency (unlikely) because it has a better 
resolution than clock().

--
Added file: http://bugs.python.org/file24306/timespec-2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13845
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13845] Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows

2012-01-23 Thread STINNER Victor

Changes by STINNER Victor victor.stin...@haypocalc.com:


Removed file: http://bugs.python.org/file24305/timespec.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13845
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Paul McMillan

Paul McMillan p...@mcmillan.ws added the comment:

 I think you're asking a bit much here :-) A broken app is a broken
 app, no matter how nice Python tries to work around it. If an
 app puts too much trust into user data, it will be vulnerable
 one way or another and regardless of how the user data enters
 the app.

I notice your patch doesn't include fixes for the entire standard
library to work around this problem. Were you planning on writing
those, or leaving that for others?

As a developer, I honestly don't know how I can state with certainty
that input data is clean or not, until I actually see the error you
propose. I can't check validity before the fact, the way I can check
for invalid unicode before storing it in my database. Once I see the
error (probably only after my application is attacked, certainly not
during development), it's too late. My application can't know which
particular data triggered the error, so it can't delete it. I'm
reduced to trial and error to remove the offending data, or to writing
code that never stores more than 1000 things in a dictionary. And I
have to accept that the standard library may not work on any
particular data I want to process, and must write code that detects
the error state and somehow magically removes the offending data.

The alternative, randomization, simply means that my dictionary
ordering is not stable, something that is already the case.

While I appreciate that the counting approach feels cleaner;
randomization is the only solution that makes practical sense.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13846] Add time.monotonic() function

2012-01-23 Thread STINNER Victor

New submission from STINNER Victor victor.stin...@haypocalc.com:

After time.wallclock() (issue #10278), let add a time.monotomic() function. It 
is similar to time.wallclock() (try to get the most accurate clock) but is not 
available if the system doesn't provide a monotonic clock. It may also fail at 
runtime if Python cannot find a monotonic clock, whereas time.clock() and 
time.wallclock() fallback on a wallclock which may go backward on NTP adjust.

The documentation of the patch should be improved :-)

wallclock() tests may be simplified or dropped because they may fail on NTP 
adjust.

--
files: monotonic.patch
keywords: patch
messages: 151868
nosy: belopolsky, haypo
priority: normal
severity: normal
status: open
title: Add time.monotonic() function
versions: Python 3.3
Added file: http://bugs.python.org/file24307/monotonic.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13846
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Jim Jewett

Jim Jewett jimjjew...@gmail.com added the comment:

On Mon, Jan 23, 2012 at 4:39 PM, Marc-Andre Lemburg
rep...@bugs.python.org wrote:

 Running (part of (*)) the test suite with debugging enabled on a 64-bit
 machine shows that slot collisions are much more frequent than
 hash collisions, which only account for less than 0.01% of all
 collisions.

Even 1 in 10,000 seems pretty high, though I suppose it is a result of
non-random input.  (For a smalldict with 8 == 2^3 slots, on a 64-bit
machine, true hash collisions should only account for 1 in 2^61 slot
collisions.)

 It also shows that slot collisions in the low 1-10 range are
 most frequent, with very few instances of a dict lookup
 reaching 20 slot collisions (less than 0.0002% of all
 collisions).

Thus the argument that collisions  N implies (possibly malicious)
data that really needs a different hash -- and that this dict instance
in particular should take the hit to use an alternative hash.  (Do
note that this alternative hash could be stored in the hash member of
the PyDictEntry; if anything actually *equal* to the key comes along,
it will have gone through just as many collisions, and therefore also
have been rehashed.)

 The great number of cases with 1 or 2 slot collisions surprised
 me. It seems that there's potential for improvement of
 the perturbation formula left.

In retrospect, this makes sense.

for (perturb = hash; ; perturb = PERTURB_SHIFT) {
i = (i  2) + i + perturb + 1;

If two objects collided then they have the same last few last few bits
in their hashes -- which means they also have the same last few bits
in their initial perturb.  And since the first probe is to slot 6i+1,
it funnels down to only even consider half the slots until the second
probe.

Also note that this explains why Randomization could make the Django
tests fail, even though 64-bit users haven't complained.  The initial
hash(mask) is the same, and the first probe is the same, and (for a
small enough dict) so are the next several.  In a dict with 2^12
slots, the first 6 tries will be the same ... so I doubt the test
cases have sufficiently large amounts of sufficiently unlucky data to
notice very often -- unless the hash itself is changed, as in the
patch.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13703] Hash collision security issue

2012-01-23 Thread Gregory P. Smith

Gregory P. Smith g...@krypto.org added the comment:

On Mon, Jan 23, 2012 at 1:32 PM, Dave Malcolm rep...@bugs.python.org wrote:

 Dave Malcolm dmalc...@redhat.com added the comment:

 I'm attaching an attempt at backporting haypo's random-8.patch to 2.7

 Changes relative to random-8.patch:

   * The randomization is off by default, and must be enabled by setting
     a new environment variable PYTHONHASHRANDOMIZATION to a non-empty string.
     (if so then, PYTHONHASHSEED also still works, if provided, in the same
     way as in haypo's patch)

   * All of the various Py_hash_t become long again (Py_hash_t was
     added in 3.2: issue9778)

   * I expanded the randomization from just PyUnicodeObject to also cover
     these types:

     * PyStringObject

     * PyBufferObject

     The randomization does not cover numeric types: if we change the hash of
     int so that hash(i) no longer equals i, we also have to change it
     consistently for long, float, complex, decimal.Decimal and
     fractions.Fraction; however, there are 3rd-party numeric types that
     have their own __hash__ implementation that mimics int.__hash__ (see
     e.g. numpy)

     As noted in http://bugs.python.org/issue13703#msg151063 and
     http://bugs.python.org/issue13703#msg151064, it's not possible
     to directly create a dict with integer keys via JSON or XML-RPC.

     This seems like a tradeoff between the risk of attack via other means
     vs breakage induced by not having hash() == hash() for the various
     equivalent numerical representations in pre-existing code.

Exactly.  I would NOT worry about hash repeatability for integers and
complex data structures.  It is not at the core of the common problem
(maybe a couple application specific problems but not a general all
python web apps severity problem).

Doing it for base byte string and unicode string like objects is
sufficient.  Good catch on doing it for buffer objects, I'd forgotten
about those. ;)  A big flaw with haypo's patch is that it only
considers unicode instead of all byte-string-ish stuff.  (the code in
issue13704 does that better).


   * To support my expanded usage of the random secret, I moved:

       PyAPI_DATA(_Py_unicode_hash_secret_t) _Py_unicode_hash_secret

     from unicodeobject.h to object.h and renamed it to:

       PyAPI_DATA(_Py_HashSecret_t) _Py_HashSecret;

     This also exposes it for usage by C extension modules, just in case
     they need it (Murphy's Law suggests we will need if we don't expose
     it).   This is an extension of the API, but warranted, I feel.  My
     plan for downstream RHEL is to add this explicitly to the RPM metadata
     as a Provides of the RPM providing libpython.so so that if something
     needs to use it, it can express a Requires on it; I assume that
     something similar is possible with .deb)

Exposing this is good.  There is a hash table implementation within
modules/expat/xmlparse.c that should probably use it as well.

   * generalized test_unicode.HashTest to support the new env var and the
     additional types.  In my version, get_hash takes a _repr string rather
     than an object, so that I can test it with a buffer().  Arguably the
     tests should thus be moved from test_unicode to somewhere else, but this
     location keeps things consistent with haypo's patch.

     haypo: in random-8.patch, within test_unicode.HashTest.test_null_hash,
     hash_empty seems to be misnamed

Lets move this to a better location in all patches.  At this point
haypo's patch is not done yet so relevant bits of what you are doing
here is likely to be fed back into the eventual 3.3 tip patch.

-gps

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13847] Catch time(), ftime(), localtime() and clock() errors

2012-01-23 Thread STINNER Victor

New submission from STINNER Victor victor.stin...@haypocalc.com:

Attach patch catchs errors on time(), ftime(), localtime() and clock(). It 
changes floattime() and _Py_gettimeofday() signature: result now indicates if 
an error occurred or not, and these function now directly raise an exception.

Catching time() error is maybe overflow: modern version of its manual page 
don't mention errors.

--
components: Library (Lib)
files: time_error.patch
keywords: patch
messages: 151871
nosy: belopolsky, haypo
priority: normal
severity: normal
status: open
title: Catch time(), ftime(), localtime() and clock() errors
versions: Python 3.3
Added file: http://bugs.python.org/file24308/time_error.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13847
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue11457] os.stat(): add new fields to get timestamps as Decimal objects with nanosecond resolution

2012-01-23 Thread STINNER Victor

STINNER Victor victor.stin...@haypocalc.com added the comment:

Attached patch prepares time.wallclock() to be able to return the result as an 
integer (seconds, nanoseconds).

--
keywords: +patch
Added file: http://bugs.python.org/file24309/time_integer.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11457
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue11457] os.stat(): add new fields to get timestamps as Decimal objects with nanosecond resolution

2012-01-23 Thread STINNER Victor

STINNER Victor victor.stin...@haypocalc.com added the comment:

With the new function time.wallclock() and time.clock_gettime() (issue #10278), 
and maybe time.monotonic() will maybe be also added (issue #13846), I now agree 
that it is important to support t2-t1 to compute a difference. Using a tuple, 
it's not easy to compute a difference.

time.wallclock(), time.clock_gettime() and time.monotonic() have a nanosecond 
resolution on Linux. Using issue #13845, time.time() will have a resolution of 
100 ns on Windows.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11457
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13845] Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows

2012-01-23 Thread STINNER Victor

STINNER Victor victor.stin...@haypocalc.com added the comment:

See also #11457 for discussion on nanosecond resolution and a potential new 
type to avoid loose of resolution of the Python float type (IEEE binary64).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13845
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue11235] Source files with date modifed in 2106 cause OverflowError

2012-01-23 Thread Antoine Pitrou

Antoine Pitrou pit...@free.fr added the comment:

Fixing this is much easier than Victor's suggestion, we just have to ignore the 
higher bits of the timestamp (that is, store it modulo 2**32). This is enough 
for the purposes of testing the freshness of a pyc file.

And by avoiding modifying the pyc format, we can also ship the fix in bugfix 
releases.

--
components: +Interpreter Core -None
nosy: +brett.cannon, pitrou
versions: +Python 2.7, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11235
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13848] io.open() doesn't check for embedded NUL characters

2012-01-23 Thread Antoine Pitrou

New submission from Antoine Pitrou pit...@free.fr:

 open(\x00abc)
Traceback (most recent call last):
  File stdin, line 1, in module
FileNotFoundError: [Errno 2] No such file or directory: ''

Contrast with 2.x open():

 open(\x00abc)
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: file() argument 1 must be encoded string without NULL bytes, not str

--
components: IO, Library (Lib)
keywords: easy
messages: 151876
nosy: hynek, pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: io.open() doesn't check for embedded NUL characters
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13848
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13849] Add tests for NUL checking in certain strs

2012-01-23 Thread Alex Gaynor

New submission from Alex Gaynor alex.gay...@gmail.com:

ATM there's no tests (at least in 2.x, I haven't checked 3.x yet) for this 
behavior:

 os.path.exists(/tmp\x00abcds)
Traceback (most recent call last):
  File stdin, line 1, in module
  File /usr/lib/python3.2/genericpath.py, line 18, in exists
os.stat(path)
TypeError: embedded NUL character

--
components: Tests
messages: 151877
nosy: alex, amaury.forgeotdarc
priority: normal
severity: normal
status: open
title: Add tests for NUL checking in certain strs
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13849
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13848] io.open() doesn't check for embedded NUL characters

2012-01-23 Thread Alex Gaynor

Changes by Alex Gaynor alex.gay...@gmail.com:


--
nosy: +alex

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13848
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13849] Add tests for NUL checking in certain strs

2012-01-23 Thread Martin v . Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

Why is that a bug? There is no feature in Python saying that the test suite 
covers the code fully (by some kind of measurement).

New tests should only be added to 3.3, unless they test for a newly-fixed bug 
(and even then the test may not be backported to the maintenance release).

--
nosy: +loewis

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13849
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13849] Add tests for NUL checking in certain strs

2012-01-23 Thread R. David Murray

R. David Murray rdmur...@bitdance.com added the comment:

Adding tests helps the other VMs, which generally are trailing behind the 
CPython releases.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13849
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13845] Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows

2012-01-23 Thread Ross Lagerwall

Changes by Ross Lagerwall rosslagerw...@gmail.com:


--
nosy: +rosslagerwall

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13845
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13846] Add time.monotonic() function

2012-01-23 Thread Ross Lagerwall

Changes by Ross Lagerwall rosslagerw...@gmail.com:


--
nosy: +rosslagerwall

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13846
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13850] Summary tables for argparse add_argument options

2012-01-23 Thread Nick Coghlan

New submission from Nick Coghlan ncogh...@gmail.com:

With the current argparse docs, it's very hard to get a quick reminder of how 
to spell the various parameters for add_argument, and just what they do. This 
issue suggests adding a Quick Reference section for add_argument, with the 
following elements:

1. Summary table with a one line description of each parameter
2. Summary table with a one line description of each alternative for the 
action parameter (including noting which other parameters are potentially 
relevant, such as 'choices' and 'metavar' for 'store' and 'const' for 
'store_const')
3. Summary table with a one line description of each alternative for the 
nargs parameter

--
assignee: docs@python
components: Documentation
messages: 151880
nosy: bethard, docs@python, ncoghlan
priority: normal
severity: normal
status: open
title: Summary tables for argparse add_argument options
versions: Python 2.7, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13850
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13850] Summary tables for argparse add_argument options

2012-01-23 Thread Nick Coghlan

Nick Coghlan ncogh...@gmail.com added the comment:

Looking at the docs, a 4th table in the quick reference section would be 
useful: the parameters for ArgumentParser itself.

Note that the ArgumentParser and add_arguments() parameters are already 
summarised in their respective entries, but there are currently no summaries at 
all for the possible action and nargs values

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13850
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-01-23 Thread Charles-François Natali

Charles-François Natali neolo...@free.fr added the comment:

I don't know what the others think, but I'm still -1 on this patch.
Not that I don't like the principle - it's actually the contrary: in a
perfect world, I think this should be made the default -and only -
behavior on POSIX. But since it may break existing code, we'd have to
keep both implementations for POSIX systems, with - at least to me -
little benefit.
Having three different implementations, with different codepaths, will
increase the cognitive burden, make the code less readable, and
debugging harder:
- user: I'm getting this error with multiprocessing
- dev: On Windows or on Unix?
- user: On Unix
- dev: Do you use the fork()+exec() version or the bare fork() version?
- user: what's fork() and exec()?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8713
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com