[issue14191] argparse doesn't allow optionals within positionals

[Glenn Linderman]

Glenn Linderman added the comment:

Paul, is this ready to merge, or are you thinking of more refinements?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18743] References to non-existant "StringIO" module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee: docs@python -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eric V. Smith]

Eric V. Smith added the comment:

On 8/16/2013 12:24 AM, Ethan Furman wrote:
> 
> Ethan Furman added the comment:
> 
>> Eric V. Smith added the comment:
>>
>> But a datetime format string can end in "0", for example.
>>
> format(datetime.datetime.now(), '%H:%M:%S.00')
>> '20:25:27.00'
> 
> Not a problem, because once the digits were removed there would still be % : 
> H M S and ., so the datetime format would 
> be called.  str format would only be called when the result of removing < ^ > 
> 0 1 2 3 4 5 6 7 8 9 was an empty string.

Ah, I misread your earlier text. You'd want to remove 's' also, wouldn't
you?

You might be able to guess whether this particular format spec is a str
format spec or not, but it can't work in the general case, because not
all types with format specifiers have been written yet. I can imagine a
type with a format specification that's identical to str's (yet produces
different output), and you'd always want to use its __format__ instead
of str.__format__.

I feel strongly this is sufficiently magic that we shouldn't do it, and
instead just implement the more robust algorithm in IntEnum.__format__.
But I'm not going to argue it any more.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Ethan Furman]

Ethan Furman added the comment:

> Eric V. Smith added the comment:
>
> But a datetime format string can end in "0", for example.
>
 format(datetime.datetime.now(), '%H:%M:%S.00')
> '20:25:27.00'

Not a problem, because once the digits were removed there would still be % : H 
M S and ., so the datetime format would 
be called.  str format would only be called when the result of removing < ^ > 0 
1 2 3 4 5 6 7 8 9 was an empty string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] '' % [1] doesn't fail

[Eric V. Smith]

Changes by Eric V. Smith :


--
title: '' % [1] doens't fail -> '' % [1] doesn't fail

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] '' % [1] doens't fail

[Eric V. Smith]

Eric V. Smith added the comment:

Objects/unicodeobject.c has this, at line 14316:

if (PyMapping_Check(args) && !PyTuple_Check(args) && !PyUnicode_Check(args))
ctx.dict = args;
else
ctx.dict = NULL;

and later at line 14348:
if (ctx.argidx < ctx.arglen && !ctx.dict) {
PyErr_SetString(PyExc_TypeError,
"not all arguments converted during string formatting");
goto onError;
}

Because list now returns true for PyMapping_Check, this code thinks the list is 
a dict and skips the error.

There's some discussion of PyMapping_Check in issue 5945.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18742] Abstract base class for hashlib

[Gregory P. Smith]

Gregory P. Smith added the comment:

your patch makes sense to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18709] SSL module fails to handle NULL bytes inside subjectAltNames general names (CVE-2013-4238)

[Christian Heimes]

Christian Heimes added the comment:

Brian Cameron from Oracle has requested a fix for Python 2.6. I have attached a 
patch for 2.6. In order to compile and test the patch I had to modify _ssl.c to 
handle OPENSSL_NO_SSL2. I also copied keycert.pem from 2.7 to fix two test 
failures. The former keycert.pem has expired.

It's a bit of a challenge to compile Python 2.6 on modern Linux OS. I had to 
set a couple of flags and overwrite MACHDEP:

export arch=$(dpkg-architecture -qDEB_HOST_MULTIARCH)
export LDFLAGS="-L/usr/lib/$arch -L/lib/$arch"
export CFLAGS="-I/usr/include/$arch"
export CPPFLAGS="-I/usr/include/$arch"
./configure --config-cache --with-pydebug
make -j4 MACHDEP=linux2

--
Added file: http://bugs.python.org/file31309/CVE-2013-4073_py26.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eric V. Smith]

Eric V. Smith added the comment:

On 8/15/2013 8:20 PM, Ethan Furman wrote:
> The characters I list are the justification chars and the digits that would 
> be used to specify the field width.  If 
> those are the only characters given then treat the MixedEnum member as the 
> member string.

But a datetime format string can end in "0", for example.

>>> format(datetime.datetime.now(), '%H:%M:%S.00')
'20:25:27.00'

I think your code would produce the equivalent of:
>>> format(str(datetime.datetime.now()), '%H:%M:%S.00')
Traceback (most recent call last):
  File "", line 1, in 
ValueError: Invalid conversion specification

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Ethan Furman]

Ethan Furman added the comment:

> Eric V. Smith added the comment:
>> Ethan Furman added the comment:
>>
>>> Hmmm.  How about defining the characters that will be supported for string 
>>> interpretation, and if there are any other
>>> characters in format spec then go int (or whatever the mix-in type is)?  
>>> I'm thinking "<^>01234566789".  Anything else
>>> ("+", all letter codes, etc.) gets the normal (host-type) treatment.
>
> Is the goal of this approach to implement __format__ in Enum instead of
> IntEnum?

Yes.

> But you can't do this in general, because in the place you implement
> __format__ you must understand the mix-in type's format strings.

Which is why I suggest concentrating on what defines an "empty" format string.  
In this case "empty" means what can we 
put in the format spec and still get string treatment.

> Consider if the mix-in type is datetime: it's format strings don't end
> in a the set of characters you list.

The characters I list are the justification chars and the digits that would be 
used to specify the field width.  If 
those are the only characters given then treat the MixedEnum member as the 
member string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16105] Pass read only FD to signal.set_wakeup_fd

[Felipe Cruz]

Felipe Cruz added the comment:

Looks like PyErr_WriteUnraisable can be a better choice. Exceptions at random 
execution points looks a little bit dirty at least for this case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eric V. Smith]

Eric V. Smith added the comment:

On 8/15/2013 7:09 PM, Ethan Furman wrote:
> 
> Ethan Furman added the comment:
> 
>> Eric V. Smith added the comment:
>>
>> I think the answers should be:
>>
>> - Formats as int if the length of the format spec is >= 1 and it ends in
>> one of "bdoxX" (the int presentation types).
> 
> Hmmm.  How about defining the characters that will be supported for string 
> interpretation, and if there are any other 
> characters in format spec then go int (or whatever the mix-in type is)?  I'm 
> thinking "<^>01234566789".  Anything else 
> ("+", all letter codes, etc.) gets the normal (host-type) treatment.

Is the goal of this approach to implement __format__ in Enum instead of
IntEnum?

But you can't do this in general, because in the place you implement
__format__ you must understand the mix-in type's format strings.
Consider if the mix-in type is datetime: it's format strings don't end
in a the set of characters you list. So I think you have to implement
__format__ on each derived-from-Enum type so you can make the best
decisions there.

I think we want to have the most possible interpretations give a str
output, and only use the base type if that's explicitly asked for. As
Eli says, that's one of the main reasons IntEnum exists in the first
place. Hence my approach for IntEnum.__format__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Ethan Furman]

Ethan Furman added the comment:

> Eric V. Smith added the comment:
>
> I think the answers should be:
>
> - Formats as int if the length of the format spec is >= 1 and it ends in
> one of "bdoxX" (the int presentation types).

Hmmm.  How about defining the characters that will be supported for string 
interpretation, and if there are any other 
characters in format spec then go int (or whatever the mix-in type is)?  I'm 
thinking "<^>01234566789".  Anything else 
("+", all letter codes, etc.) gets the normal (host-type) treatment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18742] Abstract base class for hashlib

[Christian Heimes]

Changes by Christian Heimes :


Removed file: http://bugs.python.org/file31302/hashlib_abc2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18742] Abstract base class for hashlib

[Christian Heimes]

Changes by Christian Heimes :


Removed file: http://bugs.python.org/file31295/hashlib_abc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18742] Abstract base class for hashlib

[Christian Heimes]

Christian Heimes added the comment:

Now with block_size

--
Added file: http://bugs.python.org/file31308/hashlib_abc3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eli Bendersky]

Eli Bendersky added the comment:

I guess there are merits to keeping it all in the same place.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Ethan Furman]

Ethan Furman added the comment:

> Eli Bendersky added the comment:
> Naturally, compatibility with % formatting is desired. '%s' is str, '%i' is
> int().

Can we solve that one on this issue, or do we need to make another?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eli Bendersky]

Eli Bendersky added the comment:

On Thu, Aug 15, 2013 at 3:20 PM, Eric V. Smith wrote:

>
> Eric V. Smith added the comment:
>
> On 8/15/2013 5:46 PM, Eli Bendersky wrote:
> > The whole point of IntEnum and replacing stdlib constants with it was
> friendly str & repr out of the box. This means that "just printing out" an
> enum member should have a nice string representation. And "just printing
> out" means:
> >
> > print(member)
> > "%s" % member
> > "{}".format(member)
>
> 100% agreed.
>
> > !s/!r are quite esoteric - IntEnum should behave in the nicest way
> possible out of the box.
>
> Not only that, they're not part of the __format__ protocol, anyway.
>
> > Let's just rig IntEnum's __format__ to do the right thing and not worry
> about Enum itself. I hope that mixin-with-Enum cases are rare (and most are
> IntEnum anyway), and in such rare cases users are free to lift the
> implementation from IntEnum.
>
> Agreed.
>
> And the next question is: what is "the right thing"? Does it always
> appear to be a str? Or sometimes str and sometimes int? And how far
> deviant from plain int can it be?
>
> I think the answers should be:
>
> - Formats as int if the length of the format spec is >= 1 and it ends in
> one of "bdoxX" (the int presentation types).
>
> - Possibly format as float if the format spec ends in "eEfFgGn%" (the
> float presentation types), but the utility is doubtful. However, int
> converts to float with these, so we may as well do the same.
>
> - Otherwise formats as a str.
>
>
Sounds good to me. One of IntEnum's raison d'êtres is to be an integer with
a nice string representation. So it makes sense to make it show itself as a
string, unless expicitly asked for an int. Float-conversion *is* dubious,
but I agree that following int's lead here is harmless and least surprising.

Naturally, compatibility with % formatting is desired. '%s' is str, '%i' is
int().
*
*
*

*

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16611] multiple problems with Cookie.py

[Julien Phalip]

Julien Phalip added the comment:

Updated the patch to use a tuple instead of a list for the cookie flags.

--
Added file: http://bugs.python.org/file31307/cookies-httponly-secure-3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eric V. Smith]

Eric V. Smith added the comment:

On 8/15/2013 5:46 PM, Eli Bendersky wrote:
> The whole point of IntEnum and replacing stdlib constants with it was 
> friendly str & repr out of the box. This means that "just printing out" an 
> enum member should have a nice string representation. And "just printing out" 
> means:
> 
> print(member)
> "%s" % member
> "{}".format(member)

100% agreed.

> !s/!r are quite esoteric - IntEnum should behave in the nicest way possible 
> out of the box.

Not only that, they're not part of the __format__ protocol, anyway.

> Let's just rig IntEnum's __format__ to do the right thing and not worry about 
> Enum itself. I hope that mixin-with-Enum cases are rare (and most are IntEnum 
> anyway), and in such rare cases users are free to lift the implementation 
> from IntEnum.

Agreed.

And the next question is: what is "the right thing"? Does it always
appear to be a str? Or sometimes str and sometimes int? And how far
deviant from plain int can it be?

I think the answers should be:

- Formats as int if the length of the format spec is >= 1 and it ends in
one of "bdoxX" (the int presentation types).

- Possibly format as float if the format spec ends in "eEfFgGn%" (the
float presentation types), but the utility is doubtful. However, int
converts to float with these, so we may as well do the same.

- Otherwise formats as a str.

- This will break a small number of valid int format strings, but we
should just accept that potential breakage. This is something to
consider if we're replacing existing ints with IntEnums (which I'm not
generally in favor of, anyway).

- Live with the fact that we'll need to update this code if we add any
int (or float) presentation types.

I think we might want to consider the same thing for bool.__format__,
but that's another issue. Maybe int could grow a __format_derived_type__
method implements the above, and bool and IntEnum could set their
__format__ to be that. Which probably points out that the original
int.__format__ design is flawed (as Nick pointed out), but too late to
change it.

Or, thinking even more outside the box, maybe int.__format__ could
implement the above logic if it knows it's working on a derived class.
That would change bool as well as all other int-derived types, but leave
int itself alone. More breakage, but potentially more useful. But again,
we should open another issue if we want to pursue this approach.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17930] Search not needed in combinations_with_replacement

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18121] antigravity leaks subprocess.Popen object

[Antoine Pitrou]

Antoine Pitrou added the comment:

Outdated by PEP 442.

--
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16105] Pass read only FD to signal.set_wakeup_fd

[Antoine Pitrou]

Antoine Pitrou added the comment:

Here is a cleaned up patch.
However, I'm wondering if raising an error is a good idea. The effect will be 
to get cryptic OSErrors at random execution points. Perhaps using 
PyErr_WriteUnraisable would be better?

--
versions:  -Python 2.6, Python 2.7, Python 3.3
Added file: http://bugs.python.org/file31306/wakeup_fd_error.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Eli Bendersky]

Eli Bendersky added the comment:

The whole point of IntEnum and replacing stdlib constants with it was friendly 
str & repr out of the box. This means that "just printing out" an enum member 
should have a nice string representation. And "just printing out" means:

print(member)
"%s" % member
"{}".format(member)

!s/!r are quite esoteric - IntEnum should behave in the nicest way possible out 
of the box.

Let's just rig IntEnum's __format__ to do the right thing and not worry about 
Enum itself. I hope that mixin-with-Enum cases are rare (and most are IntEnum 
anyway), and in such rare cases users are free to lift the implementation from 
IntEnum.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18738] String formatting (% and str.format) issues with Enum

[Ethan Furman]

Ethan Furman added the comment:

+def __format__(self, *args, **kwargs):
+return self.__class__._member_type_.__format__(self.value, *args, 
**kwargs)
+

With this in the Enum class all members simply forward formatting to the 
mixed-in type, meaning that IntEnum behaves exactly like int for formatting 
purposes.  Another way of saying that is that the name attribute of an IntEnum 
member will only be used when the !r or !s codes are used.

If we are all good with that, then the question remaining is how should pure 
enums handle formatting?  Should they also go the value route, or should they 
go with the name?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18301] In itertools.chain.from_iterable() there is no cls argument

[Terry J. Reedy]

Terry J. Reedy added the comment:

My counter proposal #18752 is that chain.from_iterable become a deprecated 
alias for a new function, chain_iterable.  With '@classmethod' removed, the 
current Python equivalent would work for chain_iterable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18752] Make chain.from_iterable an alias for a new chain_iterable.

[Terry J. Reedy]

New submission from Terry J. Reedy:

It has become apparent from various discussions in recent months that 
chain.from_iterable is at least as useful as chain. In fact, I now think that 
'chain' should have been what chain.from_iterable is, with current chain(a,b,c) 
done as chain((a,b,d)). But too late for that.

Based on today's pydev post* by Eric Smith, I propose that chain_iterable (or 
chain_from_iterable, but I think the 'from' is not needed) be added to 
itertools and listed in the index table and documented as a function in its own 
right. 

This would make the long discussion of how to properly document 
chain.from_iterable (#18301) moot, as the method could be simply mentioned as a 
(deprecated) alias of chain_iterable.

* "I think that [itertools.chain.from_iterable] was a mistake, too. As a
recent discussion showed, it's not exactly discoverable. The fact that
it's not mentioned in the list of functions at the top of the
documentation doesn't help. And "chain" is documented as a "module
function", and "chain.from_iterable" as a "classmethod" making it all
the more confusing.

I think itertools.combinations and itertools.combinations_with_replacement is 
the better example of related
functions that should be followed. Not nested, no special parameters
trying to differentiate them: just two different function names."

If this proposal is rejected, then chain.iterable should be added to the index 
table. That would make it more discoverable, but not less confusing.

--
messages: 195290
nosy: eric.smith, rhettinger, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: Make chain.from_iterable an alias for a new chain_iterable.
type: enhancement
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10441] some stdlib modules need to be updated to handle SSL certificate validation

[Christian Heimes]

Christian Heimes added the comment:

I agree with Antoine. I'm not sure what is going to happen if you use a single 
SSLContext for unrelated services and different hosts -- not to mention SNI.

A while ago I tried a similar patch but I was stopped by the fact that OpenSSL 
doesn't provide an API to clone / copy a SSLContext object. Perhaps we have to 
introduce a factory that creates SSLContext object from a global configuration?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10441] some stdlib modules need to be updated to handle SSL certificate validation

[Antoine Pitrou]

Antoine Pitrou added the comment:

I would like to reject this approach. I am still adding Christian to the CC 
list, in case he has something to say about it :)

--
nosy: +christian.heimes
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13477] tarfile module should have a command line

[Antoine Pitrou]

Antoine Pitrou added the comment:

Regenerated patch against latest default (fixing conflicts).

--
nosy: +pitrou
stage: needs patch -> patch review
Added file: http://bugs.python.org/file31305/tarcli.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6132] Implement the GIL with critical sections in Windows

[Antoine Pitrou]

Antoine Pitrou added the comment:

I'm closing this as out-of-date (due to the new GIL in 3.2 and upwards).

--
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17970] Mutlithread XML parsing cause segfault

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
type:  -> crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13146] Writing a pyc file is not atomic

[Antoine Pitrou]

Antoine Pitrou added the comment:

Barry, do you still want to keep this issue open?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18732] IdleHistory.History: eliminate unused parameter; other cleanup.

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> fixed
stage: needs patch -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16190] Misleading warning in random module docs

[Antoine Pitrou]

Antoine Pitrou added the comment:

Agreed with a re-wording.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18732] IdleHistory.History: eliminate unused parameter; other cleanup.

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7339dcff171f by Terry Jan Reedy in branch '2.7':
Issue #18732: Remove unused* parameter output_sep from IdleHistory.History
http://hg.python.org/cpython/rev/7339dcff171f

New changeset 3105b78d3434 by Terry Jan Reedy in branch '3.3':
Issue #18732: Remove unused* parameter output_sep from IdleHistory.History
http://hg.python.org/cpython/rev/3105b78d3434

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13647] Python SSL stack doesn't securely validate certificate (as client)

[Antoine Pitrou]

Antoine Pitrou added the comment:

I'm gonna close this entry, since there's no actual issue to fix in Python.

--
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] '' % [1] doens't fail

[R. David Murray]

R. David Murray added the comment:

Yes, I suspect you are right that that is a bug...and a long standing one :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18739] math.log of a long returns a different value of math.log of an int

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee:  -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18739] math.log of a long returns a different value of math.log of an int

[Mark Dickinson]

Mark Dickinson added the comment:

Here's a patch that simply backports the Python 3.x code to Python 2.7.

--
keywords: +patch
Added file: http://bugs.python.org/file31304/issue18739.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16190] Misleading warning in random module docs

[Christian Heimes]

Christian Heimes added the comment:

Thomas H. Ptacek pointed me to a good explanation: 
http://security.stackexchange.com/a/3939

TL;DR: Just use /dev/urandom and be happy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18744] pathological performance using tarfile

[K Richard Pixley]

K Richard Pixley added the comment:

Here's a script that tests for the problem.

--
Added file: http://bugs.python.org/file31303/tarproblem.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18744] pathological performance using tarfile

[K Richard Pixley]

K Richard Pixley added the comment:

New info...

I see the degradation on most of the linux boxes I've tried:
* ubuntu-13.04, (raring), 64-bit
* rhel-5.4 64-bit
* rhel-5.7 64-bit
* suse-11 64-bit

I see some degradation on MacOsX-10.8.4 but it's in the acceptable range, more 
like 2x than 60x.  That is still suspicious, but not as problematic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18226] IDLE Unit test for FormatParagrah.py

[Terry J. Reedy]

Terry J. Reedy added the comment:

test.regrtest is semi-useless, at least for idle, in 2.7 -- see commit message. 
First run single file directly or python -m test.test_idle.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18425] IDLE Unit test for IdleHistory.py

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18425] IDLE Unit test for IdleHistory.py

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> fixed
superseder:  -> IdleHistory.History: eliminate unused parameter; other cleanup.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18226] IDLE Unit test for FormatParagrah.py

[Roundup Robot]

Roundup Robot added the comment:

New changeset 47307e7c80e1 by Terry Jan Reedy in branch '2.7':
Issue #18226: Fix ImportError and subsequent TypeError in 2.7 backport.
http://hg.python.org/cpython/rev/47307e7c80e1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] '' % [1] doens't fail

[Andrew Svetlov]

Andrew Svetlov added the comment:

For dict it is correct from my perspective.
"" % {'a': 'b'}
tries to substitute format specs like "%(a)s" and does nothing if spec is not 
present.

But list case like
"" % [1]
confuse me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18739] math.log of a long returns a different value of math.log of an int

[Tim Peters]

Tim Peters added the comment:

+1 on fixing it in 2.7, for the reasons Mark gave.

Way back when I introduced the original scheme, log(a_long) raised an 
exception, and the `int` and `long` types had a much higher wall between them.  
The original scheme changed an annoying failure into a "pretty good" 
approximation, and because int operations never returned a long back then the 
relatively rare code using longs forced their use.

In the meantime, you can change failing cases of log(some_long) to 
log(int(some_long)).  int(some_long) will convert to int if possible (avoiding 
this path in the log code), but return some_long unchanged otherwise.  In the 
old days, int(some_long) would fail if some_long was too big to fit in an int - 
that's different now too - and for the better :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] '' % [1] doens't fail

[R. David Murray]

R. David Murray added the comment:

haypo: str % dict is a feature:

  >>> "%(a)s" % {'a': 1, 'b': 2}
  '1'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18425] IDLE Unit test for IdleHistory.py

[Roundup Robot]

Roundup Robot added the comment:

New changeset 0e9d41edb2e4 by Terry Jan Reedy in branch '2.7':
Issue #18425: Unittests for idlelib.IdleHistory. First patch by R. Jayakrishnan.
http://hg.python.org/cpython/rev/0e9d41edb2e4

New changeset c4cac5d73e9d by Terry Jan Reedy in branch '3.3':
Issue #18425: Unittests for idlelib.IdleHistory. First patch by R. Jayakrishnan.
http://hg.python.org/cpython/rev/c4cac5d73e9d

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18727] test for writing dictionary rows to CSV

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thanks. One more question: is there a reason you don't simply call getvalue() 
at the end (rather than seek(0) followed by several readline() calls)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] '' % [1] doens't fail

[STINNER Victor]

STINNER Victor added the comment:

I don't understand why str % list and str % dict behaves differently than str % 
int:

>>> 'abc' % [1]
'abc'
>>> 'abc' % ([1],)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: not all arguments converted during string formatting
>>> 'abc' % 1
Traceback (most recent call last):
  File "", line 1, in 
TypeError: not all arguments converted during string formatting
>>> 'abc' % {1:2}
'abc'
>>> 'abc' % ({1:2},)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: not all arguments converted during string formatting

--
nosy: +haypo
title: ''' % [1] doens't fail -> '' % [1] doens't fail

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] ''' % [1] doens't fail

[R. David Murray]

R. David Murray added the comment:

What is it that doesn't fail?  The expression in the title is the beginning of 
a triple quoted string with no closing triple quote.

If you mean '' % [1] not falling, it has been that way forever (well, python2.4 
is as far back as I can test), so if it is deemed worth changing it certainly 
should not be backported.

I suspect it is some consequence of the C level similarities between lists and 
dicts, since '' % {} is supposed to not produce an error.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18751] A manager's server never joins its threads

[Antoine Pitrou]

New submission from Antoine Pitrou:

The Server class in multiprocessing.managers creates daemon threads and never 
joins them. This is in contrast with e.g. the Pool class, which creates daemon 
threads but uses util.Finalize to join them.

(not joining daemon threads may have adverse effects such as resource leaks or 
otherwise unexpected glitches)

Issue spawned from http://bugs.python.org/issue8713#msg195178.

--
components: Library (Lib)
messages: 195267
nosy: pitrou, sbt
priority: normal
severity: normal
status: open
title: A manager's server never joins its threads
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8713] multiprocessing needs option to eschew fork() under Linux

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ok, I enabled faulthandler in the child process and I got the explanation:
http://buildbot.python.org/all/builders/AMD64%20Windows%20Server%202008%20%5BSB%5D%20custom/builds/5/steps/test/logs/stdio

multiprocessing's manager Server uses daemon threads... Daemon threads are not 
joined when the interpreter shuts down, they are simply "frozen" at some point. 
Unfortunately, it may happen that a deamon thread is "frozen" while it was 
doing a cyclic garbage collection, which later triggers the assert.

I'm gonna replace the assert by a plain "if", then.

--
resolution:  -> fixed
stage: needs patch -> committed/rejected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[STINNER Victor]

STINNER Victor added the comment:

Another link:
http://article.gmane.org/gmane.comp.encryption.openssl.user/48480/match=does+child+process+still+need+reseeded+after+fork

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[STINNER Victor]

STINNER Victor added the comment:

This issue was already discussed in the atfork issue:
http://bugs.python.org/issue16500#msg179838

See also:
http://www.openwall.com/lists/oss-security/2013/04/12/3
"I believe it is wrong to fix this in PostgreSQL.  Rather, this is a
bug in the OpenSSL fork protection code.  It should either install a
fork hook, or reseed the PRNG from /dev/urandom if a PID change is
detected."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13248] deprecated in 3.2/3.3, should be removed in 3.4

[Senthil Kumaran]

Changes by Senthil Kumaran :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18750] ''' % [1] doens't fail

[Andrew Svetlov]

New submission from Andrew Svetlov:

I think this is a bug.
Can be reproduced on all Pythons (from 2.6 to 3.4a).
Maybe should be fixed for 3.4 only as backward incompatible change.

--
messages: 195263
nosy: asvetlov
priority: normal
severity: normal
status: open
title: ''' % [1] doens't fail

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13248] deprecated in 3.2/3.3, should be removed in 3.4

[Larry Hastings]

Changes by Larry Hastings :


--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18533] Avoid error from repr() of recursive dictview

[Terry J. Reedy]

Terry J. Reedy added the comment:

Please visit
http://www.python.org/psf/contrib/
http://www.python.org/psf/contrib/contrib-form/
and submit a Contributor Agreement. This process is complete when '*' appears 
after your name here, as with mine and others.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18532] hashlib.HASH objects should officially expose the hash name

[Christian Heimes]

Christian Heimes added the comment:

The builtin hash algorithms still had upper case names. I fixed it in 
revision http://hg.python.org/cpython/rev/9a4949f5d15c

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18532] hashlib.HASH objects should officially expose the hash name

[Christian Heimes]

Changes by Christian Heimes :


--
stage: patch review -> committed/rejected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18533] Avoid error from repr() of recursive dictview

[Ben North]

Ben North added the comment:

Is anything further needed from me before this can be reviewed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18742] Abstract base class for hashlib

[Christian Heimes]

Christian Heimes added the comment:

Updated patch.

I'm going to add documentation when everybody is happy with the patch.

--
nosy: +pitrou
Added file: http://bugs.python.org/file31302/hashlib_abc2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18749] [issue 18606] Re: Add statistics module to standard library

[Ezio Melotti]

Ezio Melotti added the comment:

> I hope I'm doing the right thing by replying in-line. This is my
> first code review, please let me know if I'm doing something wrong.
>
> By the way, the email hasn't gone to the tracker again. Is that a
> bug in the tracker? I've taken the liberty of changing the address
> to rep...@bugs.python.org.

Apparently that doesn't work, since it created a new issue :)

The best way is to reply directly on http://bugs.python.org/review/18606/.  You 
can either reply to the individual comments (better), or to the whole message 
like you did here.  I don't know if there's a way to reply via email.

--
nosy: +ezio.melotti
resolution:  -> invalid
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18749] [issue 18606] Re: Add statistics module to standard library

[Steven D'Aprano]

New submission from Steven D'Aprano:

I hope I'm doing the right thing by replying in-line. This is my first code 
review, please let me know if I'm doing something wrong.

By the way, the email hasn't gone to the tracker again. Is that a bug in the 
tracker? I've taken the liberty of changing the address to 
rep...@bugs.python.org.

On 15/08/13 22:58, ezio.melo...@gmail.com wrote:
>
> http://bugs.python.org/review/18606/diff/8927/Lib/statistics.py
> File Lib/statistics.py (right):
>
> http://bugs.python.org/review/18606/diff/8927/Lib/statistics.py#newcode113
> Lib/statistics.py:113: __date__ = "2013-08-13"
> Are these still needed after inclusion?

Probably not. Also the licence will need to be changed.

> http://bugs.python.org/review/18606/diff/8927/Lib/statistics.py#newcode194
> Lib/statistics.py:194: """
> This would be good in the rst docs, but IMHO docstrings should be less
> verbose.
> If you end up copy/pasting all these in the rst file, you will duplicate
> all the docs and they will risk to get out of sync (in addition to have
> to update both every time).

Personally, I like having detailed docs in the docstring, at my fingers in the 
interactive interpreter. But I'll follow the consensus here.

> http://bugs.python.org/review/18606/diff/8927/Lib/statistics.py#newcode277
> Lib/statistics.py:277: assert isinstance(x, float) and
> isinstance(partials, list)
> Is this a good idea?

I think so :-) add_partials is internal/private, and so I don't have to worry 
about the caller providing wrong arguments, say a non-float. But I want some 
testing to detect coding errors. Using assert for this sort of internal 
pre-condition is exactly what assert is designed for.

> http://bugs.python.org/review/18606/diff/8927/Lib/statistics.py#newcode524
> Lib/statistics.py:524: """mode(data [, max_modes]) -> mode(s)
> The form "mode(data, max_modes=1) -> mode(s)" is preferred.

Is it? I see plenty of examples in the standard library of that form, e.g. 
str.find:

find(...)
 S.find(sub [,start [,end]]) -> int

> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics.py
> File Lib/test/test_statistics.py (right):
>
> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics.py#newcode46
> Lib/test/test_statistics.py:46: 'missing name "%s" in __all__' % name)
> FWIW This should be already covered by test___all__.

Sorry, I don't understand this. test__all__?

[...]
> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics.py#newcode144
> Lib/test/test_statistics.py:144: assert data != sorted(data)
> Why not assertNotEqual?
>
> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics.py#newcode385
> Lib/test/test_statistics.py:385: assert x == inf
> Why not assertEqual?

In general, I use bare asserts for testing code logic, even if the code is test 
code. So if I use self.assertSpam(...) then I'm performing a unit test of the 
module being tested. If I use a bare assert, I'm asserting something about the 
test logic itself.

> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics.py#newcode417
> Lib/test/test_statistics.py:417: self.assertTrue(math.isnan(sum(data)))
> Since you seem to use this quite often, it might be better to define a
> assertIsNaN (and possibly assertIsNotNaN) in NumericTestCase and provide
> a better error message in case of failure.
> The same could apply for assertIsInf.
>
> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics.py#newcode913
> Lib/test/test_statistics.py:913: self.assertTrue(isinstance(result,
> Decimal))
> assertIsInstance

I used to be able to remember all the unittest assert* methods... there are so 
many now, 31 not including deprecated aliases.

> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics_approx.py
> File Lib/test/test_statistics_approx.py (right):
>
> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics_approx.py#newcode1
> Lib/test/test_statistics_approx.py:1: """Numeric approximated equal
> comparisons and unit testing.
> Do I understand correctly that this is just an helper module used in
> test_statistics and that it doesn't actually test anything from the
> statistics module?

Correct. It does, however, test itself.

> http://bugs.python.org/review/18606/diff/8927/Lib/test/test_statistics_approx.py#newcode137
> Lib/test/test_statistics_approx.py:137: # and avoid using
> TestCase.almost_equal, because it sucks :-)
> Could you elaborate on this?

Ah, I misspelled "TestCase.AlmostEqual".

- Using round() to test for equal-to-some-tolerance is quite an idiosyncratic 
way of doing approx-equality tests. I've never seen anyone do it that way 
before. It surprises me.

- It's easy to think that ``places`` means significant figures, not decimal 
places.

- There's now a delta argument (when was that added?) that is the same as my 
absolute error tolerance ``tol``, but no relative error argument.

- You can't set a  per-instance erro

[issue18747] Re-seed OpenSSL's PRNG after fork

[Antoine Pitrou]

Antoine Pitrou added the comment:

> When the OpenSSL's CPRNG is already initialized before 3) than all child
> processes created by 3) will have almost the same PRNG state. According
> to http://bugs.ruby-lang.org/issues/4579 the PRNG will return the same
> value when PID numbers are recycled.

Thanks. Here is some discussion of the reseeding strategy:
http://marc.info/?l=openssl-dev&m=130432419329454&w=2

More precisely, instead of reseeding in the child, you can simply
perturb the PRNG with a constant in the parent, to make sure the same
PRNG state doesn't get re-used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Christian Heimes]

Christian Heimes added the comment:

Am 15.08.2013 15:14, schrieb Antoine Pitrou:
>> Python doesn't have a builtin PRNG.
> 
> Of course it does. It's in the random module, and you can seed() it:

Now you are nit-picking. Although random is a PRNG, it is not a CPRNG.
I'm clearly talking about the integrity of a CPRNG here.

>> We use the OS's CPRNG such as /dev/urandom or CryptGenRandom().
> 
> "We"?

Python's os.urandom() and _PyOS_URandom().

> Well, are they affected? I haven't understood your previous answer.
> ("OpenSSL uses its own PRNG to create (amongst others) session keys for SSL 
> connections")

Yes, they are are affected. A forking HTTPS server does:

1) listen() on a TCP port
2) accept() new TCP connection
3) fork() a child process to handle request
4) do SSL handshake in the child process
5) read/write data in child process

When the OpenSSL's CPRNG is already initialized before 3) than all child
processes created by 3) will have almost the same PRNG state. According
to http://bugs.ruby-lang.org/issues/4579 the PRNG will return the same
value when PID numbers are recycled.

> Note that it seems debatable whether it's an OpenSSL bug:
> http://www.openwall.com/lists/oss-security/2013/04/12/3

It probably is an OpenSSL bug but the declaration doesn't help us. It's
not the first time Python has to work around OpenSSL, e.g. #18709.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Python doesn't have a builtin PRNG.

Of course it does. It's in the random module, and you can seed() it:

>>> random.seed(5)
>>> random.random()
0.6229016948897019
>>> random.random()
0.7417869892607294
>>> random.seed(5)
>>> random.random()
0.6229016948897019

See e.g. issue12856. And the multiprocessing module:
http://hg.python.org/cpython/file/92039fb68483/Lib/multiprocessing/forkserver.py#l195

> We use the OS's CPRNG such as /dev/urandom or CryptGenRandom().

"We"?

> It's not only multiprocessing. What about forking webservers etc.
> that use HTTPS?

Well, are they affected? I haven't understood your previous answer.
("OpenSSL uses its own PRNG to create (amongst others) session keys for SSL 
connections")

Note that it seems debatable whether it's an OpenSSL bug:
http://www.openwall.com/lists/oss-security/2013/04/12/3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18748] libgcc_s.so.1 must be installed for pthread_cancel to work

[Maries Ionel Cristian]

New submission from Maries Ionel Cristian:

Running the file couple of times will make the interpreter fail with: 
libgcc_s.so.1 must be installed for pthread_cancel to work

>From what I've seen it is triggered from PyThread_delete_key (tries to load 
>libgcc_s.so at that time).

How does it happen?

The main thread will close fd 4 (because fh object is getting dereferenced to 
0) exactly at the same time libpthread will try to open and read libgcc_s.so 
with the same descriptor (4)

It's fairly obvious that the file handling in bug.py is a bug, but the 
interpreter should not crash like that !

This doesn't happen on python2.7. Also, python2.7 appears to be linked with 
libgcc_s.so.1 directly while the 3.x does not (I've tried 3.2 from ubuntu 
repos, and built 3.3 and 3.4 myself on ubuntu 12.04.2) - at least that's what 
ldd indicates.

--
components: Build, Extension Modules
files: bug.py
messages: 195253
nosy: ionel.mc
priority: normal
severity: normal
status: open
title: libgcc_s.so.1 must be installed for pthread_cancel to work
type: crash
versions: Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file31301/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Christian Heimes]

Christian Heimes added the comment:

Python doesn't have a builtin PRNG. We use the OS's CPRNG such as /dev/urandom 
or CryptGenRandom(). Both use a system wide state and are not affected by 
process state. OpenSSL's PRNG is different because it uses an internal state. 
AFAIK it only polls the system's entropy poll when the PRNG is used for the 
first time.

It's not only multiprocessing. What about forking webservers etc. that use 
HTTPS?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Antoine Pitrou]

Antoine Pitrou added the comment:

> The ssl module exposes OpenSSL's PRNG and advertises the API as secure
> CPRNG: http://docs.python.org/3/library/ssl.html#random-generation

AFAICT, Python's PRNG isn't reset after fork, so I don't think OpenSSL's
should be reset.
OTOH, multiprocessing does reseed the random module after fork, so it
should also do so for the ssl module if already loaded.

We may add a note in the ssl docs stating that it's better to reseed
after fork().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8713] multiprocessing needs option to eschew fork() under Linux

[Antoine Pitrou]

Antoine Pitrou added the comment:

Using the custom builders, it seems to happen randomly in test_rlock:

test_rlock (test.test_multiprocessing_spawn.WithManagerTestLock) ... Assertion 
failed: !collecting, file ..\Modules\gcmodule.c, line 1617
ok

http://buildbot.python.org/all/builders/AMD64%20Windows%20Server%202008%20%5BSB%5D%20custom

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Christian Heimes]

Christian Heimes added the comment:

The ssl module exposes OpenSSL's PRNG and advertises the API as secure CPRNG: 
http://docs.python.org/3/library/ssl.html#random-generation

OpenSSL uses its own PRNG to create (amongst others) session keys for SSL 
connections.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16500] Add an 'atfork' module

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18335] Add textwrap.dedent, .indent, as str methods.

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Antoine Pitrou]

Antoine Pitrou added the comment:

Are there any uses of the OpenSSL PRNG from Python?
Is the PRNG used for SSL session keys, or another mechanism?

--
nosy: +pitrou, sbt
type: security -> enhancement
versions:  -Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18747] Re-seed OpenSSL's PRNG after fork

[Christian Heimes]

New submission from Christian Heimes:

A couple of reports and check-in messages like

  Postgres / pgcrypto CVE-2013-1900
  http://bugs.ruby-lang.org/issues/4579
  http://www.exim.org/lurker/message/20130402.171710.92f14a60.fi.html

suggests that OpenSSL's PRNG should be reset or re-seeded after fork(). 
Otherwise child processes can generate the same or similar pseudo random values.

Python doesn't have an API to run code before and after fork yet. The patch 
uses pthread_atfork() for the task. It's available on all pthread platforms -- 
which are all official supported platforms that have fork(), too.

The patch doesn't use RAND_cleanup() like Postgres because child process would 
hav to initial the PRNG again by opening and reading from /dev/urandom. The 
atfork prepare hook pulls from random bytes from the PRNG and stores them in a 
static buffer. The child handler seeds the PRNG from that buffer + pid + 
current time. PID and current time are mixed into the state to extenuate race 
conditions.

--
components: Extension Modules
files: openssl_prng_atfork.patch
keywords: patch
messages: 195247
nosy: christian.heimes, haypo
priority: normal
severity: normal
stage: patch review
status: open
title: Re-seed OpenSSL's PRNG after fork
type: security
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file31300/openssl_prng_atfork.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18335] Add textwrap.dedent, .indent, as str methods.

[Julian Berman]

Changes by Julian Berman :


--
nosy: +Julian

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18739] math.log of a long returns a different value of math.log of an int

[Mark Dickinson]

Mark Dickinson added the comment:

Ah, Tim saw through my cunningly-laid false trail of incorrect issue numbers.  
Step 1 of my world domination plan is foiled!

Yes, let's fix this.  In my mind, it's definitely a bug that ints and longs 
aren't interchangeable here, and it would be nice to have 2.7 and 3.x behaving 
the same way.  I do have some worries about the fix breaking / changing 
existing code, but given that it'll only affect log of a long, those worries 
barely register.

--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18219] csv.DictWriter is slow when writing files with large number of columns

[Peter Otten]

Peter Otten added the comment:

Note that set operations on dict views work with lists, too. So the only change 
necessary is to replace

wrong_fields = [k for k in rowdict if k not in self.fieldnames]

with

wrong_fields = rowdict.keys() - self.filenames

(A backport to 2.7 would need to replace keys() with viewkeys())

--
nosy: +peter.otten

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18746] test_threading.test_finalize_with_trace() fails on FreeBSD buildbot

[STINNER Victor]

Changes by STINNER Victor :


--
keywords: +patch
Added file: http://bugs.python.org/file31299/finalize_clear_trace.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18746] test_threading.test_finalize_with_trace() fails on FreeBSD buildbot

[STINNER Victor]

New submission from STINNER Victor:

The following test fails on FreeBSD buildbot:

def test_finalize_with_trace(self):
# Issue1733757
# Avoid a deadlock when sys.settrace steps into threading._shutdown
assert_python_ok("-c", """if 1:
import sys, threading

# A deadlock-killer, to prevent the
# testsuite to hang forever
def killer():
import os, time
time.sleep(2)
print('program blocked; aborting')
os._exit(2)
t = threading.Thread(target=killer)
t.daemon = True
t.start()

# This is the trace function
def func(frame, event, arg):
threading.current_thread()
return func

sys.settrace(func)
""")

I ran it manually on my FreeBSD 9.1 VM, I get the following error. I don't know 
if it's the same error than the buildbot.

(...)
# clear builtins._
(...)
# restore sys.stderr
# cleanup __main__
# cleanup[1] _sysconfigdata
(...)
# cleanup[1] threading
Exception ignored in: ._remove at 
0x801b11058>
Traceback (most recent call last):
  File "/usr/home/haypo/prog/python/default/Lib/_weakrefset.py", line 38, in 
_remove
  File "x.py", line 17, in func
AttributeError: 'NoneType' object has no attribute 'current_thread'
# cleanup[1] _weakrefset
(...)
# cleanup[3] _codecs
PyThreadState_Clear: warning: thread still has a frame


The weakref is probably threading._dangling.

IMO Py_Finalize() should first clear the trace function. Tracing Python 
exception while Python is dying (exiting) is insane. I'm surprised that it 
works :-)

--
messages: 195244
nosy: haypo, ncoghlan, pitrou
priority: normal
severity: normal
status: open
title: test_threading.test_finalize_with_trace() fails on FreeBSD buildbot
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18746] test_threading.test_finalize_with_trace() fails on FreeBSD buildbot

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18296] test_os.test_trailers() is failing on AMD64 FreeBSD 9.0 dtrace 3.x

[Roundup Robot]

Roundup Robot added the comment:

New changeset 924d327da3af by Victor Stinner in branch '3.3':
Issue #18296: Try to fix TestSendfile.test_trailers() of test_os on FreeBSD
http://hg.python.org/cpython/rev/924d327da3af

New changeset 92039fb68483 by Victor Stinner in branch 'default':
(Merge 3.3) Issue #18296: Try to fix TestSendfile.test_trailers() of test_os on 
FreeBSD
http://hg.python.org/cpython/rev/92039fb68483

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18296] test_os.test_trailers() is failing on AMD64 FreeBSD 9.0 dtrace 3.x

[koobs]

koobs added the comment:

As per our IRC conversation, our 'koobs-freebsd10' bot also reproduces the 
failure and can be used to test the patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17628] str==str: compare the first character before calling memcmp()

[Christian Heimes]

Christian Heimes added the comment:

It's also bad for memory read performance if the string is rather long. The 
memory controller performs best when code reads memory sequential. The talk 
http://www.youtube.com/watch?v=MC1EKLQ2Wmg about mythbusting modern hardware 
sums it up real nice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17628] str==str: compare the first character before calling memcmp()

[STINNER Victor]

STINNER Victor added the comment:

In issue #18719, Raymond modified Python 2.7, but he didn't touch the following 
macro:

#define Py_UNICODE_MATCH(string, offset, substring) \
((*((string)->str + (offset)) == *((substring)->str)) && \
((*((string)->str + (offset) + (substring)->length-1) == *((substring)->str 
+ (substring)->length-1))) && \
 !memcmp((string)->str + (offset), (substring)->str, 
(substring)->length*sizeof(Py_UNICODE)))

It was said that looking for the last character before calling memcmp() is 
inefficient for the CPU cache. This macro should also be modified.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16463] testConnectTimeout of test_timeout TCPTimeoutTestCasefailures fails intermittently

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Or we can wrap the resolve_address() method with the @functools.lru_cache() 
> decorator.

Sounds ok to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18745] Test enum in test_json is ignorant of infinity value

[Vajrasky Kok]

New submission from Vajrasky Kok:

Test enum json in Lib/test/test_json/test_enum.py is ignorant of infinity 
values. Also, NaN, but since NaN is a weirdo, let's not take that into account.

The unit test should represent of what will work in every case. For example:

def test_floats(self):
 for enum in FloatNum:
self.assertEqual(self.dumps(enum), repr(enum.value))

This will fail if enum is infinity.

This wisdom about infinity was bestowed upon me when I was reading 
Lib/test/test_json/test_float.py.

def test_floats(self):
for num in [1617161771.7650001, math.pi, math.pi**100, math.pi**-100, 
3.1]:
self.assertEqual(float(self.dumps(num)), num)
self.assertEqual(self.loads(self.dumps(num)), num)

def test_ints(self):
for num in [1, 1<<32, 1<<64]:
self.assertEqual(self.dumps(num), str(num))
self.assertEqual(int(self.dumps(num)), num)

As you can see, in float case, we don't use str(num) because it does not work 
with infinity.

Attached the patch to refactor the test to handle infinity value. For the 
completeness sake, I added the case of negative infinity and NaN as well.

--
components: Tests
files: add_infinity_to_test_enum_in_json.patch
keywords: patch
messages: 195238
nosy: ethan.furman, vajrasky
priority: normal
severity: normal
status: open
title: Test enum in test_json is ignorant of infinity value
versions: Python 3.4
Added file: 
http://bugs.python.org/file31298/add_infinity_to_test_enum_in_json.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18682] [PATCH] remove bogus codepath from pprint._safe_repr

[Michal Vyskocil]

Michal Vyskocil added the comment:

The fast scalars approach looks great!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18743] References to non-existant "StringIO" module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17628] str==str: compare the first character before calling memcmp()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Perhaps it worth manually inline unicode_eq() in these tight inner loops. Then 
we can move "PyUnicode_GET_LENGTH(a) * PyUnicode_KIND(a)" out of the loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18744] pathological performance using tarfile

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you please provide a simple script which shows the problem?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com