[Python-Dev] The Clinic Has Landed

[Larry Hastings]

"Hello, I'd like to have an argument, please."

After more than a year of development, Argument Clinic is now checked in 
to CPython trunk.  Thanks, everyone, for your feedback and encouragement!



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The Clinic Has Landed

[Nick Coghlan]

On 19 October 2013 17:13, Larry Hastings  wrote:
> "Hello, I'd like to have an argument, please."
>
> After more than a year of development, Argument Clinic is now checked in to
> CPython trunk.  Thanks, everyone, for your feedback and encouragement!

Huzzah! Thanks for your efforts pushing through such a huge step
towards finally being able to fix this current limitation:

>>> inspect.signature(open)
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/ncoghlan/devel/py3k/Lib/inspect.py", line 1533, in signature
raise ValueError(msg)
ValueError: no signature found for builtin function 

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Context management patterns

[Glenn Linderman]

On 10/18/2013 11:38 PM, Nick Coghlan wrote:

However, that's a confusion about exception handling in general, not
about the suppress context manager in particular. The same potential
for conceptual confusion exists between:

 for name in ("somefile.tmp", "someotherfile.tmp"):
 try:
 os.remove(name)
 except FileNotFoundError:
 pass

and:

 try:
 for name in ("somefile.tmp", "someotherfile.tmp"):
os.remove(name)
 except FileNotFoundError:
 pass

At the syntactic level, when composing compound statements, the order
of nesting*always*  matters. The with/for and for/with constructs are
different, just as if/for and for/if are different. If a student makes
it through an introductory Python course without learning that much,
I'd have grave doubts about that course


Students that actually take courses might be OK... they might even read 
some of the documentation.


Indeed the problem can be produced with try/except syntax or with 
with/suppress. That has been obvious from the beginning of the 
discussion, and stated repeatedly.  I suppose it is the "context 
management" connotation that with currently has, that makes me think 
that with/suppress is reasonably likely to get misused... in context 
management uses, one puts long blocks of code under with (not always, 
but often).  In this particular case, that is a problem, yet not all the 
control flow examples would be limited to one line of code in the with 
block. So this is still a very special case, for a very small benefit, 
with an unusual limitation.


I have a new color of paint:

with pass_if( FileNotFoundError ): os_remove("somefile.tmp")

One thing about bikeshedding... after trying lots of colors, you might 
actually find one everyone likes. But if they aren't put out there to 
see, no one knows if they like it or not. And if there are a significant 
number of people that don't like the color, it either means the sun is 
in their eyes, or the color isn't the best, and that is sometimes hard 
to discern.


While I already realized that the with construct is really control flow, 
even the names that indicated that that have been suggested so far, even 
by me, didn't really capture the type of control flow. It hit me, 
reading your last message, that the key part is "pass"... it is an 
action (control flow), and it is the action that would be taken if 
written with try/except syntax.  It isn't the exception, or the 
catching, that are the key it is the passing... passing over the 
with block because the exception happened. That even implies you don't 
want to pass much, certainly not a whole loop full of stuff, or a long 
string of statements.


Sadly, all the documentation that references "suppressing" an exception, 
is misleading. Misleading enough that you convinced yourself to rename 
from ignore to suppress after reading it, even though the two are nearly 
synonymous.  The exceptions are not ignored, and the exceptions are not 
suppressed. They are handled by passing (which avoids the need for 
surrounding code to handle them, which is sort of like being ignored or 
suppressed, from the outside looking in... but pass describes the 
behavior for both inside and outside viewpoints!).



I believe your underlying concerns are actually with the non-local
flow control possibilities that are inherent in a language that offers
both exceptions and the ability to suppress them.


Are you referring to a different type of "suppress" than the one you 
just implemented? Where the language allows you to suppress an 
exception, and execution proceeds with the next statement? Where it 
would be more of a declarative style of suppression rather than an 
active "catch and pass". If so, it is more proof that "suppress" is the 
wrong term for your new feature.


Thanks again for sharing the big picture: you have before, but I see 
you've evolved it slightly.  Delayed error handling I've had occasion to 
do, but not yet in Python. The constrained jump pattern is interesting, 
but the example is not complex enough to justify the syntax: the with 
solution is actually longer... if you initialized result they'd be the 
same length.  The problem with the for/else construct is that the "not 
found" case falls through to the "found" case, which can be awkward, but 
that awkwardness isn't really demonstrated in the example. "exit_label" 
is an awkward name, too, since there are no labels, and no exit(). (yes, 
I know you meant exit the loop; naming is hard.)


I haven't yet wrapped my mind around all the possibilities of with 
creating an "empty" object, which gets populated by the various 
execution paths taken, but that combination does seem quite a bit more 
powerful than the simple with statement alone, which only sits at the 
front of a block. try/except/finally/else has lots of keywords and lots 
of places to put code :) Which doesn't always make the code clear, so 
abstractions using wi

Re: [Python-Dev] Context management patterns

[Nick Coghlan]

On 19 October 2013 20:47, Glenn Linderman  wrote:
> Thanks again for sharing the big picture: you have before, but I see you've
> evolved it slightly.  Delayed error handling I've had occasion to do, but
> not yet in Python. The constrained jump pattern is interesting, but the
> example is not complex enough to justify the syntax: the with solution is
> actually longer... if you initialized result they'd be the same length.  The
> problem with the for/else construct is that the "not found" case falls
> through to the "found" case, which can be awkward, but that awkwardness
> isn't really demonstrated in the example. "exit_label" is an awkward name,
> too, since there are no labels, and no exit(). (yes, I know you meant exit
> the loop; naming is hard.)

You exit with statements - that's why the final method is called __exit__.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] An interesting exception handling quirk

[Nick Coghlan]

I've been looking at Terry's suggestion of using a class based
implementation for contextlib.suppress, and rediscovered why I
suggested Raymond switch the original implementation to using
@contextmanager: recreating the *exact* exception subclass check from
Python is actually difficult these days.

The exception handling only looks for concrete subclasses, ignoring
the dynamic subclass hook:

>>> import abc
>>> class ExceptionABC(abc.ABC, Exception):
... pass
...
>>> ExceptionABC.register(ZeroDivisionError)

>>> issubclass(ZeroDivisionError, ExceptionABC)
True
>>> try:
... 1/0
... except ExceptionABC:
... print("It's a subclass of ExceptionABC!")
... except ZeroDivisionError:
... print("But not according to the exception handling!")
...
But not according to the exception handling!

I don't actually have a problem with the current behaviour (since
raise and except both enforce a requirement for the types involved to
be concrete subclasses of BaseException), but as far as I'm aware we
don't explicitly state anywhere that if exception handling and an
issubclass check give different answers for whether or not exception A
is a subclass of exception B, then the fault for any resulting
differences in behavious lies with the way A and/or B are defined, not
with the code using issubclass to check for exception inheritance.

Having noticed the discrepancy, I feel like it should be explicitly
recorded somewhere in the language reference, I'm just not sure where.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Context management patterns

[Nick Coghlan]

On 19 October 2013 20:47, Glenn Linderman  wrote:
> Sadly, all the documentation that references "suppressing" an exception, is
> misleading. Misleading enough that you convinced yourself to rename from
> ignore to suppress after reading it, even though the two are nearly
> synonymous.

You misunderstand the two components that formed my rationale for renaming it.

The first part of the rationale is that re-reading all of the current
docs reminded me that this pattern *already* had a consistent name in
the language and library reference (i.e. suppress), and this name was
even mentioned all the way back in PEP 343 (although it wasn't the
primary term there - that honour went to "swallow", which
understandably didn't make it into the reference docs). To avoid
introducing an inconsistency into the documentation for Python 3.4, I
had to choose between rewording all the existing context management
and with statement docs or just using the established name for the new
construct (and I chose the latter path).

The second part of my rationale stems from the fact that ignore and
suppress aren't quite synonyms, and that suppressing something is
*far* more active than ignoring it. If you completely ignore
something, it should be as if it never happened - you certainly don't
take any kind of action in response to it. When you suppress
something, by contrast, there's a clear implication that it still
happens, and then you do something later to make it go away. That
means the potential for confusion changes:

Plausible interpretations of ignore(SomeException):
- that kind of exception is genuinely ignored by never raising it in
the first place and proceeding with the next statement immediately
after the one where it would have been raised
- the exception is raised, but mostly ignored by proceeding with the
next statement in the with statement body
- the exception is raised, but kind of ignored by proceeding with the
next statement after the with statement

Plausible interpretations of suppress(SomeException):
- the exception is raised, but suppressed immediately after the
statement that raised it
- the exception is raised, but suppressed at the end of the with
statement (which is the first change the context manager has to see
it)

With "suppress", the implication that the "exception is raised" part
still happens is *much* stronger, suggesting that the name is not only
more consistent with the existing docs, but also more accurate in an
absolute sense.

Independently of that, one of the things that has become clear to me
in these threads is that many experienced Python developers don't
realise that with statements actually *are* intended to be usable as
control flow constructs (albeit in a quite restrained way, with their
influence on the flow of control being limited solely to the ability
to suppress exceptions). If coming to grips with the behaviour of
contextlib.suppress expands people's horizons beyond merely using them
for resource, state and transaction management, then I see that as a
good thing.

Now, some of the people in that last group are actually *more* likely
to be confused than beginners, since they already have the *mistaken*
impression that with statements don't affect control flow. And if you
have *that* impression, then you're more likely to guess that
suppress() has to be a state management context manager that is giving
the interpreter the instruction to not raise the named exception
types. Discovering that it *doesn't* work that way may then help to
enlighten them as to the true nature of the underlying construct.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

[Christian Heimes]

Am 19.10.2013 00:56, schrieb Guido van Rossum:
> Thanks! That's probably fine for now -- it means the standard library
> doesn't know where the root certificates are. We had a huge discussion
> about this over on python-tulip:
> https://groups.google.com/forum/#!topic/python-tulip/c_lqdFjPEbE
> 
> TL;DR: The stdlib openssl wrapper ought to know where each platform
> stores its root certificates and automatically use them, but it
> currently doesn't always. Users who really don't care but still want to
> use SSL must create an SSL context with verify_mode set to ssl.CERT_NONE
> (and live with the risk, obviously). This stuff passes on OS X only
> because there's a system openssl library that always uses the system
> root certificates.
> 
> If anyone can help fixing the ssl.py module (or the _ssl extension) so
> that sslcontext.set_default_verify_paths() uses the system root certs on
> Windows that would be a huge help. (I have tried this on an Ubuntu box
> too, and there it actually works.)

I have worked on some patches and even started to write a PEP about it.
You can find an old version of my PEP at
https://bitbucket.org/tiran/peps/src/tip/pep-.txt . The PEP contains
a list of possible locations of root CA certs.

The root CA certificate situation is troublesome. Several parsers for
Mozilla's NSS certdata.txt are plain wrong and don't handle purpose /
trust settings correctly. Even Ubuntu is affected by the bug. The
/etc/ssl/certs/ directory contains certificates that are NOT suitable
for server cert verification.

A couple of months I had a long and fruitful discussion with MAL about
the issue. Egenix PyOpenSSL installer comes with a root CA bundle. He
tried a couple of approaches to handle trust settings with OpenSSL
means. Eventually MAL had to split up the bundle into multiple files for
each purpuse, see
http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.2.1.0.1.5.html

We should *really* write a PEP about it, specify all details and get a
proper review from real experts. This stuff is super complex and highly
fragile. :(

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] An interesting exception handling quirk

[Antoine Pitrou]

On Sat, 19 Oct 2013 21:41:17 +1000
Nick Coghlan  wrote:
> 
> Having noticed the discrepancy, I feel like it should be explicitly
> recorded somewhere in the language reference, I'm just not sure where.

Since it's a quirk, I don't think it should be mentioned in the
language reference. Also, see http://bugs.python.org/issue12029

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Antoine Pitrou]

On Sat, 19 Oct 2013 14:25:28 +0200 (CEST)
christian.heimes  wrote:
>  
> -   .. note:: A fast implementation of *pbkdf2_hmac* is only available with
> -  OpenSSL 1.0 and newer. The Python implementation uses an inline
> -  version of :mod:`hmac` and is about three times slower. Contrary to
> -  OpenSSL's current code the length of the password has only a minimal
> -  impact on the runtime of the Python implementation.
> +   .. note:: A fast implementation of *pbkdf2_hmac* is available with 
> OpenSSL.
> +  The Python implementation uses an inline version of :mod:`hmac`. It is
> +  about three times slower and doesn't release the GIL.

The documentation should stop talking about the Python implementation
if the C implementation is always used by default.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] An interesting exception handling quirk

[Nick Coghlan]

On 19 October 2013 22:53, Antoine Pitrou  wrote:
> On Sat, 19 Oct 2013 21:41:17 +1000
> Nick Coghlan  wrote:
>>
>> Having noticed the discrepancy, I feel like it should be explicitly
>> recorded somewhere in the language reference, I'm just not sure where.
>
> Since it's a quirk, I don't think it should be mentioned in the
> language reference. Also, see http://bugs.python.org/issue12029

Ah, thanks for the reference. I'll comment further over there :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Christian Heimes]

Am 19.10.2013 14:54, schrieb Antoine Pitrou:
> On Sat, 19 Oct 2013 14:25:28 +0200 (CEST)
> christian.heimes  wrote:
>>  
>> -   .. note:: A fast implementation of *pbkdf2_hmac* is only available with
>> -  OpenSSL 1.0 and newer. The Python implementation uses an inline
>> -  version of :mod:`hmac` and is about three times slower. Contrary to
>> -  OpenSSL's current code the length of the password has only a minimal
>> -  impact on the runtime of the Python implementation.
>> +   .. note:: A fast implementation of *pbkdf2_hmac* is available with 
>> OpenSSL.
>> +  The Python implementation uses an inline version of :mod:`hmac`. It is
>> +  about three times slower and doesn't release the GIL.
> 
> The documentation should stop talking about the Python implementation
> if the C implementation is always used by default.

The C implementation is not used if Python is not compiled with OpenSSL
support.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Context management patterns

[Xavier Morel]

On 2013-10-19, at 08:38 , Nick Coghlan wrote:
>> The above example, especially if extended beyond two files, begs to used in
>> a loop, like your 5 line version:
>> 
>> 
>> for name in ("somefile.tmp", "someotherfile.tmp"):
>>   with suppress(FileNotFoundError):
>>os.remove(name)
>> 
>> which would be fine, of course.
>> 
>> But to some with less education about the how and why, it is not clear why
>> it couldn't be written like:
>> 
>> with suppress(FileNotFoundError):
>> 
>>for name in ("somefile.tmp", "someotherfile.tmp"):
>>os.remove(name)
>> 
>> yet to the cognoscenti, it is obvious there are seriously different
>> semantics.
> 
> However, that's a confusion about exception handling in general, not
> about the suppress context manager in particular. The same potential
> for conceptual confusion exists between:
> 
>for name in ("somefile.tmp", "someotherfile.tmp"):
>try:
>os.remove(name)
>except FileNotFoundError:
>pass
> 
> and:
> 
>try:
>for name in ("somefile.tmp", "someotherfile.tmp"):
>   os.remove(name)
>except FileNotFoundError:
>pass

It could work if the exceptions system was extended to a full-blow
conditions system though.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #18810: Be optimistic with stat calls when seeing if a directory

[Brett Cannon]

On Fri, Oct 18, 2013 at 9:06 PM, Nick Coghlan  wrote:

>
> On 19 Oct 2013 03:24, "brett.cannon"  wrote:
> >
> > http://hg.python.org/cpython/rev/11f2f4af1979
> > changeset:   86444:11f2f4af1979
> > user:Brett Cannon 
> > date:Fri Oct 18 13:24:13 2013 -0400
> > summary:
> >   Issue #18810: Be optimistic with stat calls when seeing if a directory
> > exists when checking for a package.
> >
> > Before there was an isdir check and then various isfile checks for
> > possible __init__ files when looking for a package.
> > This change drops the isdir check by leaning
> > on the assumption that a directory will not contain something named
> > after the module being imported which is not a directory. If the module
> > is a package then it saves a stat call. If there is nothing in the
> > directory with the potential package name it also saves a stat call.
> > Only if there is something in the directory named the same thing as
> > the potential package will the number of stat calls increase
> > (due to more wasteful __init__ checks).
>
> I don't follow this logic. There's now not even an existence check for the
> base name, so it reads to me like we will look for all the possible init
> file extensions even if there's *no* directory with an appropriate name.
>
> What am I missing?
>

``if cache_module in cache:``, the line above the _path_join() call and the
guard that blocks the entire package search.

-Brett


> Cheers,
> Nick.
>
> >
> > Semantically there is no change as the isdir check moved
> > down so that namespace packages continue to have no chance of
> > accidentally collecting non-existent directories.
> >
> > files:
> >   Lib/importlib/_bootstrap.py |19 +-
> >   Python/importlib.h  |  1537 +++---
> >   2 files changed, 777 insertions(+), 779 deletions(-)
> >
> >
> > diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
> > --- a/Lib/importlib/_bootstrap.py
> > +++ b/Lib/importlib/_bootstrap.py
> > @@ -1406,16 +1406,15 @@
> >  # Check if the module is the name of a directory (and thus a
> package).
> >  if cache_module in cache:
> >  base_path = _path_join(self.path, tail_module)
> > -if _path_isdir(base_path):
> > -for suffix, loader in self._loaders:
> > -init_filename = '__init__' + suffix
> > -full_path = _path_join(base_path, init_filename)
> > -if _path_isfile(full_path):
> > -return (loader(fullname, full_path),
> [base_path])
> > -else:
> > -# A namespace package, return the path if we don't
> also
> > -#  find a module in the next section.
> > -is_namespace = True
> > +for suffix, loader in self._loaders:
> > +init_filename = '__init__' + suffix
> > +full_path = _path_join(base_path, init_filename)
> > +if _path_isfile(full_path):
> > +return (loader(fullname, full_path), [base_path])
> > +else:
> > +# If a namespace package, return the path if we don't
> > +#  find a module in the next section.
> > +is_namespace = _path_isdir(base_path)
> >  # Check for a file w/ a proper suffix exists.
> >  for suffix, loader in self._loaders:
> >  full_path = _path_join(self.path, tail_module + suffix)
> > diff --git a/Python/importlib.h b/Python/importlib.h
> > --- a/Python/importlib.h
> > +++ b/Python/importlib.h
> > [stripped]
> >
> > --
> > Repository URL: http://hg.python.org/cpython
> >
> > ___
> > Python-checkins mailing list
> > python-check...@python.org
> > https://mail.python.org/mailman/listinfo/python-checkins
> >
>
>
> ___
> Python-checkins mailing list
> python-check...@python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #18416: Fix various os calls in importlib.machinery.FileFinder

[Brett Cannon]

On Fri, Oct 18, 2013 at 8:55 PM, Nick Coghlan  wrote:

>
> On 19 Oct 2013 02:01, "brett.cannon"  wrote:
> >
> > http://hg.python.org/cpython/rev/33844153cd02
> > changeset:   86438:33844153cd02
> > user:Brett Cannon 
> > date:Fri Oct 18 12:01:06 2013 -0400
> > summary:
> >   Issue #18416: Fix various os calls in importlib.machinery.FileFinder
> > now that self.path is no longer forced to '.'.
>
> Hmm, could this break subclasses in a similar way? It may be safer to keep
> the empty string -> period conversion, even though PathFinder itself
> doesn't rely on it any more.
>

Good point. I'll revert it.

-Brett


>  Cheers,
> Nick.
>
> >
> > files:
> >   Lib/importlib/_bootstrap.py | 4 +-
> >   Python/importlib.h  |  1554 +++---
> >   2 files changed, 780 insertions(+), 778 deletions(-)
> >
> >
> > diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
> > --- a/Lib/importlib/_bootstrap.py
> > +++ b/Lib/importlib/_bootstrap.py
> > @@ -1390,7 +1390,7 @@
> >  is_namespace = False
> >  tail_module = fullname.rpartition('.')[2]
> >  try:
> > -mtime = _os.stat(self.path).st_mtime
> > +mtime = _os.stat(self.path or _os.getcwd()).st_mtime
> >  except OSError:
> >  mtime = -1
> >  if mtime != self._path_mtime:
> > @@ -1432,7 +1432,7 @@
> >  """Fill the cache of potential modules and packages for this
> directory."""
> >  path = self.path
> >  try:
> > -contents = _os.listdir(path)
> > +contents = _os.listdir(path or _os.getcwd())
> >  except (FileNotFoundError, PermissionError, NotADirectoryError):
> >  # Directory has either been removed, turned into a file, or
> made
> >  # unreadable.
> > diff --git a/Python/importlib.h b/Python/importlib.h
> > --- a/Python/importlib.h
> > +++ b/Python/importlib.h
> > [stripped]
> >
> > --
> > Repository URL: http://hg.python.org/cpython
> >
> > ___
> > Python-checkins mailing list
> > python-check...@python.org
> > https://mail.python.org/mailman/listinfo/python-checkins
> >
>
>
> ___
> Python-checkins mailing list
> python-check...@python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

[Charles-François Natali]

>>> ``get_tracemalloc_memory()`` function:
>>>
>>> Get the memory usage in bytes of the ``tracemalloc`` module as a
>>> tuple: ``(size: int, free: int)``.
>>>
>>> * *size*: total size of bytes allocated by the module,
>>>   including *free* bytes
>>> * *free*: number of free bytes available to store data
>>
>> What's *free* exactly? I assume it's linked to the internal storage
>> area used by tracemalloc itself, but that's not clear at all.
>>
>> Also, is the tracemalloc overhead included in the above stats (I'm
>> mainly thinking about get_stats() and get_traced_memory()?
>> If yes, I find it somewhat confusing: for example, AFAICT, valgrind's
>> memcheck doesn't report the memory overhead, although it can be quite
>> large, simply because it's not interesting.
>
> My goal is to able to explain how *every* byte is allocated in Python.
> If you enable tracemalloc, your RSS memory will double, or something
> like that. You can use get_tracemalloc_memory() to add metrics to a
> snapshot. It helps to understand how the RSS memory evolves.
>
> Basically, get_tracemalloc_size() is the memory used to store traces.
> It's something internal to the C module (_tracemalloc). This memory is
> not traced because it *is* the traces... and so is not counted in
> get_traced_memory().
>
> The issue is probably the name (or maybe also the doc): would you
> prefer get_python_memory() / get_traces_memory() names, instead of
> get_traced_memory() / get_tracemalloc_memory()?

No, the names are fine as-is.

> FYI Objects allocated in tracemalloc.py (real objects, not traces) are
> not counted in get_traced_memory() because of a filter set up by
> default (it was not the case in previous versions of the PEP). You can
> remove the filter using tracemalloc.clear_filters() to see this
> memory. There are two exceptions: Python objects created for the
> result of get_traces() and get_stats() are never traced for
> efficiency. It *is* possible to trace these objects, but it's really
> too slow. get_traces() and get_stats() may be called outside
> tracemalloc.py, so another filter would be needed. Well, it's easier
> to never trace these objects. Anyway, they are not interesting to
> understand where your application leaks memory.

Perfect, that's all I wanted to know.

> get_object_trace(obj) is a shortcut for
> get_trace(get_object_address(obj)). I agree that the wrong size
> information can be surprising.
>
> I can delete get_object_trace(), or rename the function to
> get_object_traceback() and modify it to only return the traceback.
>
> I prefer to keep the function (modified for get_object_traceback).
> tracemalloc can be combined with other tools like Melia, Heapy or
> objgraph to combine information. When you find an interesting object
> with these tools, you may be interested to know where it was
> allocated.

If you mean modify it to return only the trace, then that's fine.
As for the name, traceback does indeed sound less confusing than
trace, but we should just make sure that the names are consistent
across the API (i.e. always use "trace" or "always use "traceback",
not both).

>>> ``get_trace(address)`` function:
>>>
>>> Get the trace of a memory block as a ``(size: int, traceback)``
>>> tuple where *traceback* is a tuple of ``(filename: str, lineno:
>>> int)`` tuples, *filename* and *lineno* can be ``None``.
>>>
>>> Return ``None`` if the ``tracemalloc`` module did not trace the
>>> allocation of the memory block.
>>>
>>> See also ``get_object_trace()``, ``get_stats()`` and
>>> ``get_traces()`` functions.
>>
>> Do you have example use cases where you want to work with a raw addresses?
>
> An address is the unique key to identify a memory block. In Python,
> you don't manipulate directly memory blocks, that's why you have a
> get_object_address() function (link objects to traces).
>
> I added get_trace() because get_traces() is very slow. It would be
> stupid to call it if you only need one trace of a memory block.
>
> I'm not sure that this function is really useful. I added it to
> workaround the performance issue, and because I believe that someone
> will need it later :-)
>
> What do you suggest for this function?

Well, I can certainly find a use-case for get_object_trace(): even if
it uses get_trace() internally, I'm not convinced that the later is
useful.
If we cannot come up with a use case for working with raw addresses,
I'm tempted to just keep get_object_trace() public, and make
get_object_address() and get_trace() private.
In short, don't make any address-manipulating function public.

>> Are those ``match`` methods really necessary for the end user, i.e.
>> are they worth being exposed as part of the public API?
>
> (Oh, I just realized that match_lineno() and may lead to bugs, I removed it.)
>
> Initially, I exposed the methods for unit tests. Later, I used them in
> Snapshot.apply_filters() to factorize the code (before I add 2
> implementations to match a fi

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

[Charles-François Natali]

2013/10/19 Nick Coghlan :
>
> Speaking of which... Charles-François, would you be willing to act as
> BDFL-Delegate for this PEP? This will be a very useful new analysis tool,
> and between yourself and Victor it looks like you'll be able to come up with
> a solid API.
>
> I just suggested that approach to Guido and he also liked the idea :)

Well, I'd be happy to help get this merged.

There's still the deadline problem: do we have to get this PEP
approved and merged within 24 hours?

cf
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

[Brett Cannon]

On Sat, Oct 19, 2013 at 10:00 AM, Charles-François Natali <
cf.nat...@gmail.com> wrote:

> 2013/10/19 Nick Coghlan :
> >
> > Speaking of which... Charles-François, would you be willing to act as
> > BDFL-Delegate for this PEP? This will be a very useful new analysis tool,
> > and between yourself and Victor it looks like you'll be able to come up
> with
> > a solid API.
> >
> > I just suggested that approach to Guido and he also liked the idea :)
>
> Well, I'd be happy to help get this merged.
>
> There's still the deadline problem: do we have to get this PEP
> approved and merged within 24 hours?


No, it just has to land before b1.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

[Nick Coghlan]

On 19 October 2013 22:44, Christian Heimes  wrote:
> Am 19.10.2013 00:56, schrieb Guido van Rossum:
> A couple of months I had a long and fruitful discussion with MAL about
> the issue. Egenix PyOpenSSL installer comes with a root CA bundle. He
> tried a couple of approaches to handle trust settings with OpenSSL
> means. Eventually MAL had to split up the bundle into multiple files for
> each purpuse, see
> http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.2.1.0.1.5.html
>
> We should *really* write a PEP about it, specify all details and get a
> proper review from real experts. This stuff is super complex and highly
> fragile. :(

At the very least, it would be good if you and/or MAL could review the
cert verification in pip. PEP 453 makes that kinda important :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #18810: Be optimistic with stat calls when seeing if a directory

[Nick Coghlan]

On 19 October 2013 23:54, Brett Cannon  wrote:
>> I don't follow this logic. There's now not even an existence check for the
>> base name, so it reads to me like we will look for all the possible init
>> file extensions even if there's *no* directory with an appropriate name.
>>
>> What am I missing?
>
>
> ``if cache_module in cache:``, the line above the _path_join() call and the
> guard that blocks the entire package search.

Ah, that's what I get for reviewing diffs on my phone and hence not
looking up the full context :)

Thanks for the clarification.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

[Christian Heimes]

Am 19.10.2013 16:14, schrieb Nick Coghlan:
> At the very least, it would be good if you and/or MAL could review
> the cert verification in pip. PEP 453 makes that kinda important
> :)

Where can I find the code for PEP 453?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Antoine Pitrou]

On Sat, 19 Oct 2013 15:36:02 +0200
Christian Heimes  wrote:
> Am 19.10.2013 14:54, schrieb Antoine Pitrou:
> > On Sat, 19 Oct 2013 14:25:28 +0200 (CEST)
> > christian.heimes  wrote:
> >>  
> >> -   .. note:: A fast implementation of *pbkdf2_hmac* is only available with
> >> -  OpenSSL 1.0 and newer. The Python implementation uses an inline
> >> -  version of :mod:`hmac` and is about three times slower. Contrary to
> >> -  OpenSSL's current code the length of the password has only a minimal
> >> -  impact on the runtime of the Python implementation.
> >> +   .. note:: A fast implementation of *pbkdf2_hmac* is available with 
> >> OpenSSL.
> >> +  The Python implementation uses an inline version of :mod:`hmac`. It 
> >> is
> >> +  about three times slower and doesn't release the GIL.
> > 
> > The documentation should stop talking about the Python implementation
> > if the C implementation is always used by default.
> 
> The C implementation is not used if Python is not compiled with OpenSSL
> support.

But that's a fringe situation. Any normal build of Python should be
compiled with OpenSSL support (and any decent binary build is). I think
the mention in the docs is distracting and will create pointless
uncertainty in the reader.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

[Nick Coghlan]

On 20 October 2013 00:44, Christian Heimes  wrote:
> Am 19.10.2013 16:14, schrieb Nick Coghlan:
>> At the very least, it would be good if you and/or MAL could review
>> the cert verification in pip. PEP 453 makes that kinda important
>> :)
>
> Where can I find the code for PEP 453?

It's the cert verification in pip that's relevant - the PEP was
updated so that ensurepip itself never talks to the internet. So I
guess that would mean checking the cert verification in pip's vendored
copy of requests:
https://github.com/pypa/pip/tree/develop/pip/vendor/requests

(So I guess if you do find any issues, they would likely be applicable
to the upstream requests package as well)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC

[Nick Coghlan]

On 19 October 2013 22:12, christian.heimes  wrote:
> http://hg.python.org/cpython/rev/e73627483d2f
> changeset:   86467:e73627483d2f
> user:Christian Heimes 
> date:Sat Oct 19 14:12:02 2013 +0200
> summary:
>   Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC
>
> files:
>   Doc/library/hashlib.rst  |   6 ++-
>   Lib/hashlib.py   |  69 +--
>   Lib/test/test_hashlib.py |  20 ++-
>   Misc/NEWS|   2 +
>   4 files changed, 86 insertions(+), 11 deletions(-)
> diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py
> --- a/Lib/test/test_hashlib.py
> +++ b/Lib/test/test_hashlib.py
> @@ -18,11 +18,13 @@
>  import unittest
>  import warnings
>  from test import support
> -from test.support import _4G, bigmemtest
> +from test.support import _4G, bigmemtest, import_fresh_module
>
>  # Were we compiled --with-pydebug or with #define Py_DEBUG?
>  COMPILED_WITH_PYDEBUG = hasattr(sys, 'gettotalrefcount')
>
> +c_hashlib = import_fresh_module('hashlib', fresh=['_hashlib'])

Looks like this import is failing on at least some platforms:

http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/6535/steps/test/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/204/steps/test/logs/stdio

Due to the build failing:
http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/6535/steps/compile/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/204/steps/compile/logs/stdio

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] pip SSL

[Christian Heimes]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Am 19.10.2013 16:59, schrieb Nick Coghlan:
> It's the cert verification in pip that's relevant - the PEP was 
> updated so that ensurepip itself never talks to the internet. So I 
> guess that would mean checking the cert verification in pip's
> vendored copy of requests: 
> https://github.com/pypa/pip/tree/develop/pip/vendor/requests
> 
> (So I guess if you do find any issues, they would likely be
> applicable to the upstream requests package as well)

Oh heck, where should I start?

The cacert.pem file is outdated. Also it's unclear who has generated
the file and how it was generated from certdata.txt. It may very well
contain revoked certificates, too. You can find the latest version of
the file at


http://hg.mozilla.org/mozilla-central/file/tip/security/nss/lib/ckfw/builtins/certdata.txt

. A proper tool is required to generate a correct PEM file. It must
understand *ALL* fields. I have some code for that but it's not ready yet.


pip uses requests and requests rolls its own code for or on top of
Python stdlib modules, e.g. urllib3 with ssl_match_hostname. The
method has the same security flaw as Python's ssl.match_hostname()
function for IDNs. I'm a bit worried that we have to review and
validate all 3rd party packages and copies of stdlib modules for issues.


The assert_fingerprint() function looks kinda funny. It uses sha1() or
md5() on the DER representation of the cert. It's not how you are
suppose to take fingerprints for cert pinning. But Python's ssl has no
way to get the SPKI from the cert yet. I'm working on that as well.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCgAGBQJSYqrBAAoJEMeIxMHUVQ1FeSAP/3C4g3Sp6Fg976C5eihDpuLo
VKB83nf708iwR990lJ6AYAiHDRjwVk6ssgYX4EfA3qQqjiAOykIQKZYcYrBA36lT
FDn3gIXkh6x1QnwEOopGqrdbhSbDqPB57zRAZrmzJp8JTvOx4FYVgmx6bi2yumst
w2m+ovWjxzUOlr1V7LM2/vzxSJXLyg+Espps3kDgX96qZvHXCfn/M39Y5R39on7v
Er3qmD5aHEOnVnA1cH/OC7O8uJm8dPrm7wocztErZWyy006chW2B8edvFpjW8iEn
StYxNw7Ko6jr2ncCwAKntVavGRtbHJowaF4l4yTCZ6suCx+LAzy7O+X90Ic1LknN
o/RLSfJeyhUOHpADwloKfjRuPk2twq46z96GauoFBThaBCca7mRS29EudWG54Dn1
tT1/7+b3FfiU1GmWqzTpxgrJiRREk+XTEVCmhq2XUdBnGQI7G6RT9BefVfYzep06
Z0hKWdIR2moC21iPcBMIOnXqscaMHjvcVOnScv05UiE5et0fB8lAfoZJ9u1G5UC4
vkifZpfOfCDMh3HXSCiRp2TEUo/GPy35P/8Tk1O602nGj3oRxPJ1fdOlIexu+9bV
S/kGwMjhyLQHDp0786AwDnv/NNOK6hJHCiZLLqX6F0+K4RdlRRd/6lVvyxKGT8ca
OXxoornL8iyvEnyti7cq
=BTNE
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #19254: Provide an optimized Python implementation of PBKDF2_HMAC

[Christian Heimes]

Am 19.10.2013 17:15, schrieb Nick Coghlan:
>> +c_hashlib = import_fresh_module('hashlib', fresh=['_hashlib'])
> 
> Looks like this import is failing on at least some platforms:
> 
> http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/6535/steps/test/logs/stdio
> http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/204/steps/test/logs/stdio
> 
> Due to the build failing:
> http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/6535/steps/compile/logs/stdio
> http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/204/steps/compile/logs/stdio

Thanks Nick!

Actually it was http://hg.python.org/cpython/rev/0a26ef834a49. The fix
in http://hg.python.org/cpython/rev/88fac1574049 should have taken care
of the issue.

Sorry for the noise, I don't have OpenSSL 0.9.8 on this machine.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Christian Heimes]

Am 19.10.2013 16:59, schrieb Antoine Pitrou:
> But that's a fringe situation. Any normal build of Python should be
> compiled with OpenSSL support (and any decent binary build is). I think
> the mention in the docs is distracting and will create pointless
> uncertainty in the reader.

HMAC_CTX_copy() is not available in OpenSSL < 1.0. My optimized code
won't run on OpenSSL 0.9.8. IMHO it's important to understand that some
code may complete in ~100ms instead of ~30ms.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

[Guido van Rossum]

You should ask Glyph too. He supplied lots of useful info about cert
checking on the python-tulip list.


On Sat, Oct 19, 2013 at 7:14 AM, Nick Coghlan  wrote:

> On 19 October 2013 22:44, Christian Heimes  wrote:
> > Am 19.10.2013 00:56, schrieb Guido van Rossum:
> > A couple of months I had a long and fruitful discussion with MAL about
> > the issue. Egenix PyOpenSSL installer comes with a root CA bundle. He
> > tried a couple of approaches to handle trust settings with OpenSSL
> > means. Eventually MAL had to split up the bundle into multiple files for
> > each purpuse, see
> >
> http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.2.1.0.1.5.html
> >
> > We should *really* write a PEP about it, specify all details and get a
> > proper review from real experts. This stuff is super complex and highly
> > fragile. :(
>
> At the very least, it would be good if you and/or MAL could review the
> cert verification in pip. PEP 453 makes that kinda important :)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Antoine Pitrou]

On Sat, 19 Oct 2013 18:05:35 +0200
Christian Heimes  wrote:
> Am 19.10.2013 16:59, schrieb Antoine Pitrou:
> > But that's a fringe situation. Any normal build of Python should be
> > compiled with OpenSSL support (and any decent binary build is). I think
> > the mention in the docs is distracting and will create pointless
> > uncertainty in the reader.
> 
> HMAC_CTX_copy() is not available in OpenSSL < 1.0. My optimized code
> won't run on OpenSSL 0.9.8. IMHO it's important to understand that some
> code may complete in ~100ms instead of ~30ms.

Well, I disagree about the importance of documenting a 3x difference on
a select function (rather than, say, a couple orders of magnitude), but
the OpenSSL version issue at least makes the situation more
plausible :-)

(even though OpenSSL 1.0 is already 3.5 years old)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pip SSL

[Donald Stufft]

One of the reasons we switched to using requests was to help centralize the SSL
handling code over to requests. So any issues could be fixed there and we just
pull in a newer version of requests.

On Oct 19, 2013, at 11:52 AM, Christian Heimes  wrote:

> Signed PGP part
> Am 19.10.2013 16:59, schrieb Nick Coghlan:
> > It's the cert verification in pip that's relevant - the PEP was 
> > updated so that ensurepip itself never talks to the internet. So I 
> > guess that would mean checking the cert verification in pip's
> > vendored copy of requests: 
> > https://github.com/pypa/pip/tree/develop/pip/vendor/requests
> > 
> > (So I guess if you do find any issues, they would likely be
> > applicable to the upstream requests package as well)
> 
> Oh heck, where should I start?
> 
> The cacert.pem file is outdated. Also it's unclear who has generated
> the file and how it was generated from certdata.txt. It may very well
> contain revoked certificates, too. You can find the latest version of
> the file at
> 
> 
> http://hg.mozilla.org/mozilla-central/file/tip/security/nss/lib/ckfw/builtins/certdata.txt
> 
> . A proper tool is required to generate a correct PEM file. It must
> understand *ALL* fields. I have some code for that but it's not ready yet.
> 
> 
> pip uses requests and requests rolls its own code for or on top of
> Python stdlib modules, e.g. urllib3 with ssl_match_hostname. The
> method has the same security flaw as Python's ssl.match_hostname()
> function for IDNs. I'm a bit worried that we have to review and
> validate all 3rd party packages and copies of stdlib modules for issues.
> 
> 
> The assert_fingerprint() function looks kinda funny. It uses sha1() or
> md5() on the DER representation of the cert. It's not how you are
> suppose to take fingerprints for cert pinning. But Python's ssl has no
> way to get the SPKI from the cert yet. I'm working on that as well.
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Victor Stinner]

Does Python officially support opsenssl < 1.0? Which OS uses such old
version?

On Windows, Python embeds its own copy of openssl for example.

Victor
Le 19 oct. 2013 18:07, "Christian Heimes"  a écrit :

> Am 19.10.2013 16:59, schrieb Antoine Pitrou:
> > But that's a fringe situation. Any normal build of Python should be
> > compiled with OpenSSL support (and any decent binary build is). I think
> > the mention in the docs is distracting and will create pointless
> > uncertainty in the reader.
>
> HMAC_CTX_copy() is not available in OpenSSL < 1.0. My optimized code
> won't run on OpenSSL 0.9.8. IMHO it's important to understand that some
> code may complete in ~100ms instead of ~30ms.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Christian Heimes]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Am 19.10.2013 20:04, schrieb Victor Stinner:
> Does Python officially support opsenssl < 1.0? Which OS uses such
> old version?
> 
> On Windows, Python embeds its own copy of openssl for example.

Mac OS X has only OpenSSL 0.9.8 and will not receive any updates to
1.0 from Apple.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCgAGBQJSYsoLAAoJEMeIxMHUVQ1FHW4P/RFILORWT9hrPQS5rC1W/ioc
JxOhAeuoWTy4cSvwOvvl9B29e7GxUVoGa029Ub6FXsptcs70odrdiJnvYG1J43z/
xZUBgLiog22wmqA35cmM46pChq8X0ERvR6Nfvuc10dQo+ePjujQfa8qiSP35kmvS
fI1YxoBxR1diSDDilbS28NrQOLevV7lQR8k6bqqvSb9JPym9OcSrl/UFa57mOIvD
kj2ClzylzPGpIWfWiNu/Nte6XNxASe69LfYCHmb12L93Wtf+s/0ZN0uALdBMfSmB
RUIAoeP6dTJm0H9AvuEHU4JFURU2Q12x15dmloBw+W3Tj3KpRV7eSCBgQOpL1UMb
u2s57JOO1Uv0Nd4OIiOBGCOO5zFbzThKYxiTrXG6HAhd2cxrNNfHKWy9wJdaFfnt
/m9rFg2vrCNWu3B0YQIuuBmIVPAy4gw+Vh+pIFLdC4vx14LGNE+E5IG9/48mBQ/7
9XDjHLSpq0EeAv6JYOm257JySQTe/BFxbwXEECM8x8rYk0dapy6DEK6kfPDmOc0M
bagHWPuBr2OT+M/MPac+vOs3jzSWaKMynOTEvm5cKLmPI/I/oupLZ8kh48RUFmaH
IN0DJXD28p2JG/NykqnQ/z25STAzQOXHcXiFXkaCgnA6LmXwPtkmCAJeqzOc8M/x
4R1akowEUir6Zw4SUjGR
=B/+G
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #18582: provide a faster C implementation of pbkdf2_hmac that works with

[Antoine Pitrou]

On Sat, 19 Oct 2013 20:04:05 +0200
Victor Stinner  wrote:
> Does Python officially support opsenssl < 1.0? Which OS uses such old
> version?

We try to support any reasonably modern POSIX-compliant system, e.g.
less than 10 years old.
Of course, we only "officially support" those platforms which have a
stable, green buildbot.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] I *might* push Python 3.4.0 alpha 4 back a week

[Larry Hastings]

A lot has landed in trunk in the last day or two: Tulip, Argument 
Clinic, and statistics just landed too.  The buildbots are upset at us 
humans--there's a lot of red.  See for yourself:


   http://buildbot.python.org/all/waterfall?category=3.x.stable

I'd like to tag Alpha 4 late tonight, but if the buildbots are still 
mostly-red it's a bad idea.  If you can help make the buildbots happy, 
and have some time in the next eight or ten hours, please hop in to 
#python-dev and ask what you can do to help.


SOS -- Save Our Schedule,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pip SSL

[Ian Cordasco]

Also the three of us maintaining requests and the author of urllib3
are all very conscious that the packaged pem file is outdated. We have
an open issue about how to rebuild it accurately while taking into
consideration (and not including) the ones that have been revoked. Any
suggestions you have can be sent to me off list or reported on the
issue tracker.

On Sat, Oct 19, 2013 at 12:57 PM, Donald Stufft  wrote:
> One of the reasons we switched to using requests was to help centralize the 
> SSL
> handling code over to requests. So any issues could be fixed there and we just
> pull in a newer version of requests.
>
> On Oct 19, 2013, at 11:52 AM, Christian Heimes  wrote:
>
>> Signed PGP part
>> Am 19.10.2013 16:59, schrieb Nick Coghlan:
>> > It's the cert verification in pip that's relevant - the PEP was
>> > updated so that ensurepip itself never talks to the internet. So I
>> > guess that would mean checking the cert verification in pip's
>> > vendored copy of requests:
>> > https://github.com/pypa/pip/tree/develop/pip/vendor/requests
>> >
>> > (So I guess if you do find any issues, they would likely be
>> > applicable to the upstream requests package as well)
>>
>> Oh heck, where should I start?
>>
>> The cacert.pem file is outdated. Also it's unclear who has generated
>> the file and how it was generated from certdata.txt. It may very well
>> contain revoked certificates, too. You can find the latest version of
>> the file at
>>
>>
>> http://hg.mozilla.org/mozilla-central/file/tip/security/nss/lib/ckfw/builtins/certdata.txt
>>
>> . A proper tool is required to generate a correct PEM file. It must
>> understand *ALL* fields. I have some code for that but it's not ready yet.
>>
>>
>> pip uses requests and requests rolls its own code for or on top of
>> Python stdlib modules, e.g. urllib3 with ssl_match_hostname. The
>> method has the same security flaw as Python's ssl.match_hostname()
>> function for IDNs. I'm a bit worried that we have to review and
>> validate all 3rd party packages and copies of stdlib modules for issues.
>>
>>
>> The assert_fingerprint() function looks kinda funny. It uses sha1() or
>> md5() on the DER representation of the cert. It's not how you are
>> suppose to take fingerprints for cert pinning. But Python's ssl has no
>> way to get the SPKI from the cert yet. I'm working on that as well.
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
>
>
> -
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pip SSL

[Glenn Linderman]

On 10/19/2013 12:46 PM, Ian Cordasco wrote:

Also the three of us maintaining requests and the author of urllib3
are all very conscious that the packaged pem file is outdated. We have
an open issue about how to rebuild it accurately while taking into
consideration (and not including) the ones that have been revoked. Any
suggestions you have can be sent to me off list or reported on the
issue tracker.
Is this another issue like the time zone database? Something that needs 
to be packaged with some versions of Python, but that needs a mechanism 
to update it later for accuracy (which, in this case, also implies 
security)?


Could a similar mechanism be used for both?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] I *might* push Python 3.4.0 alpha 4 back a week

[Victor Stinner]

2013/10/19 Larry Hastings :
> A lot has landed in trunk in the last day or two: Tulip, Argument Clinic,
> and statistics just landed too.

Wow, Python 3.4 looks great! I was waiting for statistics.

Don't forget to update the What's New in Python 3.4 document.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

[Guido van Rossum]

Let it be known to all that Charles-François Natali is hereby appointed
BDFL for this PEP.


On Sat, Oct 19, 2013 at 7:00 AM, Charles-François Natali <
cf.nat...@gmail.com> wrote:

> 2013/10/19 Nick Coghlan :
> >
> > Speaking of which... Charles-François, would you be willing to act as
> > BDFL-Delegate for this PEP? This will be a very useful new analysis tool,
> > and between yourself and Victor it looks like you'll be able to come up
> with
> > a solid API.
> >
> > I just suggested that approach to Guido and he also liked the idea :)
>
> Well, I'd be happy to help get this merged.
>
> There's still the deadline problem: do we have to get this PEP
> approved and merged within 24 hours?
>
> cf
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pip SSL

[Nick Coghlan]

On 20 Oct 2013 06:14, "Glenn Linderman"  wrote:
>
> On 10/19/2013 12:46 PM, Ian Cordasco wrote:
>>
>> Also the three of us maintaining requests and the author of urllib3
>> are all very conscious that the packaged pem file is outdated. We have
>> an open issue about how to rebuild it accurately while taking into
>> consideration (and not including) the ones that have been revoked. Any
>> suggestions you have can be sent to me off list or reported on the
>> issue tracker.
>
> Is this another issue like the time zone database? Something that needs
to be packaged with some versions of Python, but that needs a mechanism to
update it later for accuracy (which, in this case, also implies security)?
>
> Could a similar mechanism be used for both?

Once pip is installed, then "pip install --upgrade pip" will update it.
This request was about getting the *current* state reviewed prior to the
pip 1.5 release, since 1.5 is the version likely to be provided by
"ensurepip" in CPython 3.4.

As Donald noted the fact pip uses requests internally is actually a benefit
for the broader Python ecosystem, since it means fixing the cert management
and verification for pip (by fixing requests and updating the bundled
version) will fix them for a lot of other projects as well.

Cheers,
Nick.

>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pip SSL

[Nick Coghlan]

On 20 October 2013 05:46, Ian Cordasco  wrote:
> Also the three of us maintaining requests and the author of urllib3
> are all very conscious that the packaged pem file is outdated. We have
> an open issue about how to rebuild it accurately while taking into
> consideration (and not including) the ones that have been revoked. Any
> suggestions you have can be sent to me off list or reported on the
> issue tracker.

The requests issue Ian is referring to:
https://github.com/kennethreitz/requests/issues/1659

The next version of PEP 453 will include getting this resolved as part
of the integration timeline:


* by December 29th (1 week prior to the scheduled date of 3.4.0 beta 2)

  ``requests`` certificate management issue resolved
  ``ensurepip`` updated to the final release of pip 1.5, or a subsequent
  maintenance release (including a suitably updated vendored copy of
  ``requests``)


And also mentions it under the "security considerations" section for
the bootstrapping mechanism:


Only users that choose to use ``pip`` to communicate with PyPI will
need to pay attention to the additional security considerations that come
with doing so.

However, the core CPython team will also assist with reviewing and
resolving the `certificate update management issue
`__ currently
affecting the ``requests`` project (and hence ``pip``).


Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] An interesting exception handling quirk

[Armin Rigo]

Hi Nick,

On Sat, Oct 19, 2013 at 1:41 PM, Nick Coghlan  wrote:
> recreating the *exact* exception subclass check from
> Python is actually difficult these days.

Can't it be done roughly like that?

   def __exit__(self, typ, val, tb):
try:
raise typ, val
except self.exceptions:
return True
except:
return False


A bientôt,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com