Re: [Python-Dev] Show DeprecationWarning in debug mode?

[Serhiy Storchaka]

18.11.17 03:22, Victor Stinner пише:

I noticed that Python not only hides DeprecationWarning, but also
PendingDeprecationWarning and ImportWarning by default. While I
understand why we decided to hide these warnings to users for a Python
compiled in release mode, why are they hidden in Python debug builds?

I'm asking the question because in debug mode, Python shows
ResourceWarning warnings (whereas these warnings are hidden in release
mode). Why only displaying ResourceWarning, but not other warnings in
debug mode?


+1 for showing all warning (except maybe PendingDeprecationWarning) in 
the debug build! I constantly forgot about this.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Make the stable API-ABI usable

[Serhiy Storchaka]

18.11.17 03:05, Victor Stinner пише:

tl; dr I propose to extend the existing "stable API" to make it almost
as complete as the current API. For example, add back
PyTuple_GET_ITEM() to be stable API, but it becomes a function call
rather than a macro. The final question is if it's not too late to
iterate on an implementation of this idea for Python 3.7? Knowing that
the stable API doesn't affect the "current API" at all, since the "new
C API" (extended stable API) would only be accessible using an
*opt-in* flag.


There is the PyTuple_GetItem() function. The benefit of using 
PyTuple_GET_ITEM() in tight loops: a) avoid redundant arguments checks; 
b) avoid calling function overhead. Making PyTuple_GET_ITEM() a function 
will destroy the half of the benefit. And this will make the ABI larger.


First at all we need to document all API, what is stable, and in what 
version it had became stable.


Then I would separate three kinds of API physically: limited API, 
extended unstable API, and internal private API, and place their 
declarations in different headers. The headers with internal API should 
not even be visible for third-party developers.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

[Serhiy Storchaka]

18.11.17 02:01, Victor Stinner пише:

Many global variables used by the "Python runtime" were move to a new
single "_PyRuntime" variable (big structure made of sub-structures).
See Include/internal/pystate.h.

A side effect of moving variables from random files into header files
is that it's not more possible to fully initialize _PyRuntime at
"compilation time". For example, previously, it was possible to refer
to local C function (functions declared with "static", so only visible
in the current file). Now a new "initialization function" is required
to must be called.

In short, it means that using the "Python runtime" before it's
initialized by _PyRuntime_Initialize() is now likely to crash. For
example, calling PyMem_RawMalloc(), before calling
_PyRuntime_Initialize(), now calls the function NULL: dereference a
NULL pointer, and so immediately crash with a segmentation fault.


Wouldn't be better to revert (the part of) global variables moving?
I still don't see a benefit of it.


To give a more concrete example: Py_DecodeLocale() is the recommanded
function to decode bytes from the operating system, but this function
calls PyMem_RawMalloc() which does crash before
_PyRuntime_Initialize() is called. Is Py_DecodeLocale() used to
initialize Python?

For example, "void Py_SetProgramName(wchar_t *);" expects a text
string, whereas main() gives argv as bytes. Calling
Py_SetProgramName() from argv requires to decode bytes... So use
Py_DecodeLocale()...

Should we do something in Py_DecodeLocale()? Maybe crash if
_PyRuntime_Initialize() wasn't called yet?


I think Py_DecodeLocale() should be usable before calling 
Py_Initialize(). In the example in Doc/extending/extending.rst it is 
used before Py_Initialize(). If the third-party code is based on this 
example, it will crash now.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Show DeprecationWarning in debug mode?

[Victor Stinner]

Hi,

I noticed that Python not only hides DeprecationWarning, but also
PendingDeprecationWarning and ImportWarning by default. While I
understand why we decided to hide these warnings to users for a Python
compiled in release mode, why are they hidden in Python debug builds?

I'm asking the question because in debug mode, Python shows
ResourceWarning warnings (whereas these warnings are hidden in release
mode). Why only displaying ResourceWarning, but not other warnings in
debug mode?

At least, with the Python 3.7 new "developer mode" (-X dev), now you
can be totally annoyed^W^W appreciate *all* these warnings :-)

Example:
--
$ cat x.py
import warnings
warnings.warn('Resource warning', ResourceWarning)
warnings.warn('Deprecation warning', DeprecationWarning)

# Release build: ignore all :-(

$ python3 x.py

# Debug build: ignore deprecation :-|

$ ./python x.py
x.py:2: ResourceWarning: Resource warning
  warnings.warn('Resource warning', ResourceWarning)

# Developer mode: show all :-)

$ ./python -X dev x.py
x.py:2: ResourceWarning: Resource warning
  warnings.warn('Resource warning', ResourceWarning)
x.py:3: DeprecationWarning: Deprecation warning
  warnings.warn('Deprecation warning', DeprecationWarning)
--

Or maybe we should start adding new modes like -X
all-warnings-except-PendingDeprecationWarning, -X
I-really-really-love-warnings and -X warnings-hater, as Barry
proposed?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Make the stable API-ABI usable

[Victor Stinner]

Hi,

tl; dr I propose to extend the existing "stable API" to make it almost
as complete as the current API. For example, add back
PyTuple_GET_ITEM() to be stable API, but it becomes a function call
rather than a macro. The final question is if it's not too late to
iterate on an implementation of this idea for Python 3.7? Knowing that
the stable API doesn't affect the "current API" at all, since the "new
C API" (extended stable API) would only be accessible using an
*opt-in* flag.


Since I failed to find time to write a proper PEP (sorry about that),
and the incoming deadline for Python 3.7 features is getting closer, I
decided to dump my dump into this email. Sorry about the raw
formatting.

It's a third turn of feedback. The first one was done in python-ideas
last July, the second was at the CPython sprint last September at
Instagram. Both were positive.

   https://mail.python.org/pipermail/python-ideas/2017-July/046399.html

The C API of CPython is amazing. It made CPython as powerful as we
know it today. Without the C API, Python wouldn't have numpy. Without
numpy, Python wouldn't be as popular as it is nowadays. Ok, enough for
the introduction.

The C API is awful. I hate it so much! The C API is old, error prone,
too big, and expose every single CPython implementation detail... The
C API is likely the first reason why faster implementation of Python
(PyPy?) are not as popular as they should be.

The fact that the C API is so widely used prevents many evolutions of
CPython. CPython is stuck by its own (public) C API.

PyPy developers spent a lot of time to make cffi great and advertize
it all around the world. While it's a great project, the fact is that
the C API *is* still used and the motivation to rewrite old code
written with the C API to cffi, Cython or whateer else is too low.
Maybe because the carrot is not big enough. Let's make a bigger
carrot! (bigger than PyPy amazing performances? I'm not sure that it's
doable :-( we will see)

Using Cython, it's simpler to write C extensions and Cython code is
more "portable" on different Python versions, since the C API evolved
the last 10 years. For example, Python 3 renamed PyString with PyBytes
and dropped the PyInt type. Writing a C extension working on Python 2
and Python 3 requires to "pollute" the code with many #ifdef. Cython
helps to reduce them (or even avoid them? sorry, I don't know well
Cython).

The C *API* is tidely linked to the *ABI*. I tried to explain it with
an example in my article:

   https://vstinner.github.io/new-python-c-api.html

A known ABI issue is that it's not possible to load C extensions
compiled in "release mode" on a Python compiled in "debug mode". The
debug mode changes the API in a subtle way which changes the ABI and
so makes C extensions incompatible. This issue prevents many people to
use a Python debug build, whereas debug builds are very helpful to
detect bugs earlier.

Another issue is that C extensions must be recompiled for each Python
release (3.5, 3.6, 3.7, etc.). Linux vendors like Red Hat cannot
provide a single binary for multiple Python versions which prevents to
upgade the "system" Python in the lifecycle of a distribution major
version (or at least, it makes things much more complicated in term of
packaging, it multiply packages for each Python version...).

.
.
.

Don't worry (be happy!), I'm not writing this email to complain, but
to propose a solution. Aha!

I propose to modify the API step by step to add more functions to the
"stable ABI" (PEP 384) to be able to compile more and more C
extensions using the "stable API" (please try to follow, first it was
B, now it's P... seriously, the difference between the ABI and the API
is subtle, to simplify let's say that it's the same thing, ok? :-)).

I wrote a draft PEP, but I never found time to update it after the two
rounds of feedbacks (python-ideas and the sprint) to write a proper
PEP. So I will only give a link to my draft, sorry!

  https://github.com/vstinner/misc/blob/master/python/pep_c_api.rst

In short, the plan is to add back the PyTuple_GET_ITEM() *API* to the
"stable API" but change its implementation to a *function call* rather
than the existing macro, so the compiled C extension will use a
function call and so don't rely on the ABI anymore.


My plan is to have two main milestones:

(1) Python 3.7: Extend the *existing* opt-in "stable API" which
requires to compile C extensions in a special mode. Add maybe an
option in distutils to ease the compilation of a C extension with the
"stable API"?

(2) In Python 3.8, --if the project is successful and the performance
overhead is acceptable compared the advantages of having C extensions
working on multiple Python verisons--, make the "stable API (without
implementation details)" the default, but add a new opt-in option to
give access to the "full API (with implementation details)" for
debuggers and other people who understand what they do (like Cython?).

Note: currently, the "stable API" is 

Re: [Python-Dev] Python initialization and embedded Python

[Steve Dower]

On 17Nov2017 1601, Victor Stinner wrote:

In short, it means that using the "Python runtime" before it's
initialized by _PyRuntime_Initialize() is now likely to crash. For
example, calling PyMem_RawMalloc(), before calling
_PyRuntime_Initialize(), now calls the function NULL: dereference a
NULL pointer, and so immediately crash with a segmentation fault.

I'm writing this email to ask if this change is an issue or not to
embedded Python and the Python C API. Is it still possible to call
"all" functions of the C API before calling Py_Initialize()?


I thought it was never possible to call most of the C API without 
initializing, except for certain APIs that are documented as being safe. 
I've certainly crashed many times calling C APIs before initialization. 
My intuition was that the only safe ones before were those that were 
used to initialize the runtime (Py_SetPath and such), which are also the 
ones being "upgraded" as part of this work.


If we have a good idea of which ones are [un]safe now, perhaps we should 
tag them explicitly in the docs? Do we know which ones are [un]safe?


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python initialization and embedded Python

[Victor Stinner]

Hi,

The CPython internals evolved during Python 3.7 cycle. I would like to
know if we broke the C API or not.

Nick Coghlan and Eric Snow are working on cleaning up the Python
initialization with the "on going" PEP 432:
https://www.python.org/dev/peps/pep-0432/

Many global variables used by the "Python runtime" were move to a new
single "_PyRuntime" variable (big structure made of sub-structures).
See Include/internal/pystate.h.

A side effect of moving variables from random files into header files
is that it's not more possible to fully initialize _PyRuntime at
"compilation time". For example, previously, it was possible to refer
to local C function (functions declared with "static", so only visible
in the current file). Now a new "initialization function" is required
to must be called.

In short, it means that using the "Python runtime" before it's
initialized by _PyRuntime_Initialize() is now likely to crash. For
example, calling PyMem_RawMalloc(), before calling
_PyRuntime_Initialize(), now calls the function NULL: dereference a
NULL pointer, and so immediately crash with a segmentation fault.

I'm writing this email to ask if this change is an issue or not to
embedded Python and the Python C API. Is it still possible to call
"all" functions of the C API before calling Py_Initialize()?

I was bitten by the bug while reworking the Py_Main() function to
split it into subfunctions and cleanup the code to handle the command
line arguments and environment variables. I fixed the issue in main()
by calling _PyRuntime_Initialize() as soon as possible: it's now the
first instruction of main() :-) (See Programs/python.c)

To give a more concrete example: Py_DecodeLocale() is the recommanded
function to decode bytes from the operating system, but this function
calls PyMem_RawMalloc() which does crash before
_PyRuntime_Initialize() is called. Is Py_DecodeLocale() used to
initialize Python?

For example, "void Py_SetProgramName(wchar_t *);" expects a text
string, whereas main() gives argv as bytes. Calling
Py_SetProgramName() from argv requires to decode bytes... So use
Py_DecodeLocale()...

Should we do something in Py_DecodeLocale()? Maybe crash if
_PyRuntime_Initialize() wasn't called yet?

Maybe, the minimum change is to expose _PyRuntime_Initialize() in the
public C API?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Clarify the compatibility policy of lib2to3.

[Guido van Rossum]

On Fri, Nov 17, 2017 at 8:56 AM, Dong-hee Na  wrote:

> Few days ago, I submitted a patch(https://github.com/
> python/cpython/pull/4417) which updates2to3 converts
> `operator.isCallable(obj)` to `isinstance(obj, collections.abc.Callable)`.
>
> This was Serhiy Storchaka’s idea(https://bugs.python.org/issue32046) and
> I agree with his idea since `callable` is not available in all 3.x versions.
>
> However, some people would like to clarify the policy of 2to3.
> That means that 2to3 is compatible with the branch that is currently
> maintained? or it is converted to specific Python 3.X codes with the
> specific version of 2to3?
>

Ideally it would generate code that works under all *still supported*
versions of Python, not just under the version that happens to be used to
run the converter. In the context of that specific bug, callable() appeared
in Python 3.2, but the oldest Python 3 version that's still supported is
3.4, so we're safe generating callable().

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python possible vulnerabilities in concurrency

[Steve Dower]

On 15Nov2017 2053, Guido van Rossum wrote:
On Wed, Nov 15, 2017 at 6:50 PM, Guido van Rossum > wrote:


So far I learned one thing from the report. They use the term 
"vulnerabilities" liberally, defining it essentially as "bug":


All programming languages contain constructs that are incompletely
specified, exhibit undefined behaviour, are
implementation-dependent, or are difficult to use correctly. The use
of those constructs may therefore give rise to /vulnerabilities/, as
a result of which, software programs can execute differently than
intended by the writer.


They then go on to explain that sometimes vulnerabilities can be 
exploited, but I object to calling all bugs vulnerabilities -- that's 
just using a scary word to get attention for a sleep-inducing document 
containing such gems as "Use floating-point arithmetic only when 
absolutely needed" (page 230).


I looked at this report the first time it was posted and came to the 
same conclusion.


It's only valuable in the sense that it makes clear just how perfect 
your code has to be to avoid being vulnerable, and since that level of 
perfection can never be achieved, the takeaway is that you can't achieve 
security solely within the application/framework/runtime. It is 
convenient to have formally researched and collated it, so the rest of 
us can just write blog posts/PEPs stating it as fact, but I think most 
people will intuitively get the main point without referring to the report.


(Yes, I'm still interested in pushing PEP 551 forward :) I've been 
trying to get some actual companies other than Microsoft using it for 
the real-world experience, and I have a couple of conference talks 
coming up about it. There are implementations against v3.7.0a2 is at 
https://github.com/zooba/cpython/tree/pep551 and against v3.6.3 at 
https://github.com/zooba/cpython/tree/pep551_36 )


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2017-11-10 - 2017-11-17)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6261 (+14)
  closed 37553 (+46)
  total  43814 (+60)

Open issues with patches: 2409 


Issues opened (39)
==

#27494: 2to3 parser failure caused by a comma after a generator expres
https://bugs.python.org/issue27494  reopened by serhiy.storchaka

#31963: AMD64 Debian PGO 3.x buildbot: compilation failed with an inte
https://bugs.python.org/issue31963  reopened by vstinner

#32004: Allow specifying code packing order in audioop adpcm functions
https://bugs.python.org/issue32004  opened by MosesofEgypt

#32005: multiprocessing.Array misleading error message in slice assign
https://bugs.python.org/issue32005  opened by steven.daprano

#32006: multiprocessing.Array 'c' code is not documented
https://bugs.python.org/issue32006  opened by steven.daprano

#32007: nis module fails to build against glibc-2.26
https://bugs.python.org/issue32007  opened by floppymaster

#32008: Example suggest to use a TLSv1 socket
https://bugs.python.org/issue32008  opened by kroeckx

#32014: multiprocessing Server's shutdown method useless send message 
https://bugs.python.org/issue32014  opened by stevezh

#32016: Python 3.6.3 venv FAILURE
https://bugs.python.org/issue32016  opened by nihon2000

#32017: profile.Profile() has no method enable()
https://bugs.python.org/issue32017  opened by pitrou

#32019: Interactive shell doesn't work with readline bracketed paste
https://bugs.python.org/issue32019  opened by Aaron.Meurer

#32021: Brotli encoding is not recognized by mimetypes
https://bugs.python.org/issue32021  opened by Andrey

#32022: Python problem - == RESTART: Shell =
https://bugs.python.org/issue32022  opened by shamon51

#32024: Nominal decorator function call syntax is inconsistent with re
https://bugs.python.org/issue32024  opened by ncoghlan

#32025: Add time.thread_time()
https://bugs.python.org/issue32025  opened by pitrou

#32026: Memory leaks in Python on Windows
https://bugs.python.org/issue32026  opened by pjna

#32028: Syntactically wrong suggestions by the new custom print statem
https://bugs.python.org/issue32028  opened by mdraw

#32030: PEP 432: Rewrite Py_Main()
https://bugs.python.org/issue32030  opened by vstinner

#32031: Do not use the canonical path in pydoc test_mixed_case_module_
https://bugs.python.org/issue32031  opened by xdegaye

#32033: The pwd module implementation incorrectly sets some attributes
https://bugs.python.org/issue32033  opened by xdegaye

#32035: Documentation of zipfile.ZipFile().writestr() fails to mention
https://bugs.python.org/issue32035  opened by Daniel5148

#32038: Add API to intercept socket.close()
https://bugs.python.org/issue32038  opened by yselivanov

#32039: timeit documentation should describe caveats
https://bugs.python.org/issue32039  opened by barry

#32041: Cannot cast '\0' to c_void_p
https://bugs.python.org/issue32041  opened by Ilya.Kulakov

#32042: Option for comparing values instead of reprs in doctest
https://bugs.python.org/issue32042  opened by Tomáš Petříček

#32043: Add a new -X dev option: "developer mode"
https://bugs.python.org/issue32043  opened by vstinner

#32045: Does json.dumps have a memory leak?
https://bugs.python.org/issue32045  opened by rohandsa

#32046: 2to3 fix for operator.isCallable()
https://bugs.python.org/issue32046  opened by serhiy.storchaka

#32047: asyncio: enable debug mode when -X dev is used
https://bugs.python.org/issue32047  opened by vstinner

#32049: 2.7.14 does not uninstall cleanly if installation was run as S
https://bugs.python.org/issue32049  opened by niemalsnever

#32050: Deprecated python3 -x option
https://bugs.python.org/issue32050  opened by vstinner

#32051: Possible issue in multiprocessing doc
https://bugs.python.org/issue32051  opened by 1a1a11a

#32052: Provide access to buffer of asyncio.StreamReader
https://bugs.python.org/issue32052  opened by Bruce Merry

#32054: Creating RPM on Python 2 works, but Python 3 fails because of 
https://bugs.python.org/issue32054  opened by pgacv2

#32055: Reconsider comparison chaining for containment tests
https://bugs.python.org/issue32055  opened by ncoghlan

#32056: bug in Lib/wave.py
https://bugs.python.org/issue32056  opened by BT123

#32059: detect_modules() in setup.py must also search the sysroot path
https://bugs.python.org/issue32059  opened by xdegaye

#32060: Should an ABC fail if no abstract methods are defined?
https://bugs.python.org/issue32060  opened by Alex Corcoles

#32063: test_multiprocessing_forkserver failed with OSError: [Errno 48
https://bugs.python.org/issue32063  opened by vstinner



Most recent 15 issues with no replies (15)
==

#32063: test_multiprocessing_forkserver failed with OSError: [Errno 48
https://bugs.python.org/issue32063

#32056: bug in 

[Python-Dev] Clarify the compatibility policy of lib2to3.

[Dong-hee Na]

Hi, 

Few days ago, I submitted a patch(https://github.com/python/cpython/pull/4417) 
which updates2to3 converts `operator.isCallable(obj)` to `isinstance(obj, 
collections.abc.Callable)`.

This was Serhiy Storchaka’s idea(https://bugs.python.org/issue32046) and I 
agree with his idea since `callable` is not available in all 3.x versions.

However, some people would like to clarify the policy of 2to3.
That means that 2to3 is compatible with the branch that is currently 
maintained? or it is converted to specific Python 3.X codes with the specific 
version of 2to3?

Cordially,
Dong-hee
—
Dong-hee Na
Chungnam National University | Computer Science & Engineering

Tel: +82 010-3353-9127
Email: donghee.n...@gmail.com
Linkedin: https://www.linkedin.com/in/dong-hee-na-2b713b49/
Github: https://github.com/corona10

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest isolation and warnings

[Brett Cannon]

Tests are not isolated from the warnings system, so things will leak out.
Your best option is to use the context manager in the warnings module to
temporarily make all warnings raise exceptions and test for the exception
(I'm at the airport, hence why I don't know the name of the context
manager; the warnings module docs actually have a sample on how best to
write tests the involve warnings).

On Fri, Nov 17, 2017, 01:34 Christian Tismer,  wrote:

> Hi guys,
>
> when writing tests, I suddenly discovered that unittest
> is not isolated to warnings.
>
> Example:
> One of my tests emits warnings when a certain condition is
> met. Instead of reporting the error immediately, it uses
> warnings, and at the end of the test, an error is produced
> if there were warnings.
>
> if hasattr(__main__, "__warningregistry__"):
> raise RuntimeError("There are errors, see above.")
>
> By chance, I discovered that an error was suddenly triggered without
> a warning. That must mean the warning existed already from
> another test as a left-over.
>
> My question:
> Is that known, and is that intended?
> To what extent are the test cases isolated from each other?
>
> I do admit that my usage of warnings is somewhat special.
> But it is very convenient to report many errors on remote servers.
>
> Cheers -- Chris
>
> --
> Christian Tismer :^)   tis...@stackless.com
> Software Consulting  : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : https://github.com/PySide
> 14482 Potsdam: GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python possible vulnerabilities in concurrency

[Koos Zevenhoven]

On Fri, Nov 17, 2017 at 3:40 PM, Koos Zevenhoven  wrote:

> On Thu, Nov 16, 2017 at 6:53 AM, Guido van Rossum 
> wrote:
>
>> On Wed, Nov 15, 2017 at 6:50 PM, Guido van Rossum 
>> wrote:
>>>
>>>
>>> Actually it linked to http://standards.iso.org/ittf/
>>> PubliclyAvailableStandards/index.html from which I managed to download
>>> what looks like the complete c061457_ISO_IEC_TR_24772_2013.pdf (336
>>> pages) after clicking on an "I accept" button (I didn't read what I
>>> accepted :-). The $200 is for the printed copy I presume.
>>>
>>
>> So far I learned one thing from the report. They use the term
>> "vulnerabilities" liberally, defining it essentially as "bug":
>>
>> All programming languages contain constructs that are incompletely
>>> specified, exhibit undefined behaviour, are implementation-dependent, or
>>> are difficult to use correctly. The use of those constructs may therefore
>>> give rise to *vulnerabilities*, as a result of which, software programs
>>> can execute differently than intended by the writer.
>>>
>>
>> They then go on to explain that sometimes vulnerabilities can be
>> exploited, but I object to calling all bugs vulnerabilities -- that's just
>> using a scary word to get attention for a sleep-inducing document
>> containing such gems as "Use floating-point arithmetic only when absolutely
>> needed" (page 230).
>>
>>
> ​I don't like such a definition of "vulnerability" either. Some bugs can
> be vulnerabilities (those that can be exploited) and some vulnerabilities
> can be bugs. But there are definitely types of vulnerabilities that are not
> bugs––the DoS vulnerability that is eliminated by hash randomization is one.
>
> There may also be a gray area of bugs that can be vulnerabilities but only
> in some special situation. I think it's ok to call those vulnerabilities
> too.
>
>
​Just to clarify the obvious: By the above, I *don't* mean that one could
use the word "vulnerability" for any functionality that can be used in such
a way that it creates a vulnerability. For example, `eval` or `exec` or
`open` by themselves are not vulnerabilities.

––Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python possible vulnerabilities in concurrency

[Koos Zevenhoven]

On Thu, Nov 16, 2017 at 6:53 AM, Guido van Rossum  wrote:

> On Wed, Nov 15, 2017 at 6:50 PM, Guido van Rossum 
> wrote:
>>
>>
>> Actually it linked to http://standards.iso.org/ittf/
>> PubliclyAvailableStandards/index.html from which I managed to download
>> what looks like the complete c061457_ISO_IEC_TR_24772_2013.pdf (336
>> pages) after clicking on an "I accept" button (I didn't read what I
>> accepted :-). The $200 is for the printed copy I presume.
>>
>
> So far I learned one thing from the report. They use the term
> "vulnerabilities" liberally, defining it essentially as "bug":
>
> All programming languages contain constructs that are incompletely
>> specified, exhibit undefined behaviour, are implementation-dependent, or
>> are difficult to use correctly. The use of those constructs may therefore
>> give rise to *vulnerabilities*, as a result of which, software programs
>> can execute differently than intended by the writer.
>>
>
> They then go on to explain that sometimes vulnerabilities can be
> exploited, but I object to calling all bugs vulnerabilities -- that's just
> using a scary word to get attention for a sleep-inducing document
> containing such gems as "Use floating-point arithmetic only when absolutely
> needed" (page 230).
>
>
​I don't like such a definition of "vulnerability" either. Some bugs can be
vulnerabilities (those that can be exploited) and some vulnerabilities can
be bugs. But there are definitely types of vulnerabilities that are not
bugs––the DoS vulnerability that is eliminated by hash randomization is one.

There may also be a gray area of bugs that can be vulnerabilities but only
in some special situation. I think it's ok to call those vulnerabilities
too.

​––Koos​


​PS. How come I haven't seen a proposal to remove the float type from
builtins yet?-)​


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] unittest isolation and warnings

[Christian Tismer]

Hi guys,

when writing tests, I suddenly discovered that unittest
is not isolated to warnings.

Example:
One of my tests emits warnings when a certain condition is
met. Instead of reporting the error immediately, it uses
warnings, and at the end of the test, an error is produced
if there were warnings.

if hasattr(__main__, "__warningregistry__"):
raise RuntimeError("There are errors, see above.")

By chance, I discovered that an error was suddenly triggered without
a warning. That must mean the warning existed already from
another test as a left-over.

My question:
Is that known, and is that intended?
To what extent are the test cases isolated from each other?

I do admit that my usage of warnings is somewhat special.
But it is very convenient to report many errors on remote servers.

Cheers -- Chris

-- 
Christian Tismer :^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com