Re: [Python-Dev] New Python Initialization API

2019-03-28 Thread Victor Stinner
Le jeu. 28 mars 2019 à 05:27, Stephen J. Turnbull
 a écrit :
> Victor Stinner writes:
>
>  > I just fixed the mojibake issue in Python 3.8 by disabling C locale
>  > coercion and UTF-8 Mode by default. I'm not sure if nor how Python 3.7
>  > should be fixed in a minor 3.7.x release.
>
> That sounds like a potential regression.  Those two features were
> added *and turned on by default* (which really means "if you detect
> LC_TYPE=C, coerce") to relieve previously existing mojibake/
> UnicodeError issues due to ASCII-only environments that are difficult
> to configure (such as containers).  Turning them on by default was the
> controversial part -- it was known that on or off, some environments
> would have problems, and that's why they needed PEPs.  Do those issues
> return now?  If so, where is the PEP rationale for defaulting to "on"
> faulty?

If you use "python3.8", there is no change. I'm only talking about the
specific case of Python embedded in an application: when you use the C
API.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 556 threaded garbage collection & linear recursion in gc

2019-03-28 Thread Antoine Pitrou
On Wed, 27 Mar 2019 15:59:25 -0700
"Gregory P. Smith"  wrote:
> 
> That had a C++ stack trace 1000+ levels deep repeating the pattern of
> 
> ...
> @ 0x564d59bd21de 32  func_dealloc
> @ 0x564d59bce0c1 32  cell_dealloc
> @ 0x564d5839db41 48  tupledealloc
> @ 0x564d59bd21de 32  func_dealloc
> @ 0x564d59bce0c1 32  cell_dealloc
> @ 0x564d5839db41 48  tupledealloc
> ...

As Tim said, if you still have a core dump somewhere (or can reproduce
the issue) it would be nice to know why the "trashcan" mechanism didn't
trigger.

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 556 threaded garbage collection & linear recursion in gc

2019-03-28 Thread Jeroen Demeyer

On 2019-03-28 01:38, Tim Peters wrote:

The bad news is that the traschcan mechanism is excruciating, a
long-time source of subtle bugs of its own :-(


It just happens that I created a PR to fix some of the trashcan 
problems: see https://bugs.python.org/issue35983 and 
https://github.com/python/cpython/pull/11841

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] New Python Initialization API

2019-03-28 Thread Ivan Pozdeev via Python-Dev



On 27.03.2019 20:48, Victor Stinner wrote:

Hi,

I would like to add a new C API to initialize Python. I would like
your opinion on the whole API before making it public. The code is
already implemented. Doc of the new API:

https://pythondev.readthedocs.io/init_config.html


To make the API public, _PyWstrList, _PyInitError, _PyPreConfig,
_PyCoreConfig and related functions should be made public.

By the way, I would suggest to rename "_PyCoreConfig" to just
"PyConfig" :-) I don't think that "core init" vs "main init" is really
relevant: more about that below.


Let's start with two examples using the new API.

Example of simple initialization to enable isolated mode:

 _PyCoreConfig config = _PyCoreConfig_INIT;
 config.isolated = 1;

 _PyInitError err = _Py_InitializeFromConfig(&config);
By my outsider observation, the `config' argument and return code are asking to be added to Py_Initialize instead, 
`_Py_InitializeFromConfig` and `_Py_PreInitialize` look redundant.

 if (_Py_INIT_FAILED(err)) {
 _Py_ExitInitError(err);
 }
 /* ... use Python API here ... */
 Py_Finalize();

Example using the pre-initialization to enable the UTF-8 Mode (and use the
"legacy" Py_Initialize() function):

 _PyPreConfig preconfig = _PyPreConfig_INIT;
 preconfig.utf8_mode = 1;

 _PyInitError err = _Py_PreInitialize(&preconfig);
 if (_Py_INIT_FAILED(err)) {
 _Py_ExitInitError(err);
 }

 /* at this point, Python will only speak UTF-8 */

 Py_Initialize();
 /* ... use Python API here ... */
 Py_Finalize();

Since November 2017, I'm refactoring the Python Initialization code to
cleanup the code and prepare a new ("better") API to configure Python
Initialization. I just fixed the last issues that Nick Coghlan asked
me to fix (add a pre-initialization step: done, fix mojibake: done).
My work is inspired by Nick Coghlan's PEP 432, but it is not
implementing it directly. I had other motivations than Nick even if we
are somehow going towards the same direction.

Nick wants to get a half-initialized Python ("core init"), configure
Python using the Python API and Python objects, and then finish the
implementation ("main init").

I chose a different approach: put *everything* into a single C
structure (_PyCoreConfig) using C types. Using the structure, you
should be able to do what Nick wanted to do, but with C rather than
Python. Nick: please tell me if I'm wrong :-)

This work is also connected to Eric Snow's work on sub-interpreters
(PEP 554) and moving global variables into structures. For example,
I'm using his _PyRuntime structure to store a new "preconfig" state
(pre-initialization configuration, more about that below).

In November 2017, when I started to work on the Python Initialization
(bpo-32030), I identified the following problems:

* Many parts of the code were interdependent
* Code executed early in Py_Main() used the Python API before the Python API
   was fully initialized. Like code parsing -W command line option which
   used PyUnicode_FromWideChar() and PyList_Append().
* Error handling used Py_FatalError() which didn't let the caller to decide
   how to handle the error. Moreover, exit() was used to exit Python,
whereas libpython shouldn't do that: a library should not exit the
whole process! (imagine when Python is embedded inside an application)

One year and a half later, I implemented the following solutions:

* Py_Main() and Py_Initialize() code has been reorganized to respect
   priorities between global configuration variables (ex:
   Py_IgnoreEnvironmentFlag), environment variables (ex: PYTHONPATH), command
   line arguments (ex: -X utf8), configuration files (ex: pyenv.cfg), and the
   new _PyPreConfig and _PyCoreConfig structures which store the whole
   configuration.
* Python Initialization no longer uses the Python API but only C types
   like wchar_t* strings, a new _PyWstrList structure and PyMem_RawMalloc()
   memory allocator (PyMem_Malloc() is no longer used during init).
* The code has been modified to use a new _PyInitError structure. The caller
   of the top function gets control to cleanup everything before handling the
   error (display a fatal error message or simply exit Python).

The new _PyCoreConfig structure has the top-priority and provides a single
structure for all configuration parameters.

It becomes possible to override the code computing the "path configuration"
like sys.path to fully control where Python looks to import modules. It
becomes possible to use an empty list of paths to only allow builtin modules.

A new "pre-initialization" steps is responsible to configure the bare minimum
before the Python initialization: memory allocators and encodings
(LC_CTYPE locale
and the UTF-8 mode). The LC_CTYPE is no longer coerced and the UTF-8 Mode is
no longer enabled automatically depending on the user configuration to prevent
mojibake. Previously, calling Py_DecodeLocale() to get a Unicode wchar_t*
st

Re: [Python-Dev] New Python Initialization API

2019-03-28 Thread Victor Stinner
Le mer. 27 mars 2019 à 21:26, Brett Cannon  a écrit :
> On Wed, Mar 27, 2019 at 12:39 PM Steve Dower  wrote:
>> I this this should be its own PEP, since as you say it is not
>> implementing the only PEP we have (or alternatively, maybe you should
>> collaborate to update PEP 432 so that it reflects what you think we
>> ought to be implementing).
>
> I agree that if this isn't doing what PEP 432 set out but going its own way 
> we should probably discuss in regards to 432.

I'm sorry, I was in a hurry when I wrote the new PEP 587 and it seems
like I created some confusion. My PEP 587 is very similar to the PEP
432, because it is basically an implementation of the PEP 432 design.
I am collaborating closely with Nick Coghlan and Eric Snow on the
Python Initialization for 1 year and a half, and I just continued the
work they started.

The PEP 432 has been written in 2012 and has a "broader scope". Since
the PEP has been written, the code has been modified slowly towards
PEP 432 design, but not "exactly" the expected design, because of
concrete practical issues of the implementation.

The PEP 587 is the updated *implementation* of the PEP 432 which can
be seen as the overall *design*.

The PEP 587 is only a subset of the PEP 432: C API to initialize
Python, whereas PEP 432 goes further by introducing the concepts of
"core" and "main" initialization. The "core initialization" is a bare
minimum working Python only with builtin types, partial sys module and
no importlib. "Main initialization" is a fully working Python. This
part is out of the scope of the PEP 587, but the PEP 587 should be
flexible enough to allow to implement it later.

In fact, there is already a PyConfig._init_config flag (currently
named _PyCoreConfig._init_main) which only initializes Python up to
the "core initialization" if set to 0. This parameter is private since
it's unclear to me what should be the exact scope of "core" vs "main"
init.

I wrote a PR to clarify the relationship between the PEP 587 and the PEP 432:
https://github.com/python/peps/pull/955/files

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] New Python Initialization API

2019-03-28 Thread Steve Dower

On 28Mar2019 0703, Victor Stinner wrote:

In fact, there is already a PyConfig._init_config flag (currently
named _PyCoreConfig._init_main) which only initializes Python up to
the "core initialization" if set to 0. This parameter is private since
it's unclear to me what should be the exact scope of "core" vs "main"
init.


We tried to set up a video call between the interested people (Eric, 
Nick, myself, yourself, couple of others) to clarify this point, and you 
refused to join ;)


That said, the call never happened (honestly, there's not a lot of point 
in doing it without you being part of it), so we still don't have a 
clear idea of where the line should be drawn. But there are enough of us 
with fuzzy but valid ideas in our heads that we really need that 
brainstorming session to mix them together and find something feasible. 
Maybe we're best to put it off until PyCon at this point.


Cheers,
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP 570

2019-03-28 Thread Pablo Galindo Salgado
Hi everyone,

We have submitted PEP 570 for consideration to the steering council:

https://github.com/python/steering-council/issues/4

The discussion is happening on discourse:

https://discuss.python.org/t/pep-570-python-positional-only-parameters/1078

To eliminate splitting the discussion into two forums (discourse and
python-dev),
we kindly ask you to go to discourse if you want to participate in the
debate :)

Here is the full document of the PEP:

https://www.python.org/dev/peps/pep-0570/

Pablo Galindo Salgado
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] VxWorks and Python

2019-03-28 Thread Kuhl, Brian
Hi Python developers,
Victor suggested I post a little background to help the maintainers understand 
VxWorks a little better.
It can be found here
https://github.com/Wind-River/cpython/wiki/Python-on-VxWorks
If you'd like more detail in a particular area, please feel free to reply on or 
off list.

I'd also like to mention again, that Wind River is interested in setting up a 
short term consulting arrangement, with any core dev that's available.
To help accelerate Peixing's PRs acceptance
https://github.com/python/cpython/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+bpo-31904
and generally improve Cpython.

Wind River operates globally so don't let you location dissuade you.

Many thanks for all your help,

Brian
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 556 threaded garbage collection & linear recursion in gc

2019-03-28 Thread Neil Schemenauer
On 2019-03-28, Antoine Pitrou wrote:
> On Wed, 27 Mar 2019 15:59:25 -0700
> "Gregory P. Smith"  wrote:
> > 
> > That had a C++ stack trace 1000+ levels deep repeating the pattern of
> > 
> > ...
> > @ 0x564d59bd21de 32  func_dealloc
> > @ 0x564d59bce0c1 32  cell_dealloc
> > @ 0x564d5839db41 48  tupledealloc
> > @ 0x564d59bd21de 32  func_dealloc
> > @ 0x564d59bce0c1 32  cell_dealloc
> > @ 0x564d5839db41 48  tupledealloc
> > ...
> 
> As Tim said, if you still have a core dump somewhere (or can reproduce
> the issue) it would be nice to know why the "trashcan" mechanism didn't
> trigger.

To expand on this, every time tupledealloc gets called,
Py_TRASHCAN_SAFE_BEGIN also gets invoked.  It increments
tstate->trash_delete_nesting.  As Tim suggests, maybe
PyTrash_UNWIND_LEVEL is too large given the size of the C stack
frames from func_dealloc + cell_dealloc + tupledealloc.

That theory seems hard to believe though, unless the C stack is
quite small.  I see PyTrash_UNWIND_LEVEL = 50.  Perhaps the stack
could have been mostly used up before the dealloc sequence started.

The other option is that there is some bug in the trashcan
mechanism.  It certainly is some very tricky code.

Regards,

  Neil
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Steve Dower

Hi all

Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into 
Python 3.8. Here's the current text for review and comment before I 
submit to the Steering Council.


The formatted text is at https://www.python.org/dev/peps/pep-0578/ 
(update just pushed, so give it an hour or so, but it's fundamentally 
the same as what's there)


No Discourse post, because we don't have a python-dev equivalent there 
yet, so please reply here for this one.


Implementation is at https://github.com/zooba/cpython/tree/pep-578/ and 
my backport to 3.7 (https://github.com/zooba/cpython/tree/pep-578-3.7/) 
is already getting some real use (though this will not be added to 3.7, 
unless people *really* want it, so the backport is just for reference).


Cheers,
Steve

=

PEP: 578
Title: Python Runtime Audit Hooks
Version: $Revision$
Last-Modified: $Date$
Author: Steve Dower 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Jun-2018
Python-Version: 3.8
Post-History:

Abstract


This PEP describes additions to the Python API and specific behaviors
for the CPython implementation that make actions taken by the Python
runtime visible to auditing tools. Visibility into these actions
provides opportunities for test frameworks, logging frameworks, and
security tools to monitor and optionally limit actions taken by the
runtime.

This PEP proposes adding two APIs to provide insights into a running
Python application: one for arbitrary events, and another specific to
the module import system. The APIs are intended to be available in all
Python implementations, though the specific messages and values used
are unspecified here to allow implementations the freedom to determine
how best to provide information to their users. Some examples likely
to be used in CPython are provided for explanatory purposes.

See PEP 551 for discussion and recommendations on enhancing the
security of a Python runtime making use of these auditing APIs.

Background
==

Python provides access to a wide range of low-level functionality on
many common operating systems. While this is incredibly useful for
"write-once, run-anywhere" scripting, it also makes monitoring of
software written in Python difficult. Because Python uses native system
APIs directly, existing monitoring tools either suffer from limited
context or auditing bypass.

Limited context occurs when system monitoring can report that an
action occurred, but cannot explain the sequence of events leading to
it. For example, network monitoring at the OS level may be able to
report "listening started on port 5678", but may not be able to
provide the process ID, command line, parent process, or the local
state in the program at the point that triggered the action. Firewall
controls to prevent such an action are similarly limited, typically
to process names or some global state such as the current user, and
in any case rarely provide a useful log file correlated with other
application messages.

Auditing bypass can occur when the typical system tool used for an
action would ordinarily report its use, but accessing the APIs via
Python do not trigger this. For example, invoking "curl" to make HTTP
requests may be specifically monitored in an audited system, but
Python's "urlretrieve" function is not.

Within a long-running Python application, particularly one that
processes user-provided information such as a web app, there is a risk
of unexpected behavior. This may be due to bugs in the code, or
deliberately induced by a malicious user. In both cases, normal
application logging may be bypassed resulting in no indication that
anything out of the ordinary has occurred.

Additionally, and somewhat unique to Python, it is very easy to affect
the code that is run in an application by manipulating either the
import system's search path or placing files earlier on the path than
intended. This is often seen when developers create a script with the
same name as the module they intend to use - for example, a
``random.py`` file that attempts to import the standard library
``random`` module.

This is not sandboxing, as this proposal does not attempt to prevent
malicious behavior (though it enables some new options to do so).
See the `Why Not A Sandbox`_ section below for further discussion.

Overview of Changes
===

The aim of these changes is to enable both application developers and
system administrators to integrate Python into their existing
monitoring systems without dictating how those systems look or behave.

We propose two API changes to enable this: an Audit Hook and Verified
Open Hook. Both are available from Python and native code, allowing
applications and frameworks written in pure Python code to take
advantage of the extra messages, while also allowing embedders or
system administrators to deploy builds of Python where auditing is
always enabled.

Only CPython is bound to provide the native APIs as described here.
Other implementations 

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Steve Dower
The implementation can be viewed as a PR at 
https://github.com/python/cpython/pull/12613


On 28Mar2019 1535, Steve Dower wrote:

Hi all

Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into 
Python 3.8. Here's the current text for review and comment before I 
submit to the Steering Council.


The formatted text is at https://www.python.org/dev/peps/pep-0578/ 
(update just pushed, so give it an hour or so, but it's fundamentally 
the same as what's there)


No Discourse post, because we don't have a python-dev equivalent there 
yet, so please reply here for this one.


Implementation is at https://github.com/zooba/cpython/tree/pep-578/ and 
my backport to 3.7 (https://github.com/zooba/cpython/tree/pep-578-3.7/) 
is already getting some real use (though this will not be added to 3.7, 
unless people *really* want it, so the backport is just for reference).


Cheers,
Steve

=

PEP: 578
Title: Python Runtime Audit Hooks
Version: $Revision$
Last-Modified: $Date$
Author: Steve Dower 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Jun-2018
Python-Version: 3.8
Post-History:

Abstract


This PEP describes additions to the Python API and specific behaviors
for the CPython implementation that make actions taken by the Python
runtime visible to auditing tools. Visibility into these actions
provides opportunities for test frameworks, logging frameworks, and
security tools to monitor and optionally limit actions taken by the
runtime.

This PEP proposes adding two APIs to provide insights into a running
Python application: one for arbitrary events, and another specific to
the module import system. The APIs are intended to be available in all
Python implementations, though the specific messages and values used
are unspecified here to allow implementations the freedom to determine
how best to provide information to their users. Some examples likely
to be used in CPython are provided for explanatory purposes.

See PEP 551 for discussion and recommendations on enhancing the
security of a Python runtime making use of these auditing APIs.

Background
==

Python provides access to a wide range of low-level functionality on
many common operating systems. While this is incredibly useful for
"write-once, run-anywhere" scripting, it also makes monitoring of
software written in Python difficult. Because Python uses native system
APIs directly, existing monitoring tools either suffer from limited
context or auditing bypass.

Limited context occurs when system monitoring can report that an
action occurred, but cannot explain the sequence of events leading to
it. For example, network monitoring at the OS level may be able to
report "listening started on port 5678", but may not be able to
provide the process ID, command line, parent process, or the local
state in the program at the point that triggered the action. Firewall
controls to prevent such an action are similarly limited, typically
to process names or some global state such as the current user, and
in any case rarely provide a useful log file correlated with other
application messages.

Auditing bypass can occur when the typical system tool used for an
action would ordinarily report its use, but accessing the APIs via
Python do not trigger this. For example, invoking "curl" to make HTTP
requests may be specifically monitored in an audited system, but
Python's "urlretrieve" function is not.

Within a long-running Python application, particularly one that
processes user-provided information such as a web app, there is a risk
of unexpected behavior. This may be due to bugs in the code, or
deliberately induced by a malicious user. In both cases, normal
application logging may be bypassed resulting in no indication that
anything out of the ordinary has occurred.

Additionally, and somewhat unique to Python, it is very easy to affect
the code that is run in an application by manipulating either the
import system's search path or placing files earlier on the path than
intended. This is often seen when developers create a script with the
same name as the module they intend to use - for example, a
``random.py`` file that attempts to import the standard library
``random`` module.

This is not sandboxing, as this proposal does not attempt to prevent
malicious behavior (though it enables some new options to do so).
See the `Why Not A Sandbox`_ section below for further discussion.

Overview of Changes
===

The aim of these changes is to enable both application developers and
system administrators to integrate Python into their existing
monitoring systems without dictating how those systems look or behave.

We propose two API changes to enable this: an Audit Hook and Verified
Open Hook. Both are available from Python and native code, allowing
applications and frameworks written in pure Python code to take
advantage of the extra messages, while also allowing embedders or
system administrators to deploy builds of Py

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Victor Stinner
Hi,

I read quickly the PEP, I'm not sure that I understood it correctly,
so here are some early questions more about the usage of the PEP, than
its implementation.

> This is not sandboxing, as this proposal does not attempt to prevent
> malicious behavior (though it enables some new options to do so).
> See the `Why Not A Sandbox`_ section below for further discussion.

I don't understand well the overall security model. If malicious
behaviors can still occur, what is the the purpose of auditing? For
example, if an audit hook writes events into a local log file, the
attacker can easily remove this log file, no?

While you say that it's not a sandbox, you designed multiple
protections to protect auditing, and the design is very close to a
sandbox.

Example:

"``_PyObject_GenericSetAttr``, ``check_set_special_type_attr``,
``object_set_class``, ``func_set_code``, ``func_set_[kw]defaults``","
``object.__setattr__``","``(object, attr, value)``","Detect monkey
patching of types and objects. This event
is raised for the ``__class__`` attribute and any attribute on
``type`` objects.

It reminds me Python 2 Bastion module which has simply been removed
because it was unsafe:
https://docs.python.org/2/library/bastion.html

For example, using ctypes, you can access directly the underlying
dictionary of a type and modify "private" attributes. It's just an
example.


I wrote pysandbox in the past, and it was broken by default, as you
wrote in the PEP :-)

   The failure of pysandbox (2013)
   https://lwn.net/Articles/574215/

If you want to secure Python, run Python in a sandbox, don't try to
"add" a sandbox "on top" of Python (I know that it's more complicated
in practice).

Or use PyPy sandbox which has a very different design.


Le jeu. 28 mars 2019 à 23:39, Steve Dower  a écrit :
> This PEP describes additions to the Python API and specific behaviors
> for the CPython implementation that make actions taken by the Python
> runtime visible to auditing tools. Visibility into these actions
> provides opportunities for test frameworks, logging frameworks, and
> security tools to monitor and optionally limit actions taken by the
> runtime.

Is it possible to implement these features without adding a new API or
modifying Python?

Short example adding logs to open():
---
import builtins
import functools

def add_log(name, func):
@functools.wraps(func)
def wrapper(*args):
print(name, args)
return func(*args)
return wrapper

builtins.open = add_log("open", builtins.open)

open(__file__).close()
---


> Verified Open Hook
> --
>
> Most operating systems have a mechanism to distinguish between files
> that can be executed and those that can not. For example, this may be an
> execute bit in the permissions field, a verified hash of the file
> contents to detect potential code tampering, or file system path
> restrictions. These are an important security mechanism for preventing
> execution of data or code that is not approved for a given environment.
> Currently, Python has no way to integrate with these when launching
> scripts or importing modules.

In my experience, it doesn't work just because Python has too many
functions opening files indirectly or call external C libraries which
open files.

I vaguely recall an exploit in my pysandbox project which uses the
internal code of Python which displays a traceback... to read the
content of an arbitrary file on the disk :-( Game over. I would never
expect that there are so many ways to read a file in Python...

Even when I restricted pysandbox to the bare minimum of the Python
language (with no import), multiple exploits have been found.
Moreover, at the end, Python just became useful.

More generally, there are a lot of codes in Python which allow
arbitrary code injection :-( (Most of them are now fixed, hopefully!)

I did my best to modify as much functions as possible to implement the
PEP 446 "Make newly created file descriptors non-inheritable", but I
know that *many* functions call directly open() or fopen() and so
create inheritable file descriptors. For example, the Python ssl
module takes directly filenames and OpenSSL open directly files. It's
just one example.

You will never be able to cover all cases.

Having a single function which allows to open an arbitrary file
without triggering an audit event would defeat the whole purpose of
auditing, no? Again, maybe I didn't understand well the overall
purpose of the PEP, sorry.


> Table 2 provides further examples that are not required, but are
> likely to be available in CPython.
> (...)
> Performance Impact
> ==
>
> The important performance impact is the case where events are being
> raised but there are no hooks attached. This is the unavoidable case -
> once a developer has added audit hooks they have explicitly chosen to
> trade performance for functionality.

(The Linux kernel uses advance tooling to inject hooks: it has no
impact o

[Python-Dev] A request for PEP announcement format [was: PEP 570]

2019-03-28 Thread Stephen J. Turnbull
A proponent writes:

 > We have submitted PEP 570 for consideration to the steering
 > council:

I would like to ask *future* proponents to please include the title
explicitly in the first mention.  (This is not a criticism of the
previous post.  How could he know I would write this?)

In this case, it's here:

 > https://discuss.python.org/t/pep-570-Python-Positional-Only-Parameters/1078

(sorry, case changed for emphasis so clicking won't work) so no great
harm done!

Thanks in advance!

Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A request for PEP announcement format [was: PEP 570]

2019-03-28 Thread Jonathan Goble
On Thu, Mar 28, 2019 at 11:11 PM Stephen J. Turnbull <
[email protected]> wrote:

>  >
> https://discuss.python.org/t/pep-570-Python-Positional-Only-Parameters/1078
>
> (sorry, case changed for emphasis so clicking won't work)
>

Clicking actually did work for me. Not only is that portion of the link not
case-sensitive, but it doesn't even matter what it is or if it's present at
all. https://discuss.python.org/t/pep-570-fgfdgfdgdfgd-parameters/1078 and
https://discuss.python.org/t/1078 both lead to the same thread, but if I
change the number at the end to 1077, I get a different thread.

So the system only uses the number at the end to identify the thread, and
the text is strictly for the benefit of human readability and not parsed by
the server. File that in the "probably useless information" category of
your brain. ;-)

*goes back to lurking*
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A request for PEP announcement format [was: PEP 570]

2019-03-28 Thread Jeroen Demeyer

On 2019-03-29 04:08, Stephen J. Turnbull wrote:

In this case, it's here:

  > https://discuss.python.org/t/pep-570-Python-Positional-Only-Parameters/1078


So, are we supposed to discuss PEPs on discuss.python.org now? That's 
fine for me, should I create a thread like that for PEP 580 too?

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com