Ah, I forgot to say that a major enhancement for the implementation is that I wrote a lot of new unit tests for the existing Python Initialization API. Python 3.7 has most of these tests. I wrote even more tests for my new private initialization API ;-)
I wrote these tests to specify and validate the priority between the different ways to configuration Python and "rules" (a parameter setting other parameters): https://github.com/python/peps/blob/master/pep-0587.rst#priority-and-rules Victor Le mer. 27 mars 2019 à 18:48, Victor Stinner <vstin...@redhat.com> a écrit : > > Hi, > > I would like to add a new C API to initialize Python. I would like > your opinion on the whole API before making it public. The code is > already implemented. Doc of the new API: > > https://pythondev.readthedocs.io/init_config.html > > > To make the API public, _PyWstrList, _PyInitError, _PyPreConfig, > _PyCoreConfig and related functions should be made public. > > By the way, I would suggest to rename "_PyCoreConfig" to just > "PyConfig" :-) I don't think that "core init" vs "main init" is really > relevant: more about that below. > > > Let's start with two examples using the new API. > > Example of simple initialization to enable isolated mode: > > _PyCoreConfig config = _PyCoreConfig_INIT; > config.isolated = 1; > > _PyInitError err = _Py_InitializeFromConfig(&config); > if (_Py_INIT_FAILED(err)) { > _Py_ExitInitError(err); > } > /* ... use Python API here ... */ > Py_Finalize(); > > Example using the pre-initialization to enable the UTF-8 Mode (and use the > "legacy" Py_Initialize() function): > > _PyPreConfig preconfig = _PyPreConfig_INIT; > preconfig.utf8_mode = 1; > > _PyInitError err = _Py_PreInitialize(&preconfig); > if (_Py_INIT_FAILED(err)) { > _Py_ExitInitError(err); > } > > /* at this point, Python will only speak UTF-8 */ > > Py_Initialize(); > /* ... use Python API here ... */ > Py_Finalize(); > > Since November 2017, I'm refactoring the Python Initialization code to > cleanup the code and prepare a new ("better") API to configure Python > Initialization. I just fixed the last issues that Nick Coghlan asked > me to fix (add a pre-initialization step: done, fix mojibake: done). > My work is inspired by Nick Coghlan's PEP 432, but it is not > implementing it directly. I had other motivations than Nick even if we > are somehow going towards the same direction. > > Nick wants to get a half-initialized Python ("core init"), configure > Python using the Python API and Python objects, and then finish the > implementation ("main init"). > > I chose a different approach: put *everything* into a single C > structure (_PyCoreConfig) using C types. Using the structure, you > should be able to do what Nick wanted to do, but with C rather than > Python. Nick: please tell me if I'm wrong :-) > > This work is also connected to Eric Snow's work on sub-interpreters > (PEP 554) and moving global variables into structures. For example, > I'm using his _PyRuntime structure to store a new "preconfig" state > (pre-initialization configuration, more about that below). > > In November 2017, when I started to work on the Python Initialization > (bpo-32030), I identified the following problems: > > * Many parts of the code were interdependent > * Code executed early in Py_Main() used the Python API before the Python API > was fully initialized. Like code parsing -W command line option which > used PyUnicode_FromWideChar() and PyList_Append(). > * Error handling used Py_FatalError() which didn't let the caller to decide > how to handle the error. Moreover, exit() was used to exit Python, > whereas libpython shouldn't do that: a library should not exit the > whole process! (imagine when Python is embedded inside an application) > > One year and a half later, I implemented the following solutions: > > * Py_Main() and Py_Initialize() code has been reorganized to respect > priorities between global configuration variables (ex: > Py_IgnoreEnvironmentFlag), environment variables (ex: PYTHONPATH), command > line arguments (ex: -X utf8), configuration files (ex: pyenv.cfg), and the > new _PyPreConfig and _PyCoreConfig structures which store the whole > configuration. > * Python Initialization no longer uses the Python API but only C types > like wchar_t* strings, a new _PyWstrList structure and PyMem_RawMalloc() > memory allocator (PyMem_Malloc() is no longer used during init). > * The code has been modified to use a new _PyInitError structure. The caller > of the top function gets control to cleanup everything before handling the > error (display a fatal error message or simply exit Python). > > The new _PyCoreConfig structure has the top-priority and provides a single > structure for all configuration parameters. > > It becomes possible to override the code computing the "path configuration" > like sys.path to fully control where Python looks to import modules. It > becomes possible to use an empty list of paths to only allow builtin modules. > > A new "pre-initialization" steps is responsible to configure the bare minimum > before the Python initialization: memory allocators and encodings > (LC_CTYPE locale > and the UTF-8 mode). The LC_CTYPE is no longer coerced and the UTF-8 Mode is > no longer enabled automatically depending on the user configuration to prevent > mojibake. Previously, calling Py_DecodeLocale() to get a Unicode wchar_t* > string from a bytes wchar* string created mojibake when called before > Py_Initialize() if the LC_CTYPE locale was coerced and/or if the UTF-8 Mode > was > enabled. > > The pre-initialization step ensures that the encodings and memory allocators > are well defined *before* Py_Initialize() is called. > > Since the new API is currently private, I didn't document it in > Python. Moreover, the code changed a lot last year :-) But it should > now be way more stable. I started to document it in a separated > webpage: > > https://pythondev.readthedocs.io/init_config.html > > The plan is to put it in the Python documentation once it becomes public. > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
