[issue38266] Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal
Wenzel Jakob added the comment: This is great -- thank you for handling this so quickly. -- ___ Python tracker <https://bugs.python.org/issue38266> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38266] Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal
New submission from Wenzel Jakob : A commit from a few days ago and discussed in issue #37878 removed an undocumented function PyThreadState_DeleteCurrent() from Python's public API. This function was exposed for good reasons and simply removing it because it is undocumented strikes me as a somewhat brutal solution to achieve documentation coverage. This function is useful -- even essential -- to delete the current thread's thread state, which cannot be done using the still-public PyThreadState_Delete(). The documentation could simply be a line stating this explicitly. I was asked to provide an example of an actual usage: this function is used by pybind11, which is a widely used library for creating Python bindings to C++ code. pybind11 only calls PyThreadState_DeleteCurrent() in a rare set of circumstances, but there is no alternative in this case. Specifically, pybind11 can intercept a Python function call on the main() thread and delete the associated thread state, launching a new thread and continuing Python execution there (with a newly created thread state). Kind of crazy, so why is this useful? The answer is UI libraries. On some platforms, it is not legal to poll UI events on any thread other than the main thread. This means that it's impossible to implement a proper asynchronous UI event loop because Python is hogging the main thread. With the functionality in pybind11's gil_scoped_acquire, it is possible can launch an event polling loop on the main thread, continue running an interactive Python REPL on another thread, and even swap them back e.g. when the user interface is no longer needed. Best, Wenzel -- components: Interpreter Core messages: 353114 nosy: eric.snow, nanjekyejoannah, ncoghlan, tcaswell, vstinner, wenzel priority: normal severity: normal status: open title: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal type: behavior versions: Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38266> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37878] Sub-Interpreters : Document PyThreadState_DeleteCurrent()
Wenzel Jakob added the comment: Hi, pybind11 developer here. A bit on context of our usage of this function: Pybind11 supports some advanced GIL-related tricks that *require* access this function. For instance, pybind11 can intercept a Python function call on the `main()` thread and delete the associated thread state, launching a new thread and continuing Python execution there (with a newly created thread state). Kind of crazy, but why is this useful? UI libraries. On some platforms, it is not legal to poll UI events on any thread other than the main thread. This means that it's impossible to implement a proper UI event loop because Python is hogging the main thread. With the functionality in pybind11's ``gil_scoped_acquire``, I can launch an event polling loop on the main thread, continue running an interactive Python session on another thread, and even swap them back e.g. when the user interface is no longer needed. Best, Wenzel -- nosy: +wenzel ___ Python tracker <https://bugs.python.org/issue37878> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28147] Unbounded memory growth resizing split-table dicts
Wenzel Jakob added the comment: Hi, pybind11 (https://github.com/pybind/pybind11) dev here. We're seeing massive memory increases due to this bug, which completely break our test suite (and likely any use of this library, which is commonly used to bind C++ code to Python). Please look at the following issue ticket: https://github.com/pybind/pybind11/issues/558 where RSS goes to 43MB to 4.3GB for the basic set of tests. The fancy fancy test suite which also covers NumPy/SciPy can't even be run anymore. All issues disappear when the dict patch listed here is applied. We're really concerned that this is not slated to be fixed for 3.6.0. Pybind11 is not doing anything particularly special with dicts -- if this is hitting us, others will likely have issues as well. -Wenzel -- nosy: +wenzel ___ Python tracker <http://bugs.python.org/issue28147> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26154] Add private _PyThreadState_UncheckedGet() to get the current thread state
Wenzel Jakob added the comment:
I've also run into this regression. FWIW this is what I've ended up using to
work around it (it's a mess, but what are we to do..)
#if PY_VERSION_HEX >= 0x0305 && PY_VERSION_HEX < 0x03050200
extern "C" {
/* Manually import _PyThreadState_Current symbol */
struct _Py_atomic_address { void *value; };
PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current;
};
#endif
PyThreadState *get_thread_state_unchecked() {
#if PY_VERSION_HEX < 0x0300
return _PyThreadState_Current;
#elif PY_VERSION_HEX < 0x0305
return (PyThreadState*) _Py_atomic_load_relaxed(&_PyThreadState_Current);
#elif PY_VERSION_HEX < 0x03050200
return (PyThreadState*) _PyThreadState_Current.value;
#else
return _PyThreadState_UncheckedGet();
#endif
}
--
nosy: +wenzel
___
Python tracker
<http://bugs.python.org/issue26154>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25006] List pybind11 binding generator
Wenzel Jakob added the comment: Never mind -- I made an entry for it. See https://pypi.python.org/pypi/pybind11/1.0 Thanks, Wenzel -- ___ Python tracker <http://bugs.python.org/issue25006> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25006] List pybind11 binding generator
Wenzel Jakob added the comment: Dear Antoine, I wonder if this makes sense, as pybind11 is a collection of C++ header files using the Python C API. The library is meant to be used by other projects but does not generate any installable code by itself. (i.e. it isn't clear what pip install pybind11 should even do) I haven't seen any PyPI packages of this type, though I'm happy to be told otherwise. Best, Wenzel -- ___ Python tracker <http://bugs.python.org/issue25006> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25006] List pybind11 binding generator
Wenzel Jakob added the comment: Hi, just another ping regarding this ticket. Since the last message, pybind11 now has: - Documentation on readthedocs: http://pybind11.readthedocs.org/en/latest/ - Continuous integration tests: https://travis-ci.org/wjakob/pybind11 - A first stable release: https://github.com/wjakob/pybind11/releases It would be fantastic if the Python documentation referenced it as an option for binding C++ code to Python. Thanks, Wenzel -- ___ Python tracker <http://bugs.python.org/issue25006> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25006] List pybind11 binding generator
New submission from Wenzel Jakob:
Hi,
over the last few months, I've been developing a library ("pybind11") for
creating Python bindings of C++ code. It is similar in spirit to Boost.Python
but uses a very different minimalist approach.
It would be fantastic to get this library listed here:
https://docs.python.org/3/faq/extending.html
The purpose of this ticket is that somebody with permissions might add it to
this page.
pybind11 is here: https://github.com/wjakob/pybind11
Please let me know if you need any additional information.
Thank you,
Wenzel
--
assignee: docs@python
components: Documentation
messages: 249865
nosy: docs@python, wenzel
priority: normal
severity: normal
status: open
title: List pybind11 binding generator
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
___
Python tracker
<http://bugs.python.org/issue25006>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
