python-build-standalone
(https://github.com/indygreg/python-build-standalone) is a project
that produces standalone, highly portable builds of CPython that are
designed to run on as many machines as possible with no additional
dependencies. I created the project for PyOxidizer
I'm excited to announce the initial release of PyOxy: a single executable
file (C)Python distribution and alternative Python runner that gives you
more flexibility and control over how Python interpreters are run.
More details at
oxidized_importer - a pure Rust Python extension module implementing high
performance Python module importers - version 0.3 has been released.
The documentation is available at
https://pyoxidizer.readthedocs.io/en/oxidized-importer-0.3/oxidized_importer.html
.
Like previous versions of this
Gregory Szorc added the comment:
I can't remember the circumstances of me reproducing this. The added test
demonstrating that the interpreter can be initialized/finalized within a single
process seems like a reasonable solution to ensure this is supported
Gregory Szorc added the comment:
> Can you be more specific? This particular issue isn't about cross-compiling.
The point I was trying to make is that clang 13 generically implemented
--print-multiarch and the triple logic in configure related to MULTIARCH
applies to all platforms, not j
Gregory Szorc added the comment:
Note that this issue isn't macOS specific: you will get the same failure when
cross-compiling targeting Linux. e.g. --build=x86_64-unknown-linux-gnu
--host=i686-unknown-linux-gnu.
--
nosy: +indygreg
___
Python
Gregory Szorc added the comment:
I didn't want to derail this old issue too much. So I filed issue45307 to track
a solution.
--
___
Python tracker
<https://bugs.python.org/issue41
New submission from Gregory Szorc :
https://bugs.python.org/issue41994 / commit
4db8988420e0a122d617df741381b0c385af032c removed
_PyImport_FindExtensionObject() from the C API and it is no longer available in
CPython 3.10 after alpha 5.
At least py2exe and PyOxidizer rely on this API
Gregory Szorc added the comment:
While testing 3.10.0rc2, I noticed that commit
4db8988420e0a122d617df741381b0c385af032c removed
_PyImport_FindExtensionObject() from libpython. This breaks both PyOxidizer and
py2exe, which rely on this function for a custom implementation
Gregory Szorc added the comment:
I'll throw out that __file__ is strictly optional, per
https://docs.python.org/3/reference/datamodel.html. IMO it is more important to
define importlib.abc.InspectLoader than __file__ (so tracebacks have source
context).
Given that __file__ is optional
Gregory Szorc added the comment:
I cannot reproduce this on an 11.5 Intel MacBook Pro using an 11.5 SDK
targeting x86_64 10.9.
However, I can reproduce on a similarly configured M1 using the same OS and SDK
but cross-compiling to single arch x86_64.
The issue here may reside in how
Gregory Szorc added the comment:
> On Sep 1, 2021, at 22:58, Ned Deily wrote:
>
> I don't think we have ever claimed to support building on an older system
> with a newer SDK, as in building on 10.15 with an 11 SDK.
My initial report was from a 10.15 Intel machine in GitHub
Gregory Szorc added the comment:
I spoke too soon: you can reproduce this with CPython's build system and I
think this is a legit regression.
I think the function sniffing logic is subtly wrong.
Here is the logic as written:
#ifdef __APPLE__
#ifdef HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH
Gregory Szorc added the comment:
Oh, this might be my custom Modules file not passing the
HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH define properly. I suspect I was working
around this bug before by disabling the dyld bits. Sorry for the noise! (I
should have reproduced with CPython's build
Gregory Szorc added the comment:
I think patch broke building on macOS < 11 when building with a 11.0+ SDK and
targeting macOS < 11? This build configuration previously used to work with
3.9.6.
clang -Wno-unused-result -Wsign-compare -g -O0 -Wall -arch x86_64
-mmacosx-version-min=10.
Gregory Szorc added the comment:
> For stdlib modules it wouldn't be a big problem to set __file__ on
> frozen modules.
> Would that be enough to solve the problem?
What do you set __file__ to? Do you still materialize the .py[c] files on disk
and set to that? (This would make the m
Gregory Szorc added the comment:
My use case for these low-level APIs is to write tests for low-level
string/encoding handling in my custom use of the PyPreConfig and PyConfig
structs. I wanted to verify that exact byte sequences were turned into specific
representations inside of Python
Gregory Szorc added the comment:
Oh, PyOxidizer also ran into more general issues with the frozen importer in
that it broke various importlib APIs. e.g. because the frozen importer only
supports bytecode, you can't use .__loader__.get_source() to obtain the source
of a module. This makes
Gregory Szorc added the comment:
When I investigated freezing the standard library for PyOxidizer, I ran into a
rash of problems. The frozen importer doesn't behave like PathFinder. It
doesn't (didn't?) set some common module-level attributes that are documented
by the importer
Change by Gregory Szorc :
--
title: Reliance on C bit fields is C API is undefined behavior -> Reliance on C
bit fields in C API is undefined behavior
___
Python tracker
<https://bugs.python.org/issu
New submission from Gregory Szorc :
At least the PyASCIIObject struct in Include/cpython/unicodeobject.h uses bit
fields. Various preprocessor macros like PyUnicode_IS_ASCII() and
PyUnicode_KIND() access this struct's bit field.
This is problematic because according to the C specification
Gregory Szorc added the comment:
I think this was effectively fixed in 3.10 via commit
39258d3595300bc7b952854c915f63ae2d4b9c3e / bpo-43669, which removed code
creating the duplicate symbols. It is still a problem in 3.9. But possibly not
worth the backport
Gregory Szorc added the comment:
I ran into a minor issue with the ctypes extension and the
_dyld_shared_cache_contains_path dependency that was added as part of this work
(commit e8b1c038b14b5fc8120aab62c9bf5fb840274cb6).
When building on Intel macOS 10.15 using an 11.x SDK, the configure
Gregory Szorc added the comment:
Who uses this syntax?
https://github.com/search?l=Python=%22from+.__init__+import%22=Code says
a lot of random code, surprisingly/sadly.
As for python-ideas, thanks for the suggestion: I may very well craft an email
Gregory Szorc added the comment:
I worked around this in PyOxidizer by stripping a trailing `.__init__` from
module names when resolving the indexed resource data. This allows the import
to work since it can find the data now, but it also preserves the double module
object, which isn't
New submission from Gregory Szorc :
(Rereporting from https://github.com/indygreg/PyOxidizer/issues/317.)
$ mkdir foo
$ cat > foo/__init__.py < test = True
> EOF
$ cat > foo/bar.py < from .__init__ import test
> EOF
$ python3.9
Python 3.9.0 (default, Nov 1 2020, 22:
Gregory Szorc added the comment:
My workaround was to remove `|| defined(LIBRESSL_VERSION_NUMBER)`. That works
with LibreSSL 3.1.4. I'm unsure if the symbols were introduced in a specific
version though. But since python-build-standalone pins all dependencies, this
is a safe patch for me
Gregory Szorc added the comment:
I was going to work around this in python-build-standalone
(https://github.com/indygreg/python-build-standalone) by reverting to OpenSSL
for builds. But we must use LibreSSL when linking against musl libc due to some
random weirdness getting OpenSSL
Change by Gregory Szorc :
--
nosy: +christian.heimes
___
Python tracker
<https://bugs.python.org/issue41949>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Gregory Szorc :
--
type: -> compile error
___
Python tracker
<https://bugs.python.org/issue41949>
___
___
Python-bugs-list mailing list
Unsubscrib
New submission from Gregory Szorc :
Commit 54f2898fe7e4ca1f239e96284af3cc5b34d2ae02 (bpo-40645) introduced the
functions HMAC_CTX_new, HMAC_CTX_free, and HMAC_CTX_get_md.
These functions share the same names as HMAC functions provided by
OpenSSL/LibreSSL. If you attempt to statically link
New submission from Gregory Szorc :
On Windows, `sysconfig.get_path('include')` returns `Include` (capital I).
However, the actual installation path is `include` (lowercase i).
I believe the reason for the case mismatch is in this code in PC/layout/main.py:
```
if ns.include_dev
Change by Gregory Szorc :
--
nosy: +indygreg
nosy_count: 7.0 -> 8.0
pull_requests: +19218
pull_request: https://github.com/python/cpython/pull/19746
___
Python tracker
<https://bugs.python.org/issu
Gregory Szorc added the comment:
Oh, I just went to patch this and it is a dupe of 40294, which has already been
fixed and backported.
--
resolution: -> duplicate
stage: -> resolved
status: open -> closed
___
Python tracke
Change by Gregory Szorc :
--
nosy: +vstinner
___
Python tracker
<https://bugs.python.org/issue40415>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Gregory Szorc :
Most of CPython's extensions can be initialized and freed multiple times in the
same process. However, _asyncio crashes on at least CPython 3.8.2 when this is
done.
STR:
1. Create a new Python interpreter
2. Have it import _asyncio
3. Finalize
Gregory Szorc added the comment:
Actually, I can reproduce the crash without Py_RunMain(). So I don't think the
crash is related to the additional cleanup that Py_RunMain() does in addition
to Py_FinalizeEx().
But I'd like to keep this issue open to track the original question about
New submission from Gregory Szorc :
I'm attempting to perform the following actions multiple times in a single
process with CPython 3.8.2:
1) Initialize an interpreter using the PEP-587 APIs.
2) Call Py_RunMain() (which finalizes the interpreter).
However, I've encountered at least 2 crashes
Change by Gregory Szorc :
--
keywords: +patch
pull_requests: +19068
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/19746
___
Python tracker
<https://bugs.python.org/issu
New submission from Gregory Szorc :
Filing a bug to placate the requirement that pull requests have issues.
--
components: C API
messages: 367477
nosy: indygreg
priority: normal
severity: normal
status: open
title: inittab_copy not set to NULL after free, can lead to crashes when
Gregory Szorc added the comment:
Having approached this with a fresh brain, I was able to port PyOxidizer to use
the multi-phase initialization API with minimal regressions!
The relevant code exists at
https://github.com/indygreg/PyOxidizer/blob/b5aa2b3a96dbd01e9d78857e124f1052f42f86c6
New submission from Gregory Szorc :
I'm porting PyOxidizer to the PEP 587 APIs. So far, it is mostly
straightforward.
I was looking forward to adopting the multi-phase initialization API because I
thought it would enable me to get rid of the very ugly hack PyOxidizer uses to
inject its
Change by Gregory Szorc :
--
type: -> enhancement
___
Python tracker
<https://bugs.python.org/issue40333>
___
___
Python-bugs-list mailing list
Unsubscrib
Gregory Szorc added the comment:
I don't like utilizing the dynamic archive links like
https://github.com/python/cpython-source-deps/archive/libffi.zip (even if you
pin the commit) because GitHub does not guarantee the file content is
deterministic over time. I perform SHA-256 validation
New submission from Gregory Szorc :
The https://github.com/python/cpython-source-deps project is missing a source
archive release for commits to libffi needed to support building on Windows.
The latest release of libffi is version libffi-3.3.0-rc0-r1, which corresponds
to
https://github.com
Gregory Szorc added the comment:
I just noticed that there is a parallel discussion ongoing in
https://gitlab.com/python-devs/importlib_resources/issues/58.
--
___
Python tracker
<https://bugs.python.org/issue36
Gregory Szorc added the comment:
I think disallowing relative paths that are parents of the current anchor point
is a reasonable restriction and acceptable backwards incompatible behavior.
Disallowing all relative paths with slashes is a larger issue. I would support
that if designing
New submission from Gregory Szorc :
As I was attempting to implement the find_distributions() interface for
PyOxidizer, I got confused by importlib.metadata's documentation.
The documentation for this module states:
```
What this means in practice is that to support finding distribution
Gregory Szorc added the comment:
I ran into this with PyOxidizer. CPython's `configure` needs to be made aware
of the current macOS SDK version and filter out symbols not present in the
target macOS version. I worked around it by `undef`'ing some entries from the
generated `pyconfig.h` file
Gregory Szorc added the comment:
I'm a bit busy with other things this week to submit a PR.
--
___
Python tracker
<https://bugs.python.org/issue37459>
___
___
New submission from Gregory Szorc :
The documentation in importlib.rst says that a loader should implement
`get_resource_loader(fullname)`. This is the only occurrence of
`get_resource_loader` in the CPython source tree. It should be changed to
`get_resource_reader()`.
--
components
Gregory Szorc added the comment:
FWIW LICENSE.txt on Windows contains licenses for additional projects starting
on line 258 (at least in CPython 3.7.3), which contains the section header
"Additional Conditions for this Windows binary build".
But, the Doc/pythonXXX.chm does inde
New submission from Gregory Szorc :
The Modules/_ctypes/libffi_msvc/ffi.h and
Modules/_ctypes/libffi_osx/include/ffi.h files in the CPython repository
contain copies of libffi.
On at least the official Windows distributions, the LICENSE.txt does not
contain libffi's license text from
New submission from Gregory Szorc :
ctypes/__init__.py calls _ctypes.dlopen(None) on Linux as part of code
execution during module import. Unfortunately, dlopen() doesn't work if the
current executable isn't a dynamic executable.
Using a fully statically linked Python executable:
$ ldd
Gregory Szorc added the comment:
It's also worth noting that if the wrapped stream close() cascading behavior
should be configurable, then many additional types in the standard library need
a constructor argument to control this behavior. e.g. io.TextIOWrapper,
io.BufferedReader
Gregory Szorc added the comment:
Thank you for the detailed reply, Josh.
I generally agree with what you are saying. However, I have some follow-ups.
In your answer to #2, you seem to distinguish between an "fd" (implying POSIX
file descriptor) and "Python layers" (im
New submission from Gregory Szorc :
As part of implementing io.RawIOBase/io.BufferedIOBase compatible stream types
for python-zstandard, I became confused about the expected behavior of flush()
and close() when a stream is wrapping another stream.
The documentation doesn't lay out explicitly
New submission from Gregory Szorc :
The implementation of the ResourceReader API for the FileLoader class in
importlib/_bootstrap_external.py is inconsistent with regards to handling of
path separators.
Specifically, "is_resource()" returns False if "resource"
New submission from Gregory Szorc :
The _asynciomodule.c source file is compiled as part of both pythoncore.vcxproj
(providing pythonXY.dll) and _asyncio.vcxproj (providing _asyncio.pyd).
PC\config.c doesn't reference PyInit__asyncio. I'm inclined to believe that
_asynciomodule.c being built
Gregory Szorc <gregory.sz...@gmail.com> added the comment:
I was going to update some CI that tests all supported versions of Python and
pins the Python 3 versions. That CI was previously using the MSI installers for
Python 3.4. To my surprise, the MSIs stopped being generated after the
Gregory Szorc added the comment:
IMO I don't find the 3.2 docs more useful. Specifically, the behavior for
memoryview is still unclear.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
New submission from Gregory Szorc:
https://github.com/indygreg/python-zstandard/issues/26 is a bug report against
python-zstandard that a C API using the "y#" format unit is unable to accept a
memoryview instance constructed from a bytes literal. It fails with "TypeError:
Gregory Szorc added the comment:
For posterity, I think we ran into a similar problem in
https://bugzilla.mozilla.org/show_bug.cgi?id=1191877, where our code uses
apply_async():
11:09:47 INFO - Exception in thread Thread-2:
11:09:47 INFO - Traceback (most recent call last):
11:09:47
Gregory Szorc added the comment:
Thanks, Steve.
The package I was trying to build has its setup.py importing setup from
distutils, not setuptools. I'll see about porting them to the future.
For posterity, https://bitbucket.org/pypa/setuptools/issue/258, which was first
released in setuptools
Gregory Szorc added the comment:
The first sentence in my original report is ambiguous. I tested with distutils
on Python 2.7.9. But considering the code in question hasn't changed since
2011, I think it's safe to say this isn't a regression in CPython
New submission from Gregory Szorc:
distutils as of Python 2.7.9 is unable to locate vcvarsall.bat if Visual C++
Compiler for Python is the only Visual Studio distribution installed.
STR:
1) Do a fresh install of Windows + all updates
2) Install Microsoft Visual C++ Compiler for Python from
66 matches
Mail list logo
