[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This is fixed in bpo-41100

--
dependencies:  -Build failure on macOS 11 (beta)
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
superseder:  -> Build failure on macOS 11 (beta)
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The active PR in bpo-41100 will fix this issue as well.

--
dependencies: +Build failure on macOS 11 (beta)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Zorg]

Zorg  added the comment:

I understand that. I was mainly responding to a question that was asked with 
information I felt was important to provide.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ronald Oussoren]

Ronald Oussoren  added the comment:

@Zorg: you don't need to sell the usefulness of this functionality. Its just 
that the python developers are volunteers and hence have limited time to work 
on python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Zorg]

Zorg  added the comment:

Also more obviously: it's less convenient for others keeping themselves up to 
date to set up an older development environment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Zorg]

Zorg  added the comment:

> Why do you believe that is safer?  Apple goes to great lengths to provide 
> compatibility for existing applications to keep running on newer systems. 

They also don't want developers to develop using older SDKs. Hypothetically 
speaking, more information may be stored in the binaries, and there could be 
something down the road to opt into that requires all binaries to be linked 
against a certain SDK. Or your code takes an older path based on the SDK 
version at runtime -- python may include some other higher level baggage too, 
like the tkinter stuff for instance.

There is some middle ground other than weak linking; stripping out features 
from the configure part of the project and opt in to using the deprecated APIs. 
Not as ideal as weak linking, but better than building on an older SDK IMO.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Christoph Reiter]

Change by Christoph Reiter :


--
nosy:  -lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Linking with newer SDKs is needed to get access to some newer features and 
APIs. That's less relevant for Python itself, but can be relevant for the 
embedding application.  You can still compile Python itself on an older 
machine, then link it into an application build against a newer SDK. This does 
make the build process harder though.

However, there seem to be some issues when you mix code linked for different 
deployment targets, for example 
.
  This is issue contains a C extension that works correctly when CPython is 
build with a new SDK and crashed hard with the interpreters on Python.org.

This is one reason why I will at some time in the future look into this, but I 
cannot say when that will happen because I have limited time to work on this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ned Deily]

Ned Deily  added the comment:

> https://bugs.python.org/issue31359 has a better details about this issue.

Yea, the approach there is again dealing with using a newer SDK to build for an 
older system.

> I personally believe it's safer to link against a newer SDK and deal with the 
> trouble of finding out what to strip out, which will be opted into 
> implications or code paths that linking a binary against a newer SDK may 
> provide.

Why do you believe that is safer?  Apple goes to great lengths to provide 
compatibility for existing applications to keep running on newer systems.  
That's not to say that eventually certain APIs are deprecated and removed but 
Python itself makes fairly minimal demands on macOS, i.e. it pretty much does 
not use anything other than basic libc-style system calls.  The one obvious 
disadvantage is that Python apps can't take advantage of newly introduced 
system calls (introduced after 10.m) when running on those newer systems.  I 
honestly don't recall any case dating back to 10.5 or so where there have been 
any reported problems with running a 10.m Python on a 10.n system (as best as I 
can recall!).  On the other hand, trying to support weak linking and backward 
compatibility requires more code and significant increase in ongoing testing.  
It would be a good thing to have but there is a significant cost and added risk 
to do so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Zorg]

Zorg  added the comment:

https://bugs.python.org/issue31359 has a better details about this issue.

I personally believe it's safer to link against a newer SDK and deal with the 
trouble of finding out what to strip out, which will be opted into implications 
or code paths that linking a binary against a newer SDK may provide.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ned Deily]

Ned Deily  added the comment:

While, as Ronald notes, it would be nice to fully support weak linking so that 
building Python on macOS 10.n with a deployment target for 10.m where m < n so 
that an application with an embedded or bundled Python can run on 10.m through 
10.n systems, in the mean time there is a simple and safe alternative that has 
been used for years: build the Python on 10.m (or use a ready-built Python 
built on 10.m or lower).  That's what we do for the macOS Pythons provided by 
python.org installers, e.g. the installer variants that claim 10.9+ support are 
actually built on 10.9.  If you need an app bundle with an embedded Python, 
py2app can greatly simplify the process of constructing one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Binaries currently work on the system you compiled on and any later release of 
macOS.  That's the only "promise" there currently is.

Anything better requires work, and hence someone willing to do that work. 

I'd love to be able to build on the latest macOS and deploy to older versions, 
but that's functionality I don't need myself at the moment which means this 
does not have priority for me. 

BTW. I haven't checked yet how invasive supporting weak linking for these 
symbols will be. That depends on where the symbols are used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Christoph Reiter]

Christoph Reiter  added the comment:

related: issue31359

--
nosy: +lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Zorg]

Zorg  added the comment:

The minimum OS target (which looks like 10.13) should ideally be well defined 
and tested using the latest SDK, rather than there being "no promises". Or at 
least it should be documented that this isn't currently supported, or how to 
disable specific features and which ones to disable. I downloaded the source 
tarball from https://www.python.org/downloads/release/python-370/

Python then isn't viable currently for embedding inside applications 
(https://docs.python.org/3/extending/embedding.html). 3rd party app developers 
on macOS typically have a policy at least supporting back to N-1 or N-2'th OS, 
and in many cases it shouldn't be their dependencies that drive this. 

The other two functions require 10.13:
futimens
utimensat

Fortunately the -Wunguarded-availability warning is enabled which spits out 
warnings on which functions aren't available when targeting a min OS 
(MACOSX_DEPLOYMENT_TARGET)

Can't say I can contribute anything here unfortunately :(. I had to do 
something though, so I hacked the configure scripts and pyconfig.h to make sure 
the clock_* and the above functions I mentioned above weren't being used. 
getentropy was easy to fix more properly because it already has a fallback at 
runtime in case it's not working, so it was just an additional NULL check. 
Looks like the functions are weakly referenced when building to an older OS 
target.

A bad step in future direction would be to continue making "no promises" and 
start using functionality only available on 10.14, 10.15, etc without fallback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The primary Python.org installer supports 10.9 and later by way of being 
compiled on 10.9, and there is also an installer that supports 10.6. 

There currently is no promise that building on 10.13 results in code that works 
on earlier versions of macOS.

That said, I agree that building on later macOS releases and deploying to older 
ones is useful, and this is something that I used to do in the past (I 
currently have the good fortune to only need to deploy to macOS 10.13). 

It is possible to add runtime checks for APIs, that's something I've done in 
the past to for the same reason, and that code is still present (for example in 
posixmodule.c).

Is this something you can and want to work on?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34597] Python needs to check existence of functions at runtime for targeting older macOS platforms

[Zorg]

New submission from Zorg :

If one wants to compile Python and embed it in their applications, or more 
generally, deploy the framework / libraries to older systems, then Python needs 
to check the existence of functions at runtime that are unavailable to Python's 
supported minimum OS.

For example, Python makes use of:
clock_gettime
getentropy

These functions are only available on macOS 10.12. Python currently states it 
supports 10.9. Checking for their existence at compile time isn't good enough; 
compiling Python on 10.13 and deploying back to 10.9 or 10.11 for example will 
not work, and cause a crash at runtime. One can make sure the functions are 
weakly linked and check they aren't NULL to get around this.

There may be more such functions Python is using; I haven't extensively 
searched for all usages.

Alternatively, Python could state that it does not properly support embedding 
(probably not desirable), or Python could bump the minimum supported OS 
requirements in its documentation.

Else one might be able to compile Python by turning off such features? But the 
user needs to know what those are, Python gives no warnings when 
MACOSX_DEPLOYMENT_TARGET/--mmacosx-version-min are low enough, and Python 
wouldn't be able to use those functions when they are actually available on 
newer systems.

--
components: macOS
messages: 324698
nosy: Zorg, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python needs to check existence of functions at runtime for targeting 
older macOS platforms
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com