[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Ned Deily  added the comment:


New changeset c421c66a58a6caae30f0679d7e61411418e67cec by Ned Deily in branch 
'2.7':
bpo-36231: Support building on macOS without /usr/include (GH-13773) (GH-14256)
https://github.com/python/cpython/commit/c421c66a58a6caae30f0679d7e61411418e67cec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +14086
pull_request: https://github.com/python/cpython/pull/14256

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Change by Ned Deily :


--
versions: +Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Ned Deily  added the comment:

Thank you again for the suggested PR. Using "xcrun --show-sdk-path" at 
configure time *is* appealing.  Unfortunately, it does not cover all of the 
necessary use cases.

One, the --show-sdk-path option is not available on old versions of xcrun, 
versions we still build with to support older releases of macOS.  Second, some 
users build Python for macOS with compiler tool chains other than the 
Apple-supplied ones (for example, current gcc) that do not necessarily support 
the transparent selection of header and library files location via xcrun and 
friends.  Third, capturing the selected SDK path at configure time is no 
guarantee that the same SDK path will be used for extension module builds when 
setup.py runs.  With the Apple-supplied tools, the actual SDK path used is 
determined each time the compiler front-end is invoked and depends on the 
then-current selected values (e.g. the most recent value set by 'xcode-select 
--switch') and the current value of environment variables (e.g. like 
DEVELOPER_DIR and SDKROOT). Or a different or non-Apple compiler could now be 
in use by overriding CC.  In other words, lots of edge cases largely due to the 
inherent fle
 xibility of Apple's compiler frontend.

The other issue here is the behavior of setup.py in trying to make reasonable 
default choices for finding header and library files for the extension modules 
it is building; to do so, it tries to guess what the compiler frontend is going 
to do and that's the real hack.  Life would be much simpler if Python relied on 
a modern autotools build setup - although some of the same issues of dynamic 
SDK locations would still apply but at least they would apply consistently - 
and not the legacy mixture of build tools we have today.  Unfortunately, it 
would be a big deal to replace the current build system and, while it would be 
desirable, that's a very big project.

One other point: while scraping the output of the compiler is hacky, it is a 
well-known and widely-used technique and is already in use elsewhere in 
setup.py; this code was adapted from that in add_multiarch_paths().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Jeroen Demeyer]

Change by Jeroen Demeyer :


--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Dmitrii Pasechnik]

Dmitrii Pasechnik  added the comment:

I find it puzzling that a relatively clean and short solution using autotools 
and the Apple-approved way to get the location of the headers is rejected, and 
a hacky, longer way that scrapes output of the compiler is being pushed instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Ned Deily  added the comment:


New changeset c7302116573d853d3181133477d9d0e4d4d3abfd by Ned Deily in branch 
'3.7':
bpo-36231:  Support building on macOS without /usr/include (GH-13773) (GH-14208)
https://github.com/python/cpython/commit/c7302116573d853d3181133477d9d0e4d4d3abfd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +14046
pull_request: https://github.com/python/cpython/pull/14208

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Ned Deily  added the comment:


New changeset 0288dd6a5192074fcd5aa0db5d3513c3880209ca by Ned Deily in branch 
'master':
bpo-36231:  Support building on macOS without /usr/include (GH-13773)
https://github.com/python/cpython/commit/0288dd6a5192074fcd5aa0db5d3513c3880209ca


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Change by Ned Deily :


--
pull_requests: +13658
pull_request: https://github.com/python/cpython/pull/13773

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Dmitrii Pasechnik]

Dmitrii Pasechnik  added the comment:

In case,I have opened PR https://github.com/python/cpython/pull/12825
to provide our solution to this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Dmitrii Pasechnik]

Change by Dmitrii Pasechnik :


--
pull_requests: +12750
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report and for the PR offer but let's hold off on that for the 
moment: I'm planning to merge a somewhat different approach.

--
assignee:  -> ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Dmitrii Pasechnik]

Dmitrii Pasechnik  added the comment:

I won't mind to provide a PR for this. but it is not clear what the goal should 
be. Is it to build a working OSX Python with as few external to Xcode deps (it 
seems that only lzma is needed) as possible?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Erik Bray]

Erik Bray  added the comment:

Perhaps it would be better if the `xcrun --show-sdk-path` thing were run at 
configure-time and its result shoved into a variable we can read with 
sysconfig.get_config_var()

--
nosy: +erik.bray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Dmitrii Pasechnik]

Dmitrii Pasechnik  added the comment:

Needless to say, subprocess is most certainly an overkill, something less 
involved would do the job, without the need for all the module dependencies of 
subprocess.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36231] no "proper" header files on macOS 10.14 Mojave

[Dmitrii Pasechnik]

New submission from Dmitrii Pasechnik :

Neither Xcode nor its command-line tools on macOS 10.14 Mojave come with header 
files installed in /usr/ and other "normal" directories.

This is not documented in https://devguide.python.org/setup/#macos-and-os-x

While an extra step to handle this, i.e. to install the headers, is available 
(see a discussion on https://bugs.python.org/issue34956), Apple stated that 
this workaround will disappear.

It is thus highly desirable to provide a way to deal with headers located not 
at /usr/include, but at `xcrun --show-sdk-path`/usr/include.
A small change in setup.py along the lines of the following:

--- a/setup.py
+++ b/setup.py
@@ -134,7 +134,8 @@ def macosx_sdk_root():
 cflags = sysconfig.get_config_var('CFLAGS')
 m = re.search(r'-isysroot\s+(\S+)', cflags)
 if m is None:
-sysroot = '/'
+import subprocess
+sysroot = subprocess.check_output(["xcrun", 
"--show-sdk-path"]).decode("utf-8").strip('\n')
 else:
 sysroot = m.group(1)
 return sysroot
@@ -146,6 +147,7 @@ def is_macosx_sdk_path(path):
 """
 return ( (path.startswith('/usr/') and not path.startswith('/usr/local'))
 or path.startswith('/System/')
+or path.startswith('/Applications/')
 or path.startswith('/Library/') )

with the necessary changes to enable various modules (see attachment for a 
complete POC diff against the current master), the result builds and passes all 
the (enabled) tests on an OSX 10.13 system with empty /usr/include and 
/usr/local/include containing only LZMA headers.

Needless to say, a proper patch would not modify Modules/Setup, it'd adjust 
configure.ac etc.

--
components: macOS
files: noincludedirs_OSX.patch
keywords: patch
messages: 337461
nosy: dimpase, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: no "proper" header files on macOS 10.14 Mojave
type: compile error
versions: Python 3.8
Added file: https://bugs.python.org/file48192/noincludedirs_OSX.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com