[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ned Deily]

Ned Deily  added the comment:

Thanks for the easy-to-reproduce test case! It looks like you are running into 
the problem described in Issue22490 where venv with macOS framework builds 
could run into problems. A fix for this issue has recently been merged and will 
be released in upcoming 3.8.3 (a 3.8.3 release candidate should be available in 
the very near future) and 3.7.8.  I've verified that your test case now works 
with the current HEAD of 3.8 and fails with 3.8.2.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs 
fragile

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

Ruben  added the comment:

I just tested on a fresh macOS 10.15.3 Virtual Machine (on the same host 
machine as the one of previous messages) and I can reproduce the problem: 

1. Install Command Line Tools (xcode-build --install)
2. Download Python 3.8.1
3. Extract tarball
4. Create a file make.py with the following python code
```
import subprocess

subprocess.run(["./configure"])
subprocess.run(["make", "-j4"])
```
and save it in the Python-3.8.1 source tree
6. Create a venv doing `python -m venv venv` in the source tree
7. Activate the virtual env `source venv/bin/activate`
5. Run it doing `python3 make.py`

The build fails with the error described before. 

If you now, instead, run the make directly `make -j8` the buids proceeds as 
expected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

Ruben  added the comment:

Sorry, the error was missing, well, the actual error: 

```
/opt/local/bin/ranlib: file: libpython3.8.a(dynamic_annotations.o) has no 
symbols
/opt/local/bin/ranlib: file: libpython3.8.a(pymath.o) has no symbols
gcc -Wl,-stack_size,100  -framework CoreFoundation -o python.exe 
Programs/python.o libpython3.8.a -ldl   -framework CoreFoundation
gcc -Wl,-stack_size,100  -framework CoreFoundation -o 
Programs/_testembed Programs/_testembed.o libpython3.8.a -ldl   -framework 
CoreFoundation
./python.exe -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
exit 1 ; \
fi
 CC='gcc' LDSHARED='gcc -bundle -undefined dynamic_lookup' OPT='-DNDEBUG -g 
-fwrapv -O3 -Wall'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''   ./python.exe -E 
./setup.py  build
running build
running build_ext
error: [Errno 2] No such file or directory: 
'/usr/local/include/python3.8/pyconfig.h'
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

Change by Ruben :


Added file: https://bugs.python.org/file49053/make.log.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

Change by Ruben :


Added file: https://bugs.python.org/file49054/make.venv.log.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

Change by Ruben :


Added file: https://bugs.python.org/file49052/make.venv.log.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

Change by Ruben :


Added file: https://bugs.python.org/file49051/make.py.venv2.log.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40261] Build of Python where make is called from subprocess, within a virtualenv, breaks on macOS

[Ruben]

New submission from Ruben :

This problem is driving me crazy. I'm working with a library 
(python-for-android) that builds python during a build process. 

Now, during this process, the classical workflow with `./configure` and `make` 
is being called from a subprocess. To try to isolate the issue, I call `make` 
to build Python from a file, let's call it `make.py`, in the source tree, where 
I simply do (after running `./configure`, manually or from a subprocess):

```
import subprocess; 
subprocess.run(["make", "-j8"])
```

I get the following error: 

```
/opt/local/bin/ranlib: file: libpython3.8.a(dynamic_annotations.o) has no 
symbols
/opt/local/bin/ranlib: file: libpython3.8.a(pymath.o) has no symbols
gcc -Wl,-stack_size,100  -framework CoreFoundation -o python.exe 
Programs/python.o libpython3.8.a -ldl   -framework CoreFoundation
gcc -Wl,-stack_size,100  -framework CoreFoundation -o 
Programs/_testembed Programs/_testembed.o libpython3.8.a -ldl   -framework 
CoreFoundation
./python.exe -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
exit 1 ; \
fi
 CC='gcc' LDSHARED='gcc -bundle -undefined dynamic_lookup' OPT='-DNDEBUG -g 
-fwrapv -O3 -Wall'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''   ./python.exe -E 
./setup.py  build
```

Is this a bug of my OS, my env configuration, venv/virtualenv, Python makefile 
or, most probably, myself? 

I attached the log files from the following runs: 
- make from the shell without virtualenv (`make.log`)
- make from shell within virtualenv (`make.venv.log`)
- make from python subprocess without virtualenv (`make.py.log`)
- make from python subprocess within a virtualenv creted with pew 
(`make.py.venv.log)
- make from python subprocess within a virtualenv created with venv 
(`make.py.venv2.log`)

My system details:

macOS 10.15.4

Steps to reproduce: 

1. Download Python-3.8.2
2. Extract the tgz
3. Run ./configure
4. cat `make.py` in the source tree where `make.py` is:
```
import subprocess

subprocess.run(["env"])
subprocess.run(["make", "-j8"])
```
5. Run python make.py

--
components: Installation, macOS
files: make.py.venv.log.gz
messages: 366232
nosy: ned.deily, rdbisme, ronaldoussoren
priority: normal
severity: normal
status: open
title: Build of Python where make is called from subprocess, within a 
virtualenv, breaks on macOS
type: compile error
versions: Python 3.8
Added file: https://bugs.python.org/file49050/make.py.venv.log.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com