[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Thanks Mitchell for looking into this. The proposed workaround looks
good to me.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
Invalid
Status in python2.7 source package in Bionic:
Won't Fix
Status in python2.7 source package in Focal:
In Progress
Status in python2.7 source package in Jammy:
In Progress
Status in python2.7 source package in Kinetic:
Invalid
Status in python2.7 source package in Lunar:
Invalid
Status in python2.7 source package in Mantic:
Invalid
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
I just want to check if there are any updates with this bug?
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
In Progress
Status in python2.7 source package in Bionic:
Won't Fix
Status in python2.7 source package in Focal:
Won't Fix
Status in python2.7 source package in Jammy:
Won't Fix
Status in python2.7 source package in Kinetic:
Won't Fix
Status in python2.7 source package in Lunar:
Won't Fix
Status in python2.7 source package in Mantic:
Won't Fix
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Hi Mitchell, Do you have any update for this bug?
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
In Progress
Status in python2.7 source package in Bionic:
Won't Fix
Status in python2.7 source package in Focal:
Won't Fix
Status in python2.7 source package in Jammy:
Won't Fix
Status in python2.7 source package in Kinetic:
Won't Fix
Status in python2.7 source package in Lunar:
Won't Fix
Status in python2.7 source package in Mantic:
Won't Fix
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Thanks for providing the workarounds. While the workarounds work in cases when
user is willing to compile extensions from the python package sources and aware
of these problems, for the most common cases work arounds are not feasible.
There are 100’s of python packages with extension modules and the most common
use case is to install them using pip. The general expectation from end user is
to have them compile with right flags and work without any extra work. So it’s
going to be very challenging for us to communicate end users about this
potential problem which only occurs with Python 2 on Ubuntu distributions.
I’m also not clear about the regression problem mentioned above. Why would we
expect regressions if the patch itself is trying to fix the bug and use right
optimizations (in this case O2)?
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
In Progress
Status in python2.7 source package in Bionic:
Won't Fix
Status in python2.7 source package in Focal:
Won't Fix
Status in python2.7 source package in Jammy:
Won't Fix
Status in python2.7 source package in Kinetic:
Won't Fix
Status in python2.7 source package in Lunar:
Won't Fix
Status in python2.7 source package in Mantic:
Won't Fix
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Thanks for the update Fabio Martins !
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
In Progress
Status in python2.7 source package in Bionic:
In Progress
Status in python2.7 source package in Focal:
In Progress
Status in python2.7 source package in Jammy:
In Progress
Status in python2.7 source package in Kinetic:
In Progress
Status in python2.7 source package in Lunar:
In Progress
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Hi Ghadi Rahme,
I see this ticket is in progress, just want to check when the fix will be
updated in Ubuntu's packages?
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
In Progress
Status in python2.7 source package in Bionic:
In Progress
Status in python2.7 source package in Focal:
In Progress
Status in python2.7 source package in Jammy:
In Progress
Status in python2.7 source package in Kinetic:
In Progress
Status in python2.7 source package in Lunar:
In Progress
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Thanks for looking into this. On python3, can we make sure if we are
looking into right output file?
>From your pasted output on Python3, I do see, there is -O2 but I see
from next command the query is against test.so file which seems
incorrect. From the compilation output, the file name when python3 is
used is test.cpython-38-aarch64-linux-gnu.so.
On my machine (Ubuntu 20.04) with Python3:-
ubuntu@ip-172-31-18-56:~/test$ python3 setup.py build_ext --inplace
running build_ext
building 'test' extension
creating build
creating build/temp.linux-aarch64-3.8
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g
-fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security
-g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security
-Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.8 -c testmodule.c
-o build/temp.linux-aarch64-3.8/testmodule.o
creating build/lib.linux-aarch64-3.8
aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
-Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions
-Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2
build/temp.linux-aarch64-3.8/testmodule.o -o
build/lib.linux-aarch64-3.8/test.cpython-38-aarch64-linux-gnu.so
copying build/lib.linux-aarch64-3.8/test.cpython-38-aarch64-linux-gnu.so ->
ubuntu@ip-172-31-18-56:~/test$ strings test.cpython-38-aarch64-linux-gnu.so |
grep -- -O
GNU C17 9.4.0 -mlittle-endian -mabi=lp64 -g -g -g -g -O2 -O2
-fstack-protector-strong -fwrapv -fstack-protector-strong -fPIC
-fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
New
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is
[Touch-packages] [Bug 2002043] Re: Python extension modules get built using wrong compiler flags with python2
Hi Fabio Martins,
Thanks for looking into this issue. Attached tar file contains setup.py and
steps to reproduce the issue. In general, the behavior is same with any package
building with extensions.
** Attachment added: "testprog.tar.gz"
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5640852/+files/testprog.tar.gz
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
New
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch
there as well. Patch used on python3, is not modifying the cflags completely
and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2002043] [NEW] Python extension modules get built using wrong compiler flags with python2
Public bug reported:
Compiling a Python extension using Python2 (Python 2.7.18) is making use
of wrong compiler flags, hence dropping required optimizations when
required. This is happening only when python2 is installed from Ubuntu's
repositories. By default, Python's distutils module uses compiler and
linker flags used to compile Python itself to be used to compile
extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on
libpython2.7-stdlib package is altering the original upstream implementation of
distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch -
http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of
configure_cflags is modifying cflags. This code is result of ubuntu's patch and
doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags,
extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch there
as well. Patch used on python3, is not modifying the cflags completely and
instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on
python3.
** Affects: python2.7 (Ubuntu)
Importance: Undecided
Status: New
** Summary changed:
- Python extension modules get build using wrong compiler flags with python2
+ Python extension modules get built using wrong compiler flags with python2
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/2002043
Title:
Python extension modules get built using wrong compiler flags with
python2
Status in python2.7 package in Ubuntu:
New
Bug description:
Compiling a Python extension using Python2 (Python 2.7.18) is making
use of wrong compiler flags, hence dropping required optimizations
when required. This is happening only when python2 is installed from
Ubuntu's repositories. By default, Python's distutils module uses
compiler and linker flags used to compile Python itself to be used to
compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig
module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-Wdate-time -D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation
flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time
-D_FORTIFY_SOURCE=2 -g
-fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=.
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c testmodule.c -o
