[issue39432] Distutils generates the wrong export symbol for unicode module names

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[da-woods]

da-woods  added the comment:

It looks like this wasn't quite fixed by the patch: the patch encoded 
`_` when it should have encoded ``.

I've submitted a revised version to setuptools 
https://github.com/pypa/setuptools/pull/2646. My impression is that distutils 
is no longer updated and so there's no value in submitting this patch to Python 
too. However, I can do so if it would be used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[miss-islington]

Change by miss-islington :


--
pull_requests: +17928
pull_request: https://github.com/python/cpython/pull/18548

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Change by Stefan Behnel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Stefan Behnel  added the comment:


New changeset 5bf58cef151249f1cca92166d1b70693348da9d8 by Miss Islington (bot) 
in branch '3.8':
bpo-39432: Implement PEP-489 algorithm for non-ascii "PyInit_*" symbol names in 
distutils (GH-18150) (GH-18546)
https://github.com/python/cpython/commit/5bf58cef151249f1cca92166d1b70693348da9d8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[miss-islington]

Change by miss-islington :


--
pull_requests: +17924
stage: backport needed -> patch review
pull_request: https://github.com/python/cpython/pull/18546

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Petr Viktorin]

Petr Viktorin  added the comment:

I'm with Stefan on "Definitely 3.8". It's a bug fix (for a rarely used feature).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Ned Deily]

Ned Deily  added the comment:

We should not be changing Distutils 3.7 behavior at this late point in its life 
cycle, particularly since AFAIK this issue has not come up before.  Let's see 
what Łukasz thinks for 3.8.

--
nosy: +lukasz.langa
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Steve Dower]

Steve Dower  added the comment:

issue39555 and PR 18357 have the fix for the buildbot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[STINNER Victor]

STINNER Victor  added the comment:

On Windows, names get a "_d" suffix for debug. Extract of build_ext.py:

def get_libraries(self, ext):
"""Return the list of libraries to link against when building a
shared extension.  On most platforms, this is just 'ext.libraries';
on Windows, we add the Python library (eg. python20.dll).
"""
# The python library is always needed on Windows.  For MSVC, this
# is redundant, since the library is mentioned in a pragma in
# pyconfig.h that MSVC groks.  The other Windows compilers all seem
# to need it mentioned explicitly, though, so that's what we do.
# Append '_d' to the python import library on debug builds.
if sys.platform == "win32":
from distutils._msvccompiler import MSVCCompiler
if not isinstance(self.compiler, MSVCCompiler):
template = "python%d%d"
if self.debug:
template = template + '_d'
(...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[STINNER Victor]

STINNER Victor  added the comment:

I put a breakpoint before the error:

test_unicode_module_names (distutils.tests.test_build_ext.BuildExtTestCase) ... 
> c:\vstinner\python\master\lib\distutils\tests
\test_build_ext.py(316)test_unicode_module_names()
-> self.assertRegex(cmd.get_ext_filename(modules[0].name), r'foo\..*')
(Pdb) p modules
[, 
]
(Pdb) p modules[0].name
'foo'
(Pdb) p modules[1].name 
'föö'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[STINNER Victor]

STINNER Victor  added the comment:

Python 3.6 doesn't accept bugfixes anymore:
https://devguide.python.org/#status-of-python-branches

The bugfix can go into 3.7 and 3.8.

--
versions:  -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[STINNER Victor]

STINNER Victor  added the comment:

The test fails on Windows. Example on AMD64 Windows8.1 Refleaks 3.x:
https://buildbot.python.org/all/#/builders/157/builds/76

==
FAIL: test_unicode_module_names 
(distutils.tests.test_build_ext.BuildExtTestCase)
--
Traceback (most recent call last):
  File 
"D:\buildarea\3.x.ware-win81-release.refleak\build\lib\distutils\tests\test_build_ext.py",
 line 315, in test_unicode_module_names
self.assertRegex(cmd.get_ext_filename(modules[0].name), r'foo\..*')
AssertionError: Regex didn't match: 'foo\\..*' not found in 
'foo_d.cp39-win_amd64.pyd'

==
FAIL: test_unicode_module_names 
(distutils.tests.test_build_ext.ParallelBuildExtTestCase)
--
Traceback (most recent call last):
  File 
"D:\buildarea\3.x.ware-win81-release.refleak\build\lib\distutils\tests\test_build_ext.py",
 line 315, in test_unicode_module_names
self.assertRegex(cmd.get_ext_filename(modules[0].name), r'foo\..*')
AssertionError: Regex didn't match: 'foo\\..*' not found in 
'foo_d.cp39-win_amd64.pyd'

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Stefan Behnel  added the comment:

Ok, this is merged into 3.9. To which versions should we backport it?
Definitely 3.8, definitely not 3.5, probably not 3.6 (since it's not a security 
issue). Ned, what about 3.7?

--
nosy: +ned.deily
stage: patch review -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[miss-islington]

miss-islington  added the comment:


New changeset 9538bc9185e934bee2bd5ae2cda2b2e92a61906d by Stefan Behnel in 
branch 'master':
bpo-39432: Implement PEP-489 algorithm for non-ascii "PyInit_*" symbol names in 
distutils (GH-18150)
https://github.com/python/cpython/commit/9538bc9185e934bee2bd5ae2cda2b2e92a61906d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Change by Stefan Behnel :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Change by Stefan Behnel :


--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Change by Stefan Behnel :


--
keywords: +patch
pull_requests: +17536
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/18150

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Change by Stefan Behnel :


--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[Stefan Behnel]

Change by Stefan Behnel :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39432] Distutils generates the wrong export symbol for unicode module names

[da-woods]

New submission from da-woods :

Distuitls generates "export symbols" for extension modules to help ensure that 
they have have the correct linkage on Windows.

https://github.com/python/cpython/blob/0d30ae1a03102de07758650af9243fd31211325a/Lib/distutils/command/build_ext.py#L692

It generates the correct symbol in most causes, but if the filename contains 
unicode characters then it creates the wrong symbol, causing linkage errors.

The behaviour should be updated to reflect PEP-489: 
https://www.python.org/dev/peps/pep-0489/#export-hook-name

--
components: Distutils
messages: 360555
nosy: da-woods, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: Distutils generates the wrong export symbol for unicode module names
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com