[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2011-09-29 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: The PEP 393 has been accepted. There is no more narrow or wide build, Py_UNICODE is now always wchar_t. We have also a stable ABI which doesn't depend on the internal stucture of Unicode strings. In Python 3.2, the name of

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2011-09-29 Thread Daniel Stutzbach
Daniel Stutzbach stutzb...@google.com added the comment: Gladly. :-) -- resolution: - out of date stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-12-28 Thread Éric Araujo
Changes by Éric Araujo mer...@netwok.org: -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___ ___ Python-bugs-list

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-12-07 Thread Alexander Belopolsky
Changes by Alexander Belopolsky belopol...@users.sourceforge.net: -- nosy: +belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___ ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-09-03 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: I just noticed that the previous new patch I uploaded was actually my original patch. Oops. Anyway... In a discussion on the capi list, a few people objected to compiling in Unicode-agnostic mode by default (e.g., it would

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-09-03 Thread Guido van Rossum
Changes by Guido van Rossum gu...@python.org: -- nosy: -gvanrossum ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___ ___ Python-bugs-list

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-09-03 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: +#undef HAVE_USABLE_CHAR_T Isn't it HAVE_USABLE_WCHAR_T? +struct PY_UNICODE_STRUCT; +typedef struct PY_UNICODE_STRUCT Py_UNICODE_STRUCT; +#define Py_UNICODE Py_UNICODE_STRUCT Why not simply `typedef struct PY_UNICODE_STRUCT Py_UNICODE`? The

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-09-03 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: Thanks, Antoine. Those are all good suggestions. Attached is an updated patch. Marc-Andre, Martin: would you like to look this over as well? -- Added file: http://bugs.python.org/file18730/unicode-3.patch

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: Daniel Stutzbach wrote: Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 11:35 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: One of the more important cases you are missing is the

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 8:07 AM, Martin v. Löwis rep...@bugs.python.orgwrote: 1. add a flag to PyModuleDef, indicating whether the module was built in UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module,

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sun, May 9, 2010 at 10:12 AM, Marc-Andre Lemburg rep...@bugs.python.orgwrote: I'm just not sure how you could check for optimization compiler bugs/features using the buildbots. Once I have a patch that I'm modestly

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Daniel Stutzbach
Changes by Daniel Stutzbach dan...@stutzbachenterprises.com: Removed file: http://bugs.python.org/file17273/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Daniel Stutzbach
Changes by Daniel Stutzbach dan...@stutzbachenterprises.com: Removed file: http://bugs.python.org/file17274/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: I just thought of another risk inherit in this approach. If the extension module is composed of more than one C file, the extension author may inadvertently compile the file defining the PyModuleDef in Unicode-agnostic mode but compile

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Daniel Stutzbach
Changes by Daniel Stutzbach dan...@stutzbachenterprises.com: Removed file: http://bugs.python.org/file17248/unicode.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-09 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: Here is a new patch. Usage, short version: By default, modules compile in Unicode-agnostic mode, where Py_UNICODE is an incomplete type. Unicode-agnostic modules will load in both UCS2 and UCS4 interpreters. If a module

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: Daniel Stutzbach wrote: New submission from Daniel Stutzbach dan...@stutzbachenterprises.com: Currently, Python can be built with an internal Unicode representation of UCS2 or UCS4. To prevent extension modules compiled with the

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: I propose a different approach: 1. add a flag to PyModuleDef, indicating whether the module was built in UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module, instead of having the dynamic linker do so. 2. provide a

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: Martin v. Löwis wrote: Martin v. Löwis mar...@v.loewis.de added the comment: I propose a different approach: 1. add a flag to PyModuleDef, indicating whether the module was built in UCS-2 or UCS-4 mode. Then let the interpreter

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 5:03 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: If you can propose a different method of reliably protecting against mixed Unicode build module loads, that would be great. We could then get

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: Daniel Stutzbach wrote: Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 5:03 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: If you can propose a different method of reliably protecting

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 10:16 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: Are you sure this doesn't get optimized away in practice ? I'm sure it doesn't get optimized away by gcc 4.3, where I tested it. :) Sure,

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: Are you sure this doesn't get optimized away in practice ? I'm sure it doesn't get optimized away by gcc 4.3, where I tested it. :) Did you mean to include the hunk from msg105295 as part of the patch? If so, wouldn't that defeat the

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 8:07 AM, Martin v. Löwis rep...@bugs.python.org wrote: 1. add a flag to PyModuleDef, indicating whether the module was built in UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module,

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 11:02 AM, Martin v. Löwis rep...@bugs.python.org wrote: Did you mean to include the hunk from msg105295 as part of the patch? If so, wouldn't that defeat the whole point of the patch? My intention is to

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: In most cases that will Just Work, without requiring the module writer to modify their PyModuleDef. I'd rather modify PyModuleDef_Base, to include a flags field, and perhaps put the api version into it, as well. That's an ABI change,

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 11:20 AM, Martin v. Löwis rep...@bugs.python.org wrote: I'd rather modify PyModuleDef_Base, to include a flags field, and perhaps put the api version into it, as well. That's an ABI change,

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: Daniel Stutzbach wrote: Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 10:16 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: Are you sure this doesn't get optimized away in practice ?

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: On Sat, May 8, 2010 at 11:35 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: One of the more important cases you are missing is the argument parser in Python: Thanks. I've had my head buried in c-api/unicode.html and

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: In Unicode-agnostic mode, instead of leaving Py_UNICODE, PyUnicodeObject, and many functions undefined, I wonder if it would be sufficient to declare Py_UNICODE like this: struct PY_UNICODE_TYPE; typedef struct

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Arfrever Frehtes Taifersar Arahesis
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-08 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: In Unicode-agnostic mode, instead of leaving Py_UNICODE, PyUnicodeObject, and many functions undefined, I wonder if it would be sufficient to declare Py_UNICODE like this: Sounds good. -- ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread Daniel Stutzbach
New submission from Daniel Stutzbach dan...@stutzbachenterprises.com: Currently, Python can be built with an internal Unicode representation of UCS2 or UCS4. To prevent extension modules compiled with the wrong Unicode representation from linking, unicodeobject.h #defines many of the Unicode

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread Daniel Stutzbach
Changes by Daniel Stutzbach dan...@stutzbachenterprises.com: -- keywords: +patch Added file: http://bugs.python.org/file17248/unicode.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: Forcing the compile-time and link-time encodings to match is tricky. The goal is: - Allow Unicode-agnostic modules to link, regardless of unicode settings - Cause a link failure if the unicode settings are mismatched for a

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: Adding people to the Nosy List who participated in the original thread on python-ideas, several months ago. Hope you don't mind. :-) -- nosy: +gvanrossum, scoder, zooko ___ Python

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread Daniel Stutzbach
Daniel Stutzbach dan...@stutzbachenterprises.com added the comment: I've been thinking about this a bit more. There are three types of symbols in unicodeobject.h: 1. Functions that are always safe to use 2. Functions that are only safe if the module is compiled with the same Unicode settings

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread R. David Murray
R. David Murray rdmur...@bitdance.com added the comment: Adding MvL because he wrote the ABI PEP, and MAL because he cares about the Unicode interface. -- nosy: +lemburg, loewis, r.david.murray ___ Python tracker rep...@bugs.python.org

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___ ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

2010-05-07 Thread Zooko O'Whielacronx
Zooko O'Whielacronx zo...@zooko.com added the comment: Option 1: Make Unicode-agnosticism the default and force anyone who cares about the Unicode setting to include a separate header. If they don't include that header, they can only call safe functions and can't poke at