[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: > 0:00:22 [ 16/419/2] test_unicode crashed (Exit code -11) Please open a new issue to track this bug. -- ___ Python tracker ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Victor, looks good to me: 0:00:26 [ 23/419/3] test_utf8_mode passed. I don't know wether it is related, but test_unicode crash dumps here: 0:00:22 [ 16/419/2] test_unicode crashed (Exit code -11) Fatal Python error: Segmentation fault

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: New changeset 7d35f79012db89ce9a152a77ac6809eb9c34a35d by Victor Stinner in branch '3.7': bpo-34403: Always implement _Py_GetForceASCII() (GH-10235) https://github.com/python/cpython/commit/7d35f79012db89ce9a152a77ac6809eb9c34a35d --

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by STINNER Victor : -- pull_requests: +9547 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: New changeset 21220bbe65108f5a763ead24a6b572f80d84c9e2 by Victor Stinner in branch '3.7': bpo-34403: Fix initfsencoding() for ASCII (GH-10233) https://github.com/python/cpython/commit/21220bbe65108f5a763ead24a6b572f80d84c9e2 --

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: Michael Osipov: Oops, my commit b2457efc78b74a1d6d1b77d11a939e886b8a4e2c broke the filesystem encoding on HP-UX. It should be fixed by my commit 905f1ace5f7424e314ca7bed997868a2a3044839. -- ___ Python tracker

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: New changeset 905f1ace5f7424e314ca7bed997868a2a3044839 by Victor Stinner in branch 'master': bpo-34523: Fix config_init_fs_encoding() for ASCII (GH-10232) https://github.com/python/cpython/commit/905f1ace5f7424e314ca7bed997868a2a3044839 --

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by STINNER Victor : -- pull_requests: +9546 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by STINNER Victor : -- pull_requests: +9545 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: > I am the AIX(tools) Michael, Michael O is the HP-UX Michael :p Oh, I didn't notice that you two have the same first name :-) > So I was not the one asking. IMHO - as the PEP was new, if I understood correctly, in 3.7 - would be "nice" to see it back

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: On 28/08/2018 23:14, STINNER Victor wrote: > STINNER Victor added the comment: > >> Can we backport this to 3.7 at least? I am the AIX(tools) Michael, Michael O is the HP-UX Michael :p So I was not the one asking. IMHO - as the PEP was new, if I understood

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: On 28/08/2018 20:43, Michael Felt wrote: > Attached is the output with LC_ALL=C in the prefix. If you were hoping > for "dangling processes - your hopes are affirmed. Previous mail ended with: == Tests result: FAILURE == 375 tests OK. 13 tests failed:    

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: On 28/08/2018 13:20, STINNER Victor wrote: > I updated my PR 8969 to implement properly my idea. With this PR, on HP-UX > with C or POSIX locale, Python now uses ASCII for its "filesystem encoding": > sys.getfilesystemencoding() returns "ascii". > > Michael:

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: > Please close, issue fixed. Thank you very much. You're welcome ;-) -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Please close, issue fixed. Thank you very much. -- ___ Python tracker ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: > Can we backport this to 3.7 at least? My policy is to focus on the master branch to support a new platform. Then add a buildbot and find a core developer to maintain this platform. See the PEP 11 for details. I would prefer to see a full test suite

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Can we backport this to 3.7 at least? -- ___ Python tracker ___ ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: New changeset d500e5307aec9c5d535f66d567fadb9c587a9a36 by Victor Stinner in branch 'master': bpo-34403: On HP-UX, force ASCII for C locale (GH-8969) https://github.com/python/cpython/commit/d500e5307aec9c5d535f66d567fadb9c587a9a36 --

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Victor, this looks good to me: > osipovmi@blnn724x:/var/osipovmi/cpython [] > $ git fetch vstinner > remote: Counting objects: 65, done. > remote: Compressing objects: 100% (18/18), done. > remote: Total 65 (delta 41), reused 43 (delta

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: No time to compile for a couple of days. Stress from others wins instead. Maybe on Friday. Sent from my iPhone > On 28 Aug 2018, at 13:20, STINNER Victor wrote: > > > STINNER Victor added the comment: > >> -check('utf8=0',

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: > -check('utf8=0', [arg.decode(encoding, 'surrogateescape')], > +check('utf8=0', [arg.decode('ascii', 'surrogateescape')], >LC_ALL='C') > (...) > == Tests result: SUCCESS == Good, it works. I updated my PR 8969 to

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Here is the output to your questions: > osipovmi@blnn724x:/var/osipovmi/cpython [] > $ git checkout hpux_force_ascii > Branch 'hpux_force_ascii' set up to track remote branch 'hpux_force_ascii' > from 'vstinner'. > Switched to a new branch

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: ... > byte 0xA7 decoded to Unicode character U+00A7 ... Well, it confirms what I expected: nl_langinfo(CODESET) announces "roman8", but mbstowcs() uses Latin1 encoding in practice. So I wrote the PR 8969 which forces the ASCII encoding in that case. I'm not

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by STINNER Victor : -- pull_requests: +8442 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Please see here: > osipovmi@blnn724x:~ [] > $ uname -a > HP-UX blnn724x B.11.31 U ia64 HP-UX > osipovmi@blnn724x:~ [] > $ locale > LANG=de_DE.utf8 > LC_CTYPE="de_DE.utf8" > LC_COLLATE="de_DE.utf8" > LC_MONETARY="de_DE.utf8" >

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: > File "/var/osipovmi/cpython/Lib/test/test_utf8_mode.py", line 214, in check > self.assertEqual(args, ascii(expected), out) > AssertionError: "['h\\xa7\\xe9']" != "['h\\xcf\\xd5']" > - ['h\xa7\xe9'] > + ['h\xcf\xd5'] > : roman8:['h\xa7\xe9'] Hum, it

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Running off: 217af1d38db3e1e875180c6fa160f0fc80e46003 > $ ./python -m test test_utf8_mode > Run tests sequentially > 0:00:00 [1/1] test_utf8_mode > test test_utf8_mode failed -- Traceback (most recent call last): > File

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: It unfortunately does not: > osipovmi@blnn724x:/var/osipovmi/cpython [] > $ git branch > 3.6 > 3.7 > bpo-14568 > bpo-34401 > bpo-34403 > bpo-34412 > bpo-34448 > bpo-34449 > bpo-34519 > master >

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Victor, looking to... -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

STINNER Victor added the comment: Hi, I'm the author of the UTF-8 Mode PEP (PEP 540) and its implementation. I wrote test_utf8_mode. I wasn't sure that it was a good idea to hardcode the locale encoding depending on the platform. The fact that AIX and HP-UX use different locale encoding

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by STINNER Victor : -- pull_requests: +8441 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by Michael Osipov <1983-01...@gmx.net>: -- keywords: +patch pull_requests: +8440 stage: -> patch review ___ Python tracker ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Now I know why this cannot with Roman 8: it contains chars which are multibyte in Unicode (UTF-8) which cannot be mapped into a 7-bit/8-bit encoding. Therefore CP1252 does not work because it has Unicode chars too. ISO-8859-1 solely

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Wow, this is pretty surprising. The very same patch for AIX works on HP-UX flawlessly: $ ./python -m test test_utf8_mode Run tests sequentially 0:00:00 [1/1] test_utf8_mode == Tests result: SUCCESS == 1 test OK. Total duration: 2 sec

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: On 27/08/2018 15:22, Michael Osipov wrote: > Michael Osipov <1983-01...@gmx.net> added the comment: > > So I changed the test code to: > > diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py > index 26e2e13ec5..d9f8a3ed8b 100644 > ---

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Maybe Victor Stinner has some insights here. -- nosy: +vstinner ___ Python tracker ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Maybe skipping the test is the best thing: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index 26e2e13ec5..d6c4b321be 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -12,7 +12,7 @@ from

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: So I changed the test code to: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index 26e2e13ec5..d9f8a3ed8b 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -208,7 +208,7 @@ class

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: I think you are absoltely right. > In any case, it seems to be broken for any system that does not have UTF-8 as > default. You likely mean ASCII. Python assumes that LANG=C is ASCII which is not the case for AIX and HP-UX. Your patch

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: As the AIX complaint is (was once the PR merges): AssertionError: "['h\\xc3\\xa9\\xe2\\x82\\xac']" != "['h\\udcc3\\udca9\\udce2\\udc82\\udcac']" - ['h\xc3\xa9\xe2\x82\xac'] + ['h\udcc3\udca9\udce2\udc82\udcac'] And the HP-UX complaint is: File

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: It might be as simple as what I saw for AIX: diff --git a/Lib/test/test_utf8_mode.py b/Lib/test/test_utf8_mode.py index 26e2e13ec5..3e918fd54c 100644 --- a/Lib/test/test_utf8_mode.py +++ b/Lib/test/test_utf8_mode.py @@ -219,6 +219,8 @@ class

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by Łukasz Langa : -- keywords: +3.7regression versions: -Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by Łukasz Langa : -- dependencies: +test_cmd_line test_utf8_mode test_warnings fail in all FreeBSD 3.x (3.8) buildbots ___ Python tracker ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Change by Michael Osipov <1983-01...@gmx.net>: -- versions: +Python 3.6, Python 3.8 ___ Python tracker ___ ___ Python-bugs-list

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Felt added the comment: Although the default is different (i.e., roman8 versus latin1 (iso8859-1)) both HP-UX and AIX (like Windows, cp1252) this issue and issue 33347 are related. As I mentioned in https://bugs.python.org/issue34347#msg323319 the string seen by self.get_output() is

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Michael Osipov <1983-01...@gmx.net> added the comment: Thanks, I'll do that. Hopefully I can provide a patch for. Though, I am convinced that I have to write a custom codec for roman8 to make all at stuff work flawlessly. -- ___ Python tracker

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

Terry J. Reedy added the comment: You might get more information asking questions on python-list. -- nosy: +terry.reedy ___ Python tracker ___

[issue34403] test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions

New submission from Michael Osipov <1983-01...@gmx.net>: Running from 3.7 branch on HP-UX 11.31 ia64, 32 bit, big endian. The test output is: > Re-running failed tests in verbose mode > Re-running test 'test_utf8_mode' in verbose mode > test_cmd_line (test.test_utf8_mode.UTF8ModeTests) ... FAIL