[issue36792] [Windows] time: crash on formatting time with de_DE locale

Eryk Sun added the comment: ucrt in Windows 10 v2004 uses an internal wide-character version of the time-zone name, which gets updated by _tzset() and kept in sync with the _tzname encoded strings. Also, note that the current implementation uses the locale's ANSI code page for both "de-DE"

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg361476 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Steve Dower added the comment: > Will Python support a release for as long as it's supported by the enterprise > version? For example, support for 1709 enterprise ends on 2020-04-14, so if > we followed that, then Python 3.9 would require Windows 10 1803 or higher. > That seems wrong while

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Eryk Sun added the comment: That the CRT caches the tzname strings as ANSI multibyte strings is frustrating -- whether or not it's buggy. I would expect there to be a _wtzname cache of the native OS strings that wcsftime uses directly, with no potential for failed encodings (e.g. empty

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Eryk Sun added the comment: > Even some well known locale names still use the utf-8 code page. Most > seem to uncommon, but at least es-BR (Brazil) does and would still > fall victim to these UCRT bugs. es-BR is a custom locale for the Spanish language in Brazil, as opposed to the common

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Eryk Sun added the comment: > Perhaps it would be easy to do the replacement of underscores with > hyphens on Windows in this function? I think that's safe enough, yes? Recent releases of ucrt implement this translation from underscore to hyphen for us, so this suggestion is no longer

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Change by Carsten Fuchs : -- nosy: +Carsten Fuchs ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: > * the experimental UTF-8 support was enabled because "de_DE" is not a > known Windows locale name - try with "de-DE" > > Perhaps it would be easy to do the replacement of underscores with hyphens > on Windows in this function? I think that's safe enough, yes?

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: I can confirm that using "de-DE" does indeed avoid the crash. -- ___ Python tracker ___ ___

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Steve Dower added the comment: I've received a detailed response from the UCRT team, and there are a few pieces here. * the fact that tzname is cached in ACP is known and will be fixed * the decoding bug is real, but it's due to the experimental UTF-8 support * the experimental UTF-8 support

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: I have managed to setup a VM that can reproduce the error. Unfortunately, the error (heap corruption) is coming from within the UCRT. Attempting to work around that, I came across another error in the UCRT. Due to these errors in all available UCRT

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: I've added another test executable (issue36792-2.zip) which should bring some insight into where things are going wrong. Please run and post the results. -- Added file: https://bugs.python.org/file48324/issue36792-2.zip

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Thanks again! I will have some more tests for you to try tomorrow as I am out of time for today. I'm currently of the belief that there is something Python is going to have to do to work around an issue within the CRT, but more testing will prove that

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: This is the result \issue36792>test.exe The current locale is now: C The time zone is: 'Mitteleuropõische Sommerzeit' (28 characters) The updated locale is now: de_DE The time zone is: '' (-1 characters) NB something is wrong with that

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Final test, this time, no Python what so ever. I've added a zip containing a simple C program (source and .exe) that performs the same test. The output should be similar to: The current locale is now: C The time zone is: 'Mountain Daylight Time' (22

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: And this is the result. old locale: C count: 28 value: Mitteleuropäische Sommerzeit new locale: de_DE count: -1 value: Windows fatal exception: code 0xc374 Looks like print('new locale:', crt_locale._wsetlocale(0, 'de_DE')) print('count:',

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Here is another test, this time removing Python from the equation (mostly :) import ctypes, struct crt_locale = ctypes.CDLL('api-ms-win-crt-locale-l1-1-0', use_errno=True) crt_time = ctypes.CDLL('api-ms-win-crt-time-l1-1-0', use_errno=True)

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: print('count:', crt_time.wcsftime(wbuf, 1024, '%Z', tm)) also fails but crt_convert = ctypes.CDLL('api-ms-win-crt-convert-l1-1-0', use_errno=True) print('count:', crt_convert.mbstowcs(wbuf, buf, 1024)) seems to work okay. --

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: You can safely execute each line individually (omitting the aforementioned count/value pairs) or depending on how the copy/paste is being done, just paste the script into a text editor (notepad) and comment out those lines. Then copy-paste that modified

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Thanks for your patience with this Charlie, but please try another run this time without the strftime() and mbstowcs() calls. Honest, we are getting closer! -- ___ Python tracker

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: If the process crashes at the first print statement, I'm not sure how I can run the tests. Or should I try them separately? -- ___ Python tracker

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: The code crashes on this line: print('count:', crt_time.strftime(buf, 1024, b'%Z', tm)) -- Added file: https://bugs.python.org/file48306/Report.wer ___ Python tracker

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Change by Charlie Clark : Added file: https://bugs.python.org/file48307/WER9DB9.tmp.WERInternalMetadata.xml ___ Python tracker ___ ___

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Thanks for the reminder Eryk Sun. This means the test needs to be run yet one more time :) import ctypes, locale, struct crt_time = ctypes.CDLL('api-ms-win-crt-time-l1-1-0', use_errno=True) locale.setlocale(locale.LC_ALL, 'de_DE') buf =

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Eryk Sun added the comment: > libc = ctypes.cdll.msvcrt That's the private CRT of Windows, not the Universal CRT for applications. In a release build (python.exe), use ctypes.CDLL('ucrtbase', use_errno=True). In a debug build (python_d.exe), use ctypes.CDLL('ucrtbased', use_errno=True). I

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: import ctypes, struct libc = ctypes.cdll.msvcrt buf = ctypes.create_string_buffer(1024) tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1) print('count:', libc.strftime(buf, 1024, b'%Z', tm)) print('value:', buf.value) wbuf =

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Oops, I forgot to add in my snippet, the setlocale() call prior to calling the C strftime() function. So an updated test: import locale locale.setlocale(locale.LC_ALL, 'de_DE') import ctypes, struct libc = ctypes.cdll.msvcrt buf =

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Charlie Clark added the comment: import ctypes, struct libc = ctypes.cdll.msvcrt buf = ctypes.create_string_buffer(1024) tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1) print('count:', libc.strftime(buf, 1024, b'%Z', tm)) print('value:', buf.value) count: 28 value:

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Jeremy Kloth added the comment: Ok, now let's try it using the C runtime directly: import ctypes, struct libc = ctypes.cdll.msvcrt buf = ctypes.create_string_buffer(1024) tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1) print('count:', libc.strftime(buf, 1024, b'%Z', tm))

[issue36792] [Windows] time: crash on formatting time with de_DE locale

Change by STINNER Victor : -- title: zipfile.writestr causes a Python crash on Windows if the locale is set -> [Windows] time: crash on formatting time with de_DE locale ___ Python tracker