#23271: Makemessages can corrupt existing .po files on Windows
-------------------------------------+-------------------------------------
Reporter: danielmenzel | Owner: nobody
Type: Bug | Status: new
Component: | Version: master
Internationalization |
Severity: Release blocker | Resolution:
Keywords: makemessages utf8 | Triage Stage: Accepted
unicode |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by daphshez):
* status: closed => new
* resolution: fixed =>
Comment:
Regression fails on my machine (Windows 7 64, Django development version
head, with
{{{
locale.getpreferredencoding()=='cp1255'
}}}
{{{
======================================================================
ERROR: test_po_file_encoding_when_updating
(i18n.test_extraction.BasicExtractorTests)
Update of PO file doesn't corrupt it with non-UTF-8 encoding on
Python3+Windows (#23271)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\data\dev\django\django\tests\i18n\test_extraction.py", line
407, in test_po_file_encoding_when_updating
management.call_command('makemessages', locale=['pt_BR'], verbosity=0)
File "C:\data\dev\django\django\django\core\management\__init__.py",
line 118, in call_command
return command.execute(*args, **defaults)
File "C:\data\dev\django\django\django\core\management\base.py", line
398, in execute
output = self.handle(*args, **options)
File
"C:\data\dev\django\django\django\core\management\commands\makemessages.py",
line 322, in handle
self.write_po_file(potfile, locale)
File
"C:\data\dev\django\django\django\core\management\commands\makemessages.py",
line 455, in write_po_file
msgs, errors, status = gettext_popen_wrapper(args)
File
"C:\data\dev\django\django\django\core\management\commands\makemessages.py",
line 39, in
gettext_popen_wrapper
stdout, stderr, status_code = popen_wrapper(args,
os_err_exc_type=os_err_exc_type)
File "C:\data\dev\django\django\django\core\management\utils.py", line
27, in popen_wrapper
output, errors = p.communicate()
File "C:\Python34\lib\subprocess.py", line 959, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "C:\Python34\lib\subprocess.py", line 1234, in _communicate
stdout = stdout[0]
IndexError: list index out of range
}}}
Also, this seems also to be the source of #21928 which isn't resolved for
other people.
The problem is that Popen still tries to encode msgmerge output stream
using DEFAULT_LOCALE_ENCODING, when the data is actually in UTF-8.
A way around it is to call Popen with universal_newlines=true, and then
take care of line ending inside gettext_popen_wrapper. I'll post a patch
soon.
(BTW the current solution is for Windows only. Linux uses have to set
undocumented environment variables. I think we should strive to solve this
for Linux users as well).
--
Ticket URL: <https://code.djangoproject.com/ticket/23271#comment:10>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/070.033bffcd590a20cb5aefcc9b71aff7b7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
