Francesco Romani has submitted this change and it was merged. Change subject: startup: Change system default encoding to utf8 ......................................................................
startup: Change system default encoding to utf8 In Python 2, the system default encoding is 'ascii'. This causes mixing of unicode and non-ascii strings (e.g. utf8 encoded) to fail with UnicodeDecodeError or UnicodeEncodeError. The trigger for this failures is starting using the built-in json library, that returns all values as unicode strings, even if the value is ascii. For example: >>> u'ascii' + '\xd7\x90' Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 0: ordinal not in range(128) Python tries to decode the second value implicitly, and fails since this is a utf8 encoded string. To avoid such issues, the entire application must be changed to use only strings or only unicode internally, and never mix these types. In the distant future, when we run on Python 3, this will be true. For now, the only way to prevent these issues systematically is to change the system default encoding to 'utf8'. Changing the default encoding is done with sys.setdefaultencoding(), but this function exists only during startup, and can be called only from sitecustomize module. We keep now sitecustomize.py module, and change PYTHONPATH so it will be loaded on startup. This change is not needed in Python 3 since the default encoding is already utf8, and Python does not do any implicit decoding or encoding when mixing 'str' and 'bytes' types. Change-Id: Icc3f072a499ba4034bdbedd09eb60d7a3a32f9c8 Bug-Url: https://bugzilla.redhat.com/1281940 Signed-off-by: Nir Soffer <nsof...@redhat.com> Reviewed-on: https://gerrit.ovirt.org/48661 Continuous-Integration: Jenkins CI Reviewed-by: Milan Zamazal <mzama...@redhat.com> Reviewed-by: Francesco Romani <from...@redhat.com> Reviewed-by: Piotr Kliczewski <piotr.kliczew...@gmail.com> Reviewed-on: https://gerrit.ovirt.org/51273 --- M debian/vdsm.install M init/daemonAdapter M tests/Makefile.am A tests/unicode_test.py M vdsm.spec.in M vdsm/Makefile.am A vdsm/sitecustomize.py 7 files changed, 84 insertions(+), 0 deletions(-) Approvals: Nir Soffer: Verified Jenkins CI: Passed CI tests Francesco Romani: Looks good to me, approved -- To view, visit https://gerrit.ovirt.org/51273 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: merged Gerrit-Change-Id: Icc3f072a499ba4034bdbedd09eb60d7a3a32f9c8 Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: ovirt-3.6 Gerrit-Owner: Nir Soffer <nsof...@redhat.com> Gerrit-Reviewer: Francesco Romani <from...@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsof...@redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczew...@gmail.com> Gerrit-Reviewer: Yaniv Bronhaim <ybron...@redhat.com> Gerrit-Reviewer: gerrit-hooks <automat...@ovirt.org> _______________________________________________ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
