Bug#934928: win32-loader FTBFS on stable - any idea ?
Control: clone -1 -2 Control: reopen -2 src:nsis Control: retitle -2 NSIS: Farsi translation contains impossible conversions (YEH to CP1256) Control: tags -2 upstream Le samedi, 24 août 2019, 11.52:53 h CEST Simon McVittie a écrit : > > Unable to convert processed string "نوشتن ممکن نیست: " to codepage 1256 > > That string appears to be part of the Farsi (fa) translation of nsis, > found in "Contrib/Language files/Farsi.nlf" in nsis_3.04-1. It is indeed > not possible to convert it to Windows codepage 1256: > > $ python3 > > >>> "نوشتن ممکن نیست".encode('cp1256') > > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3.7/encodings/cp1256.py", line 12, in encode > return codecs.charmap_encode(input,errors,encoding_table) > UnicodeEncodeError: 'charmap' codec can't encode character '\u06cc' in > position 12: character maps to $ unicode U+06CC > U+06CC ARABIC LETTER FARSI YEH > > (I don't know whether converting this string to CP1256 is an appropriate > thing to be doing.) > > If that's the problem, maybe it would be possible to work around this by > adjusting or disabling the Farsi translation, or by replacing the use of > CP1256 with UTF-16 or something? Indeed. According to this thread on unicode-ml in 2001 [1], this seems to only be supported in "not-exactly-CP1256" old Windows environments. There's a similar bug in Pidgin [2], which brings the following comment [3]: ;; As this file needs to be encoded in CP1256 and CP1256 doesn't support U+06CC ;; and U+0654 characters, I have removed all U+0654 characters and replaced U+06CC ;; with U+064A in the middle of the words and with U+0649 at the end of the words. ;; The Presian text will display correctly but the encoding is incorrect. It seems they entirely disabled persian translation for their installer now [4]. So; it seems to me that this is something that should be addressed in NSIS (upstream), so cloning and reassigning, so that we have a reference bug to point to. Cheers, OdyX [1] https://unicode.org/mail-arch/unicode-ml/y2001-m10/0197.html NSIS [2] https://developer.pidgin.im/ticket/2573 [3] https://developer.pidgin.im/attachment/ticket/2573/persian.2.nsh [4] https://bitbucket.org/pidgin/main/src/7c5b54ec931b03b9354e0d2fffcac40b13e4aaa5/pidgin/win32/nsis/create_nsis_translations.pl#lines-110 signature.asc Description: This is a digitally signed message part.
Bug#934928: win32-loader FTBFS on stable - any idea ?
Le dimanche, 25 août 2019, 12.20:16 h CEST Simon McVittie a écrit : > On Sat, 24 Aug 2019 at 18:49:20 +0200, Didier 'OdyX' Raboud wrote: > > The difference I finally found was that the buildds use LANG=C.UTF-8 and > > LC_ALL=C.UTF-8 whereas mine was not enforcing these (and so I was building > > with LC_ALL=POSIX). > > This was a change in sbuild 0.78.0, so in practice a difference between > buildds hosted on <= stretch (which used LC_ALL=POSIX) and buildds hosted > on >= buster (which use LC_ALL=C.UTF-8). Ah, thanks for the background. > > +# A non-UTF-8 locale is needed for the NSIS build to convert some > > language > > strings > > +LC_ALL := POSIX > > +export LC_ALL > > This is the first time I've encountered a package where changing the locale > to C.UTF-8 *causes* FTBFS - normally the failure mode is that unit tests > assume a UTF-8 (or at least legacy 8-bit) locale, and fail in LC_ALL=POSIX > (or equivalently LC_ALL=C). win32-loader is special™ :-) > It would be interesting to know whether the Farsi locale *works* in the > resulting build; but if it doesn't, then that probably isn't a regression, > because version 0.9.4 in buster would probably be broken in the same way. I have not tested win32-loader in non-Wine win32 environments recently, so we'd need someone with a Farsi Windows to test this. But I see that Thomas is doing a lot of work on the experimental branch, and we should let this hit unstable soon! Cheers, OdyX signature.asc Description: This is a digitally signed message part.
Bug#934928: win32-loader FTBFS on stable - any idea ?
Le samedi, 24 août 2019, 20.26:37 h CEST Adam D. Barratt a écrit : > On Sat, 2019-08-24 at 18:49 +0200, Didier 'OdyX' Raboud wrote: > > @Adam: I assume I need to bump the version number and upload again, > > right? > > Yep. > > (I assume the changelog etc still end up generated as UTF-8.) Yep. So uploaded. -- OdyX signature.asc Description: This is a digitally signed message part.
Bug#934928: win32-loader FTBFS on stable - any idea ?
On Sat, 24 Aug 2019 at 18:49:20 +0200, Didier 'OdyX' Raboud wrote: > The difference I finally found was that the buildds use LANG=C.UTF-8 and > LC_ALL=C.UTF-8 whereas mine was not enforcing these (and so I was building > with LC_ALL=POSIX). This was a change in sbuild 0.78.0, so in practice a difference between buildds hosted on <= stretch (which used LC_ALL=POSIX) and buildds hosted on >= buster (which use LC_ALL=C.UTF-8). Forcing the POSIX (or equivalently C) locale effectively means you are reverting a small part of the behaviour of newer sbuild, to be more like the old sbuild where the win32-loader currently in buster was built, so it seems a good minimal change to address this. > +# A non-UTF-8 locale is needed for the NSIS build to convert some language > strings > +LC_ALL := POSIX > +export LC_ALL This is the first time I've encountered a package where changing the locale to C.UTF-8 *causes* FTBFS - normally the failure mode is that unit tests assume a UTF-8 (or at least legacy 8-bit) locale, and fail in LC_ALL=POSIX (or equivalently LC_ALL=C). It would be interesting to know whether the Farsi locale *works* in the resulting build; but if it doesn't, then that probably isn't a regression, because version 0.9.4 in buster would probably be broken in the same way. smcv
Bug#934928: win32-loader FTBFS on stable - any idea ?
On Sat, 2019-08-24 at 18:49 +0200, Didier 'OdyX' Raboud wrote: > @Adam: I assume I need to bump the version number and upload again, > right? Yep. (I assume the changelog etc still end up generated as UTF-8.) Regards, Adam
Bug#934928: win32-loader FTBFS on stable - any idea ?
Hi there Simon, thanks for the detailed investigation. Le samedi, 24 août 2019, 11.52:53 h CEST Simon McVittie a écrit : > On Fri, 23 Aug 2019 at 18:35:59 +0200, Didier 'OdyX' Raboud wrote: > > I have uploaded win32-loader to buster to fix the out-of-sync archive > > keys, but it has now repeatedly failed to build from source on the buster > > buildds: > > > > https://buildd.debian.org/status/logs.php?pkg=win32-loader=0.9.4%2Bdeb > > 10u1 > > > > I can't reproduce this in a local buster schroot, so I'm slightly at loss. > > I can reproduce this by trying to rebuild 0.9.4 in sbuild (a buster > schroot created with sbuild-createchroot, hosted on a buster VM, using > vectis[1]) if that's any help? Presumably there is some difference > between your buster schroot and the one sbuild would use. See below > for a full package list. Yep. The difference I finally found was that the buildds use LANG=C.UTF-8 and LC_ALL=C.UTF-8 whereas mine was not enforcing these (and so I was building with LC_ALL=POSIX). By fixing this, I could reproducibly fail to build win32-loader :-) > > Unable to convert processed string "نوشتن ممکن نیست: " to codepage 1256 > > That string appears to be part of the Farsi (fa) translation of nsis, > found in "Contrib/Language files/Farsi.nlf" in nsis_3.04-1. It is indeed > not possible to convert it to Windows codepage 1256: (…) > > (I don't know whether converting this string to CP1256 is an appropriate > thing to be doing.) > > If that's the problem, maybe it would be possible to work around this by > adjusting or disabling the Farsi translation, or by replacing the use of > CP1256 with UTF-16 or something? Hrm. For a stable upload, this seems to be enough to let the build go through: --- a/debian/rules +++ b/debian/rules @@ -24,6 +24,10 @@ BUILT_USING_LIST := $(shell set -e; \ NSIS_VERSION := $(shell dpkg-query -f='$${Version}' -W nsis ) +# A non-UTF-8 locale is needed for the NSIS build to convert some language strings +LC_ALL := POSIX +export LC_ALL + %: dh $@ Any idea of a more targeted fix? @Adam: I assume I need to bump the version number and upload again, right? (debdiff attached) Cheers, OdyXdiff -Nru win32-loader-0.9.4+deb10u1/debian/changelog win32-loader-0.9.4+deb10u2/debian/changelog --- win32-loader-0.9.4+deb10u1/debian/changelog 2019-08-16 08:53:00.0 +0200 +++ win32-loader-0.9.4+deb10u2/debian/changelog 2019-08-24 18:44:18.0 +0200 @@ -1,3 +1,9 @@ +win32-loader (0.9.4+deb10u2) buster; urgency=medium + + * Fix stable FTBFS through enforcing LC_ALL=POSIX + + -- Didier Raboud Sat, 24 Aug 2019 18:44:18 +0200 + win32-loader (0.9.4+deb10u1) buster; urgency=medium * Rebuild in stable to embed the latest debian-archive-keyring diff -Nru win32-loader-0.9.4+deb10u1/debian/rules win32-loader-0.9.4+deb10u2/debian/rules --- win32-loader-0.9.4+deb10u1/debian/rules 2019-08-16 08:53:00.0 +0200 +++ win32-loader-0.9.4+deb10u2/debian/rules 2019-08-24 18:44:18.0 +0200 @@ -24,6 +24,10 @@ NSIS_VERSION := $(shell dpkg-query -f='$${Version}' -W nsis ) +# A non-UTF-8 locale is needed for the NSIS build to convert some language strings +LC_ALL := POSIX +export LC_ALL + %: dh $@ signature.asc Description: This is a digitally signed message part.
Bug#934928: win32-loader FTBFS on stable - any idea ?
On Fri, 23 Aug 2019 at 18:35:59 +0200, Didier 'OdyX' Raboud wrote: > I have uploaded win32-loader to buster to fix the out-of-sync archive keys, > but it has now repeatedly failed to build from source on the buster buildds: > > https://buildd.debian.org/status/logs.php?pkg=win32-loader=0.9.4%2Bdeb10u1 > > I can't reproduce this in a local buster schroot, so I'm slightly at loss. I can reproduce this by trying to rebuild 0.9.4 in sbuild (a buster schroot created with sbuild-createchroot, hosted on a buster VM, using vectis[1]) if that's any help? Presumably there is some difference between your buster schroot and the one sbuild would use. See below for a full package list. 0.9.4 was originally compiled in January, so a lot has changed since then. In particular the mingw gcc 7 has been replaced by gcc 8. > Unable to convert processed string "نوشتن ممکن نیست: " to codepage 1256 That string appears to be part of the Farsi (fa) translation of nsis, found in "Contrib/Language files/Farsi.nlf" in nsis_3.04-1. It is indeed not possible to convert it to Windows codepage 1256: $ python3 >>> "نوشتن ممکن نیست".encode('cp1256') Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.7/encodings/cp1256.py", line 12, in encode return codecs.charmap_encode(input,errors,encoding_table) UnicodeEncodeError: 'charmap' codec can't encode character '\u06cc' in position 12: character maps to $ unicode U+06CC U+06CC ARABIC LETTER FARSI YEH (I don't know whether converting this string to CP1256 is an appropriate thing to be doing.) If that's the problem, maybe it would be possible to work around this by adjusting or disabling the Farsi translation, or by replacing the use of CP1256 with UTF-16 or something? smcv [1] https://salsa.debian.org/smcv/vectis Packages in my sbuild chroot when I reproduced this build failure: adduser_3.118 adwaita-icon-theme_3.30.1-1 apt_1.8.2 autoconf_2.69-11 automake_1:1.16.1-4 autopoint_0.19.8.1-9 autotools-dev_20180224.1 base-files_10.3 base-passwd_3.5.46 bash_5.0-4 binutils_2.31.1-16 binutils-common_2.31.1-16 binutils-mingw-w64-i686_2.31.1-11+8.3 binutils-mingw-w64-x86-64_2.31.1-11+8.3 binutils-x86-64-linux-gnu_2.31.1-16 bsdmainutils_11.1.2+b1 bsdutils_1:2.33.1-0.1 build-essential_12.6 bzip2_1.0.6-9.1 ca-certificates_20190110 coreutils_8.30-3 cpio-win32_2.12+dfsg-9 cpp_4:8.3.0-1 cpp-8_8.3.0-6 dash_0.5.10.2-5 dbus_1.12.16-1 dbus-user-session_1.12.16-1 dconf-gsettings-backend_0.30.1-2 dconf-service_0.30.1-2 debconf_1.5.71 debhelper_12.1.1 debian-archive-keyring_2019.1 debianutils_4.8.6.1 dh-autoreconf_19 dh-strip-nondeterminism_1.1.2-1 diffutils_1:3.7-3 dmsetup_2:1.02.155-3 dpkg_1.19.7 dpkg-dev_1.19.7 dwz_0.12-3 e2fsprogs_1.44.5-1 fakeroot_1.23-1 fdisk_2.33.1-0.1 file_1:5.35-4 findutils_4.6.0+git+20190209-2 fontconfig_2.13.1-2 fontconfig-config_2.13.1-2 fonts-dejavu-core_2.37-1 g++_4:8.3.0-1 g++-8_8.3.0-6 g++-mingw-w64_8.3.0-6+21.3~deb10u1 g++-mingw-w64-i686_8.3.0-6+21.3~deb10u1 g++-mingw-w64-x86-64_8.3.0-6+21.3~deb10u1 gawk_1:4.2.1+dfsg-1 gcc_4:8.3.0-1 gcc-8_8.3.0-6 gcc-8-base_8.3.0-6 gcc-mingw-w64_8.3.0-6+21.3~deb10u1 gcc-mingw-w64-base_8.3.0-6+21.3~deb10u1 gcc-mingw-w64-i686_8.3.0-6+21.3~deb10u1 gcc-mingw-w64-x86-64_8.3.0-6+21.3~deb10u1 gettext_0.19.8.1-9 gettext-base_0.19.8.1-9 glib-networking_2.58.0-2 glib-networking-common_2.58.0-2 glib-networking-services_2.58.0-2 gpgv_2.2.12-1 gpgv-win32_2.2.12-1 grep_3.3-1 groff-base_1.22.4-3 grub-common_2.02+dfsg1-20 grub-pc-bin_2.02+dfsg1-20 gsettings-desktop-schemas_3.28.1-1 gtk-update-icon-cache_3.24.5-1 gzip_1.9-3 gzip-win32_1.9-3 hicolor-icon-theme_0.17-2 hostname_3.21 icoutils_0.32.3-2.1 imagemagick_8:6.9.10.23+dfsg-2.1 imagemagick-6-common_8:6.9.10.23+dfsg-2.1 imagemagick-6.q16_8:6.9.10.23+dfsg-2.1 init-system-helpers_1.56+nmu1 intltool-debian_0.35.0+20060710.5 ipxe_1.0.0+git-20190125.36a4c85-1 libacl1_2.2.53-4 libapparmor1_2.13.2-10 libapt-pkg5.0_1.8.2 libarchive-zip-perl_1.64-1 libargon2-1_0~20171227-0.2 libasan5_8.3.0-6 libatk-bridge2.0-0_2.30.0-5 libatk1.0-0_2.30.0-2 libatk1.0-data_2.30.0-2 libatomic1_8.3.0-6 libatspi2.0-0_2.30.0-7 libattr1_1:2.4.48-4 libaudit-common_1:2.8.4-3 libaudit1_1:2.8.4-3 libavahi-client3_0.7-4+b1 libavahi-common-data_0.7-4+b1 libavahi-common3_0.7-4+b1 libbinutils_2.31.1-16 libblkid1_2.33.1-0.1 libbsd0_0.9.1-2 libbz2-1.0_1.0.6-9.1 libc-bin_2.28-10 libc-dev-bin_2.28-10 libc-l10n_2.28-10 libc6_2.28-10 libc6-dev_2.28-10 libcairo-gobject2_1.16.0-4 libcairo2_1.16.0-4 libcap-ng0_0.7.9-2 libcap2_1:2.25-2 libcc1-0_8.3.0-6 libcolord2_1.4.3-4 libcom-err2_1.44.5-1 libcroco3_0.6.12-3 libcryptsetup12_2:2.1.0-5 libcups2_2.2.10-6 libdatrie1_0.2.12-2 libdb5.3_5.3.28+dfsg1-0.5 libdbus-1-3_1.12.16-1 libdconf1_0.30.1-2 libde265-0_1.0.3-1+b1 libdebconfclient0_0.249 libdevmapper1.02.1_2:1.02.155-3 libdpkg-perl_1.19.7 libefiboot1_37-2 libefivar1_37-2 libelf1_0.176-1.1 libencode-locale-perl_1.05-1 libepoxy0_1.5.3-0.1 libexpat1_2.2.6-2 libext2fs2_1.44.5-1
Bug#934928: win32-loader FTBFS on stable - any idea ?
Hi there Thomas, I have uploaded win32-loader to buster to fix the out-of-sync archive keys, but it has now repeatedly failed to build from source on the buster buildds: https://buildd.debian.org/status/logs.php?pkg=win32-loader=0.9.4%2Bdeb10u1 I can't reproduce this in a local buster schroot, so I'm slightly at loss. Do you have any idea why it fails to build? Would you see a minimal patch to let it build in stable? Many thanks in advance, Cheers, OdyX signature.asc Description: This is a digitally signed message part.