Bug#468209: Towards multi-arch: Multi-Arch: same file conflicts
On Fri, 25 Nov 2011, Steve Langasek wrote: No, having to split this data out into separate packages is a significant cost for maintainers and on the archive and simply the wrong way to do it. Automatic package splits for the likes of tdebs are fine, but we should not be forced to split binary packages in the archive for data files such as .mo files that could readily be made architecture-independent. Ok, I'm not 100% convinced, but after reading this, I'm willing to ask gettext authors that they make little-endian the default again, or at least they tell me how we could achieve that (configure options, environment variables, whatever). Steve: Could you please report this as a *new* bug and summarize the problems that native endianness in gettext create to multi-arch? (I would prefer to keep this issue as a different one from the old bug). A Subject like msgfmt creates mo files in native endianness would be fine. Thanks. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#468209: Towards multi-arch: Multi-Arch: same file conflicts
On Sat, Nov 19, 2011 at 10:55:53AM +, Neil Williams wrote: As long as the behaviour is always *consistent* across native builds and cross-builds, I would be happy with having all .mo files with the same endianness. By preference, little endian. I agree; little-endian is both the most common endianness for current hardware, and the endianness used by our lowest-end release-supported hardware, so little-endian would make the most sense. But the cost of using big-endian is probably not so high that it would be worth arguing about either. And is it worth splitting out a -l10n or -data package from a library just so the library itself can be M-A: same? (I suppose a side benefit is you can use Recommends and cut down a little on the size of your strict Dependency closure.) Yes, it is worth having -l10n, -tdeb, -data packages, precisely to get the library M-A: same. MultiArch wasn't a consideration when #468209 was opened or when the TDeb proposal was created in 2008. No, having to split this data out into separate packages is a significant cost for maintainers and on the archive and simply the wrong way to do it. Automatic package splits for the likes of tdebs are fine, but we should not be forced to split binary packages in the archive for data files such as .mo files that could readily be made architecture-independent. Having at first put a lot of time into generating .mo files which have matching endianness to the DEB_HOST_ARCH, I have changed my mind on exactly how this should work. Emdebian has tried .mo files which differ between architectures and it isn't worth the effort. Santiago was right the first time, I was wrong: let gettext deal with the load at runtime and don't fuss about the endianness of the file in /usr/share. *However*, I think that this means that we *should* make every .mo file to always be little endian. If the .mo files are always little-endian, then there's no need at all to split the package. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Bug#468209: Towards multi-arch: Multi-Arch: same file conflicts
On Fri, 18 Nov 2011 23:39:20 -0600 Peter Samuelson pe...@p12n.org wrote: [Jakub Wilk] The most common reasons for cross-architecture differences appear to be (in random order): - Compiling GNU message catalogs with gettext, which uses native endianness (see bug #468209). (Hmm, I did get v.carried away earlier in this bug report didn't I? Sorry, Santiago. I genuinely thought this was a lot more important at the time. All that cross-building was driving me scatty.) Having read that bug log, it's not clear to me whether there's a consensus about what to do about these. Neil thinks we need native endian .mo (which is problematic for multiarch) Native isn't quite the right meaning - same endianness as the DEB_HOST_ARCH platform is what I intended to implement in Emdebian based on the --endianness option to msgfmt. (Native could mean DEB_BUILD_ARCH which is worse for cross-building than just picking one endianness and sticking with it.) , others think we need .mo to be Arch: all and dont-care-endian. Has any consensus emerged? After more work on exactly what's going on with endianness and .mo files, Emdebian switched to making our TDebs Arch:all and letting gettext deal with the endianness before the Squeeze release, by which time the cross-built version of Emdebian was already inoperable. I should have followed that up to the bug log - it was closed and archived at that point but I forgot to check it. Sorry. http://www.emdebian.org/emdebian/tdebs.html As long as the behaviour is always *consistent* across native builds and cross-builds, I would be happy with having all .mo files with the same endianness. By preference, little endian. Like Santiago (468209#66), I maintain a library which has translations. For that package I have put the .mo files into a -data package (qof-data) precisely because that allows the Architecture:any libqof2 to depend on the Arch:all qof-data, thereby solving the MultiArch problem. This is also compatible with the TDeb proposal for Arch:all TDebs which contain .mo files (as well as the more difficult problem of the translated debconf templates file) because packages like qof-data can easily be processed as TDebs once those mechanisms are available. And is it worth splitting out a -l10n or -data package from a library just so the library itself can be M-A: same? (I suppose a side benefit is you can use Recommends and cut down a little on the size of your strict Dependency closure.) Yes, it is worth having -l10n, -tdeb, -data packages, precisely to get the library M-A: same. MultiArch wasn't a consideration when #468209 was opened or when the TDeb proposal was created in 2008. There are other reasons to split out the .mo files: 1. Updates to translations should not require source NMU's. 2. Translation data should not be distributed in architecture-dependent packages. 3. Translators should have a common interface for getting updates into Debian (possibly with automated TDeb generation after i18n team review). However, the TDeb proposal for Debian is stalled due to disagreement with the dpkg maintainers over the implementation mechanism. I want to see the arrival of a Multiarch-aware dpkg in unstable before raising that discussion again. In the meantime, I'm happy for all libraries packaging translations to add -l10n, -tdeb, -data or -common Arch:all packages in order to meet the higher priority of implementing MultiArch. Indirectly, this would help the eventual implementation of TDebs. If there's to be a release goal for that, I'm happy to help with it. Santiago wrote: In such case, making those packages to depend on another Arch: all package containing just the translations would solve the issue, would it not? (For the record, I happen to maintain a library containing translations, and I have always seen it as an anomaly, this would force me to do what I feel is the right thing). I agree with this completely and, as above, have fixed the anomaly that way for one library which uses gettext translations. There remains some disagreement: Santiago wrote: Yes, I now see what the problem is, but I don't see that making every .mo file to be always little endian again is the best solution. We could also tell dpkg somehow that different files in /usr/share/locale are ok in this case. Having at first put a lot of time into generating .mo files which have matching endianness to the DEB_HOST_ARCH, I have changed my mind on exactly how this should work. Emdebian has tried .mo files which differ between architectures and it isn't worth the effort. Santiago was right the first time, I was wrong: let gettext deal with the load at runtime and don't fuss about the endianness of the file in /usr/share. *However*, I think that this means that we *should* make every .mo file to always be little endian. I'm sorry if this seems like an about turn but what I originally wanted from #468209 was merely the documentation of the option so that
Bug#468209: Towards multi-arch: Multi-Arch: same file conflicts
not that i'm multi-lingual (i use google to translate!) don't we get all .mo avail. in packages already? (i hope) # locate *.mo | wc 13341 13341 648305 ... and if build/tar admins say choose another method why not try asking them what? can't i delete .mo locally if i'm bitwise desperate for disk space? have fun! JohnHendrickson Neil Williams wrote: On Fri, 18 Nov 2011 23:39:20 -0600 Peter Samuelson pe...@p12n.org wrote: [Jakub Wilk] The most common reasons for cross-architecture differences appear to be (in random order): - Compiling GNU message catalogs with gettext, which uses native endianness (see bug #468209). (Hmm, I did get v.carried away earlier in this bug report didn't I? Sorry, Santiago. I genuinely thought this was a lot more important at the time. All that cross-building was driving me scatty.) Having read that bug log, it's not clear to me whether there's a consensus about what to do about these. Neil thinks we need native endian .mo (which is problematic for multiarch) Native isn't quite the right meaning - same endianness as the DEB_HOST_ARCH platform is what I intended to implement in Emdebian based on the --endianness option to msgfmt. (Native could mean DEB_BUILD_ARCH which is worse for cross-building than just picking one endianness and sticking with it.) , others think we need .mo to be Arch: all and dont-care-endian. Has any consensus emerged? After more work on exactly what's going on with endianness and .mo files, Emdebian switched to making our TDebs Arch:all and letting gettext deal with the endianness before the Squeeze release, by which time the cross-built version of Emdebian was already inoperable. I should have followed that up to the bug log - it was closed and archived at that point but I forgot to check it. Sorry. http://www.emdebian.org/emdebian/tdebs.html As long as the behaviour is always *consistent* across native builds and cross-builds, I would be happy with having all .mo files with the same endianness. By preference, little endian. Like Santiago (468209#66), I maintain a library which has translations. For that package I have put the .mo files into a -data package (qof-data) precisely because that allows the Architecture:any libqof2 to depend on the Arch:all qof-data, thereby solving the MultiArch problem. This is also compatible with the TDeb proposal for Arch:all TDebs which contain .mo files (as well as the more difficult problem of the translated debconf templates file) because packages like qof-data can easily be processed as TDebs once those mechanisms are available. And is it worth splitting out a -l10n or -data package from a library just so the library itself can be M-A: same? (I suppose a side benefit is you can use Recommends and cut down a little on the size of your strict Dependency closure.) Yes, it is worth having -l10n, -tdeb, -data packages, precisely to get the library M-A: same. MultiArch wasn't a consideration when #468209 was opened or when the TDeb proposal was created in 2008. There are other reasons to split out the .mo files: 1. Updates to translations should not require source NMU's. 2. Translation data should not be distributed in architecture-dependent packages. 3. Translators should have a common interface for getting updates into Debian (possibly with automated TDeb generation after i18n team review). However, the TDeb proposal for Debian is stalled due to disagreement with the dpkg maintainers over the implementation mechanism. I want to see the arrival of a Multiarch-aware dpkg in unstable before raising that discussion again. In the meantime, I'm happy for all libraries packaging translations to add -l10n, -tdeb, -data or -common Arch:all packages in order to meet the higher priority of implementing MultiArch. Indirectly, this would help the eventual implementation of TDebs. If there's to be a release goal for that, I'm happy to help with it. Santiago wrote: In such case, making those packages to depend on another Arch: all package containing just the translations would solve the issue, would it not? (For the record, I happen to maintain a library containing translations, and I have always seen it as an anomaly, this would force me to do what I feel is the right thing). I agree with this completely and, as above, have fixed the anomaly that way for one library which uses gettext translations. There remains some disagreement: Santiago wrote: Yes, I now see what the problem is, but I don't see that making every .mo file to be always little endian again is the best solution. We could also tell dpkg somehow that different files in /usr/share/locale are ok in this case. Having at first put a lot of time into generating .mo files which have matching endianness to the DEB_HOST_ARCH, I have changed my mind on exactly how this should work. Emdebian has tried .mo files which differ between architectures and it isn't worth the effort. Santiago was right the first time, I was
