Re: [xz-devel] Internationalization of man pages
On 2020-02-14 Mario Blättermann wrote: > Am Mi., 12. Feb. 2020 um 19:05 Uhr schrieb Lasse Collin > : > > On 2020-02-12 Mario Blättermann wrote: > > > Lasse Collin schrieb am Fr., 7. Feb. > > > 2020, 15:32: > > > > The extra po4a options like unknown_macros=untranslated > > > > aren't needed in XZ Utils. > > > > > > Not for now, but with this options it is better because po4a > > > changes its behavior from time to time. Keeping the options would > > > be safer. > > > > Hmmm, I don't know what kind of behavior changes po4a may have (or > > has had), > > In fact, it's the standard set of options which we use in > manpages-l10n to cover all imaginable problems and peculiarities in > Groff and Mdoc code. If you are sure you don't need these options, > just omit them. OK, makes sense when you have to handle man pages from many sources. I am pretty sure that those aren't needed in xz so now I feel fine to omit them. Thanks. > I've created a new pot file and merged with the existing translation. > The result is attached. Thanks. I committed it. I tested the makefile changes on old FreeBSD and OpenBSD images I happened to have around and the man page installation and symlink creation worked fine without GNU tools. So I guess it's OK to include the German man pages in xz 5.2.5. Should I mention something specific in the NEWS file about the German man pages? Like "these are newer than those in manpages-de 2.16"? I can imagine packagers being annoyed when files like this move from one package to another. Giving a hint about which file is newer could perhaps make them slightly less annoyed. > > The uninstall code can remove more files than the install part > > installs: if the package included German xz.1 but not German > > xzdec.1, the uninstall target will still remove both if they exist. > > It's a minor bug that I think is OK to leave unfixed. > > > This is only relevant in some odd cases when somebody installs into > /usr/ which is only for the files from distribution packages. Then it > could happen that a file vanishes, which actually belongs to the > manpages-de package. Well, it's some kind of undesired behaviour, but > users should normally know that they must not install anything in > /usr/ manually. In this particular example, if manpages-de contained both xz.1 and xzdec.1, the "make install" would overwrite the German xz.1 (but not xzdec.1). So some damage would already happen at the install step. Currently the installation of translated man pages is tied to NLS support. When the required NLS libraries aren't available for translating the program messages, configure disables NLS support. This also disables the installation of the translated man pages. I don't know if this matters. I don't even know how many OSes support translated man pages. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] Internationalization of man pages
Hm, forgot the attachment … Am Fr., 14. Feb. 2020 um 18:55 Uhr schrieb Mario Blättermann : > > Hello Lasse. > > Am Mi., 12. Feb. 2020 um 19:05 Uhr schrieb Lasse Collin > : > > > > On 2020-02-12 Mario Blättermann wrote: > > > Lasse Collin schrieb am Fr., 7. Feb. 2020, > > > 15:32: > > > > - po4a is never needed when building from a release tarball. > > > > > > This means, distribution packagers don't have to bother with po4a, > > > and the translated man pages will be installed automatically (unless > > > DISABLE_NLS) is set...? > > > > Exactly. > > > > > > The extra po4a options like unknown_macros=untranslated aren't > > > > needed in XZ Utils. > > > > > > Not for now, but with this options it is better because po4a changes > > > its behavior from time to time. Keeping the options would be safer. > > > > Hmmm, I don't know what kind of behavior changes po4a may have (or has > > had), > > In fact, it's the standard set of options which we use in > manpages-l10n to cover all imaginable problems and peculiarities in > Groff and Mdoc code. If you are sure you don't need these options, > just omit them. > > > so I may be missing something important. My thinking was: > > > > - groff_code=verbatim affects .de, .ie, and .if. These must be > > avoided anyway so getting an error from these is a good thing. > > > > - unknown_macros=untranslated: I hope that once po4a understands a > > macro, future versions will understand it too. Since there are no > > macros unknown to po4a now, I would like to get an error if I added > > such a macro. > > > > - untranslated="a.RE,\|": I don't understand this. It sounds like > > that certain macros shouldn't be translated but it made no > > difference with xz man pages. So perhaps it was useful where this > > line was copied. > > > > I can understand if unknown_macros=untranslated is needed for > > compatibility with old po4a versions, so it can be added if needed, > > although I would prefer the default pedantic behavior myself. > > > > > > - README wasn't updated yet to mention that man pages can be > > > > translated. > > > > > > After the new TP domain has been created (see below), you should add > > > a hint there. > > > > I will. > > > > > The German translation wasn't committed yet as po.de says this: > > > > > > > > This file is distributed under the same license as the > > > > manpages-de package. > > > > > > > > Which hopefully is trivial to change. :-) > > > > > > No problem, as the one and only author of the translation, I can > > > change the license to whatever you want. > > > > The original is public domain so that is somewhat preferred. If you > > prefer a permissive license it's OK to me, it just needs to be > > mentioned in the file COPYING too. Man page translation is a large > > amount of work after all. > > > I've created a new pot file and merged with the existing translation. > The result is attached. > > > > > If you wish the German man page translation to be part of 5.2.5, it > > > > could be good to test that "make DESTDIR=something install" also > > > > works on something else than GNU/Linux (the scripting for those > > > > details is ugly). The test should be done so that there are > > > > translated man pages available in the po4a/man directory, e.g. by > > > > using de.po from the first post of this thread. > > > > > > I will test it next weekend, but I only use GNU/Linux, I don't have MS > > > Windows, BSD, macOS or anything else. > > > > Thanks. While testing on GNU/Linux is useful still, trying on some *BSD > > or other POSIX system would be needed in case the makefile or shell > > scripting has portability issues. Perhaps someone else can try it. > > > > It should work fine even when --program-prefix=foo- or such configure > > options are used to get foo-xz, foo-unxz and so on. "make > > DESTDIR=something uninstall" should work too, although I guess the > > uninstall part is practically never needed. > > > > The uninstall code can remove more files than the install part > > installs: if the package included German xz.1 but not German xzdec.1, > > the uninstall target will still remove both if they exist. It's a minor > > bug that I think is OK to leave unfixed. > > > This is only relevant in some odd cases when somebody installs into > /usr/ which is only for the files from distribution packages. Then it > could happen that a file vanishes, which actually belongs to the > manpages-de package. Well, it's some kind of undesired behaviour, but > users should normally know that they must not install anything in > /usr/ manually. > > > Best Regards, > Mario > > > > BTW, what about to create a pre-release tarball which contains an > > > up-to-date *.pot file for the man pages and send it to the TP > > > coordinator? > > > > For 5.2.5, the non-GNU/Linux testing should be done first. Anything > > else than German probably won't happen quickly enough to hit 5.2.5, and > > I suppose you can run
Re: [xz-devel] Internationalization of man pages
Hello Lasse. Am Mi., 12. Feb. 2020 um 19:05 Uhr schrieb Lasse Collin : > > On 2020-02-12 Mario Blättermann wrote: > > Lasse Collin schrieb am Fr., 7. Feb. 2020, > > 15:32: > > > - po4a is never needed when building from a release tarball. > > > > This means, distribution packagers don't have to bother with po4a, > > and the translated man pages will be installed automatically (unless > > DISABLE_NLS) is set...? > > Exactly. > > > > The extra po4a options like unknown_macros=untranslated aren't > > > needed in XZ Utils. > > > > Not for now, but with this options it is better because po4a changes > > its behavior from time to time. Keeping the options would be safer. > > Hmmm, I don't know what kind of behavior changes po4a may have (or has > had), In fact, it's the standard set of options which we use in manpages-l10n to cover all imaginable problems and peculiarities in Groff and Mdoc code. If you are sure you don't need these options, just omit them. > so I may be missing something important. My thinking was: > > - groff_code=verbatim affects .de, .ie, and .if. These must be > avoided anyway so getting an error from these is a good thing. > > - unknown_macros=untranslated: I hope that once po4a understands a > macro, future versions will understand it too. Since there are no > macros unknown to po4a now, I would like to get an error if I added > such a macro. > > - untranslated="a.RE,\|": I don't understand this. It sounds like > that certain macros shouldn't be translated but it made no > difference with xz man pages. So perhaps it was useful where this > line was copied. > > I can understand if unknown_macros=untranslated is needed for > compatibility with old po4a versions, so it can be added if needed, > although I would prefer the default pedantic behavior myself. > > > > - README wasn't updated yet to mention that man pages can be > > > translated. > > > > After the new TP domain has been created (see below), you should add > > a hint there. > > I will. > > > The German translation wasn't committed yet as po.de says this: > > > > > > This file is distributed under the same license as the > > > manpages-de package. > > > > > > Which hopefully is trivial to change. :-) > > > > No problem, as the one and only author of the translation, I can > > change the license to whatever you want. > > The original is public domain so that is somewhat preferred. If you > prefer a permissive license it's OK to me, it just needs to be > mentioned in the file COPYING too. Man page translation is a large > amount of work after all. > I've created a new pot file and merged with the existing translation. The result is attached. > > > If you wish the German man page translation to be part of 5.2.5, it > > > could be good to test that "make DESTDIR=something install" also > > > works on something else than GNU/Linux (the scripting for those > > > details is ugly). The test should be done so that there are > > > translated man pages available in the po4a/man directory, e.g. by > > > using de.po from the first post of this thread. > > > > I will test it next weekend, but I only use GNU/Linux, I don't have MS > > Windows, BSD, macOS or anything else. > > Thanks. While testing on GNU/Linux is useful still, trying on some *BSD > or other POSIX system would be needed in case the makefile or shell > scripting has portability issues. Perhaps someone else can try it. > > It should work fine even when --program-prefix=foo- or such configure > options are used to get foo-xz, foo-unxz and so on. "make > DESTDIR=something uninstall" should work too, although I guess the > uninstall part is practically never needed. > > The uninstall code can remove more files than the install part > installs: if the package included German xz.1 but not German xzdec.1, > the uninstall target will still remove both if they exist. It's a minor > bug that I think is OK to leave unfixed. > This is only relevant in some odd cases when somebody installs into /usr/ which is only for the files from distribution packages. Then it could happen that a file vanishes, which actually belongs to the manpages-de package. Well, it's some kind of undesired behaviour, but users should normally know that they must not install anything in /usr/ manually. Best Regards, Mario > > BTW, what about to create a pre-release tarball which contains an > > up-to-date *.pot file for the man pages and send it to the TP > > coordinator? > > For 5.2.5, the non-GNU/Linux testing should be done first. Anything > else than German probably won't happen quickly enough to hit 5.2.5, and > I suppose you can run ./po4a/update-po from xz.git to get xz-man.pot. > > For the development branch, maybe once 5.2.5 is done. I will do it at > least when there is new code for --help. With good luck that doesn't > take too long to happen. > > > And moreover, maybe there are some changes since I've worked on the > > German translation? Not
Re: [xz-devel] Internationalization of man pages
On 2020-02-12 Mario Blättermann wrote: > Lasse Collin schrieb am Fr., 7. Feb. 2020, > 15:32: > > - po4a is never needed when building from a release tarball. > > This means, distribution packagers don't have to bother with po4a, > and the translated man pages will be installed automatically (unless > DISABLE_NLS) is set...? Exactly. > > The extra po4a options like unknown_macros=untranslated aren't > > needed in XZ Utils. > > Not for now, but with this options it is better because po4a changes > its behavior from time to time. Keeping the options would be safer. Hmmm, I don't know what kind of behavior changes po4a may have (or has had), so I may be missing something important. My thinking was: - groff_code=verbatim affects .de, .ie, and .if. These must be avoided anyway so getting an error from these is a good thing. - unknown_macros=untranslated: I hope that once po4a understands a macro, future versions will understand it too. Since there are no macros unknown to po4a now, I would like to get an error if I added such a macro. - untranslated="a.RE,\|": I don't understand this. It sounds like that certain macros shouldn't be translated but it made no difference with xz man pages. So perhaps it was useful where this line was copied. I can understand if unknown_macros=untranslated is needed for compatibility with old po4a versions, so it can be added if needed, although I would prefer the default pedantic behavior myself. > > - README wasn't updated yet to mention that man pages can be > > translated. > > After the new TP domain has been created (see below), you should add > a hint there. I will. > The German translation wasn't committed yet as po.de says this: > > > > This file is distributed under the same license as the > > manpages-de package. > > > > Which hopefully is trivial to change. :-) > > No problem, as the one and only author of the translation, I can > change the license to whatever you want. The original is public domain so that is somewhat preferred. If you prefer a permissive license it's OK to me, it just needs to be mentioned in the file COPYING too. Man page translation is a large amount of work after all. > > If you wish the German man page translation to be part of 5.2.5, it > > could be good to test that "make DESTDIR=something install" also > > works on something else than GNU/Linux (the scripting for those > > details is ugly). The test should be done so that there are > > translated man pages available in the po4a/man directory, e.g. by > > using de.po from the first post of this thread. > > I will test it next weekend, but I only use GNU/Linux, I don't have MS > Windows, BSD, macOS or anything else. Thanks. While testing on GNU/Linux is useful still, trying on some *BSD or other POSIX system would be needed in case the makefile or shell scripting has portability issues. Perhaps someone else can try it. It should work fine even when --program-prefix=foo- or such configure options are used to get foo-xz, foo-unxz and so on. "make DESTDIR=something uninstall" should work too, although I guess the uninstall part is practically never needed. The uninstall code can remove more files than the install part installs: if the package included German xz.1 but not German xzdec.1, the uninstall target will still remove both if they exist. It's a minor bug that I think is OK to leave unfixed. > BTW, what about to create a pre-release tarball which contains an > up-to-date *.pot file for the man pages and send it to the TP > coordinator? For 5.2.5, the non-GNU/Linux testing should be done first. Anything else than German probably won't happen quickly enough to hit 5.2.5, and I suppose you can run ./po4a/update-po from xz.git to get xz-man.pot. For the development branch, maybe once 5.2.5 is done. I will do it at least when there is new code for --help. With good luck that doesn't take too long to happen. > And moreover, maybe there are some changes since I've worked on the > German translation? Not in the content itself, but sometimes newer > versions of po4a introduce some other formatting. There is a tiny amount of content changed/added. > But regarding new translations: don't expect too much. I don't expect anything. The German translation was a big surprise already. > You don't have to wait a long time for new translations; as a rule of > thumb, you can sync the po files from TP with your Git two weeks > after publishing the pre-release tarball. And if new po files come in > somewhat later, they are a good basis for the next release of xz. OK. :-) -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] Internationalization of man pages
Hello Lasse, Lasse Collin schrieb am Fr., 7. Feb. 2020, 15:32: Hello! It's been a few months of silence but in the past two weeks have finally been able get something done in XZ Utils. I have committed support for translated man pages. Notes: - po4a is never needed when building from a release tarball. This means, distribution packagers don't have to bother with po4a, and the translated man pages will be installed automatically (unless DISABLE_NLS) is set...? - po4a is needed to build xz.git if one wants translated man pages. If po4a is missing, building will still work normally without translated man pages. - po4a/xz-man.pot, po4a/*.po, and the translated output in po4a/man/$lang are updated by running po4a/update-po. It is only run automatically by autogen.sh and "make mydist" (which I use to create the release tarballs). Running "make" never runs po4a/update-po. - To add a new translation, see the comment in po4a/po4a.conf. The extra po4a options like unknown_macros=untranslated aren't needed in XZ Utils. Not for now, but with this options it is better because po4a changes its behavior from time to time. Keeping the options would be safer. - It is OK if only some man pages are available for a language. Symlinks like unxz.1 -> xz.1 are created if and only if the target man page exists for the language. Yes, depending on the translation teams it can happen that a previously translated man page doesn't get built anymore because the translation doesn't reach the 80% threshold for po4a (after the original text has changed and nobody has completed the translation again). - The translated man pages aren't installed if --disable-nls is used. - README wasn't updated yet to mention that man pages can be translated. After the new TP domain has been created (see below), you should add a hint there. The German translation wasn't committed yet as po.de says this: This file is distributed under the same license as the manpages-de package. Which hopefully is trivial to change. :-) No problem, as the one and only author of the translation, I can change the license to whatever you want. The v5.2 branch should be very close to what XZ Utils 5.2.5 release will look like. The mess with new/updated translations needs to be sorted out to some extent, and then I can write NEWS and release a new version. If you wish the German man page translation to be part of 5.2.5, it could be good to test that "make DESTDIR=something install" also works on something else than GNU/Linux (the scripting for those details is ugly). The test should be done so that there are translated man pages available in the po4a/man directory, e.g. by using de.po from the first post of this thread. I will test it next weekend, but I only use GNU/Linux, I don't have MS Windows, BSD, macOS or anything else. BTW, what about to create a pre-release tarball which contains an up-to-date *.pot file for the man pages and send it to the TP coordinator? Maybe some other translators are interested in to provide localized man pages for xz... And moreover, maybe there are some changes since I've worked on the German translation? Not in the content itself, but sometimes newer versions of po4a introduce some other formatting. But regarding new translations: don't expect too much. It's a big chunk, and usually there are only a few people who are willing to translate man pages. I think Yuri Chornoivan (Ukrainian) and Rafael Fontenelle (Brazilian Portuguese) will soon start with, but for other languages, who knows... You don't have to wait a long time for new translations; as a rule of thumb, you can sync the po files from TP with your Git two weeks after publishing the pre-release tarball. And if new po files come in somewhat later, they are a good basis for the next release of xz. Best Regards, Mario
Re: [xz-devel] Internationalization of man pages
Hello! It's been a few months of silence but in the past two weeks have finally been able get something done in XZ Utils. I have committed support for translated man pages. Notes: - po4a is never needed when building from a release tarball. - po4a is needed to build xz.git if one wants translated man pages. If po4a is missing, building will still work normally without translated man pages. - po4a/xz-man.pot, po4a/*.po, and the translated output in po4a/man/$lang are updated by running po4a/update-po. It is only run automatically by autogen.sh and "make mydist" (which I use to create the release tarballs). Running "make" never runs po4a/update-po. - To add a new translation, see the comment in po4a/po4a.conf. The extra po4a options like unknown_macros=untranslated aren't needed in XZ Utils. - It is OK if only some man pages are available for a language. Symlinks like unxz.1 -> xz.1 are created if and only if the target man page exists for the language. - The translated man pages aren't installed if --disable-nls is used. - README wasn't updated yet to mention that man pages can be translated. The German translation wasn't committed yet as po.de says this: This file is distributed under the same license as the manpages-de package. Which hopefully is trivial to change. :-) The v5.2 branch should be very close to what XZ Utils 5.2.5 release will look like. The mess with new/updated translations needs to be sorted out to some extent, and then I can write NEWS and release a new version. If you wish the German man page translation to be part of 5.2.5, it could be good to test that "make DESTDIR=something install" also works on something else than GNU/Linux (the scripting for those details is ugly). The test should be done so that there are translated man pages available in the po4a/man directory, e.g. by using de.po from the first post of this thread. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] Internationalization of man pages
On 2019-06-29 Mario Blättermann wrote: > Let me summarize what I think about the next versions: Some new UI > translations have arrived which would justify a bugfix release > v5.2.5. All other things (re-wrapping the critical strings and man > page translations) should wait until the master branch is ready > for a new release (v5.4). Thanks. I need to look at 5.2.5 soon. I plan to include only those new translations that look good enough in sense of word wrapping and alignments. Since there might be auto-word-wrapping in a later version, I won't spend time discusing with translators to get the problems fixed in v5.2 strings. I hope this plan isn't too rude. About automatic word wrapping: I was about to commit a version but noticed that gettext tools don't like what I did as I got a lot of warnings like this: warning: internationalized messages should not contain the '\v' escape sequence I repurposed the standard C control character escapes. It was slightly simpler than using a separate escape character in the string, but perhaps that was a stupid idea. Looking at gettext source, those control chars seem to be fully supported along with octal escapes. It's just that five standard escapes give a warning which cannot be silenced. But perhaps po-editors don't like them or warn if a translation contains a different number of those characters, I don't know, but perhaps you or someone else know or can test. I uploaded the patch here, it applies on top of xz.git master: https://tukaani.org/temp/0001-xz-Add-automatic-word-wrapping-for-long-help.patch The word wrapping is documented starting on line 118. The docs could perhaps be better but I hope the idea is simple enough from translators' point of view. If it's not, I don't know if word wrapping is realistic as there has to be some sort of ultra-simple markup to make it practical (the escape codes can be changed if really need). I still haven't fully convinced myself that this kind of word wrapping is a great idea but on the other hand it would be really nice if this could solve the line length and alignment problems for good. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] Internationalization of man pages
Hello Lasse, Am Fr., 28. Juni 2019 um 13:02 Uhr schrieb Lasse Collin : > > On 2019-06-26 Lasse Collin wrote: > > The overlong lines in --help simply should be fixed by the > > translators. In the near future xz won't have auto-wrap support for > > --help strings. > > I will reconsider this. It might be less painful to wrap multibyte > strings in portable C than I had assumed (excluding languages that don't > use spaces). So perhaps there might be some support for auto-wrapped > --help in the near future (but not in 5.2.5) after all. I try to get > back to this next week. > Let me summarize what I think about the next versions: Some new UI translations have arrived which would justify a bugfix release v5.2.5. All other things (re-wrapping the critical strings and man page translations) should wait until the master branch is ready for a new release (v5.4). Best Regards, Mario
Re: [xz-devel] Internationalization of man pages
On 2019-06-23 Mario Blättermann wrote: > Am Sa., 22. Juni 2019 um 23:53 Uhr schrieb Lasse Collin > : > > - Is xzdec-man.pot intentionally there or should it be part of > > xz-man.pot? > > > It's an artifact from a separate creation of a pot file for xzdec.1. > Can be safely removed, > its content is already in xz-man.pot. OK. :-) > For my understanding, a packager should either ship the package with > complete translations (both UI and man pages) or no translations at > all (using "configure --disable-nls"). Yes. It's also easy to add a separate --enable-translated-man-pages if needed. > As far as I can see, sufficiently current packages are available for > all mainstream distributions [2]. Portability problems tend to occur outside mainstream distributions (often non-free OSes) and there one can be happy if the GNU Autotools are available easily enough. It's those cases where extras like po4a could be annoying if one is trying to build xz.git instead of relying on a pre-created source package. This shouldn't be hard to handle, that is, if po4a is missing and one builds xz.git, one might not get translated man pages but the build otherwise works. Another important thing is to ensure that official tarballs won't accidentally get released without translated man pages. > Have a look at [3]. This is an attempt to get the man pages of the > Psmisc tools translatable. Thanks. My Automake skills have gotten rusty so that gives some ideas to me; at the same time I see things in the patch that don't look so right. The interesting thing about the patch is how it runs "$(MAKE) install-man" with overriden variables. *If* that is a safe thing to do (i.e. if it doesn't count as too bad abuse of Automake internals) it simplifies the installation part nicely. > > For example, I wonder if for > > each translated xz.1 there needs to be a unxz.1 and xzcat.1 (as > > symlinks) too like there are for English versions. > > > The symlinks are also needed for all languages, otherwise calling > "man unxz.1" would lead to the English version of xz.1. I'm aware. I was hoping to be lazy. ;-) > > > A German translation (de.po) is already included. > > > > Wow, even xz.1 is there, translating it is a lot of work. > > > I've translated the man pages over the last years, step by step, and > xz.1 was the last and biggest step. :-) Since you must have read the man page carefully, I wonder if you have wishes or suggestion how to improve the original English version. > Do you need some help with proofreading and testing the translations > from TP? In some form maybe but I don't know what exactly I want. We discussed in private email about the problems in many (not all!) new translations: - Over-long lines in --help and --long-help - Column heading alignment issues in --list --verbose and other places The alignment issues I promised to fix it in the master branch with code that splits each heading name into a separate string and the alignment is handled in code. I'm hesitant to include it in the v5.2 branch and xz 5.2.5 since v5.2 should get just bug fixes to reduce the chance of regressions. So something should be done if new translations are wanted in 5.2.5 but I didn't decide what that something should be. The overlong lines in --help simply should be fixed by the translators. In the near future xz won't have auto-wrap support for --help strings. I know that it would help translators but it's non-trivial code and I don't want to introduce external dependencies just for that. Some GNU packages use GNU argp functions which provide auto-wrap but it's broken for multibyte locales because the code assumes 1 byte = 1 character = 1 terminal-column. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
