[bug-gettext] [bug #27740] Feature Request: Support Here Doc & Now Doc syntaxes for PHP in xgettext

[Daiki Ueno]

Update of bug #27740 (project gettext):

  Status:   Fixed => None   
 Open/Closed:  Closed => Open   

___

Follow-up Comment #5:

Reopening.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #52159] msgfmt don't correctly build description field in appdata.xml files

[Daiki Ueno]

Follow-up Comment #1, bug #52159 (project gettext):

I would suggest to keep this under XML, because it is basically about that
msgfmt doesn't implement the "merge" operation of itstool, but only the "join"
operation:
http://itstool.org/documentation/basic-usage.html

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] Bug#876498: gettext: msgunfmt: heap corruption

[Daiki Ueno]

Jakub Wilk  writes:

> Package: gettext
> Version: 0.19.8.1-4
>
> msgunfmt crashes on the attached file:
>
>   $ zcat bad.mo.gz | msgunfmt
>   *** Error in `msgunfmt': corrupted size vs. prev_size: 0x57b0abf0 ***
>   ...
>   Aborted
>
> Unhelpful backtrace:

Running msgunfmt under valgrind might give you more hints.  Anyway, I am
suspecting this is caused by a missing NUL termination in
get_sysdep_string in read-mo.c, which should be fixed by the attached patch.

Regards,
-- 
Daiki Ueno
>From 3c66e050e344ec890f0c1e467753c2ed46bc7bb8 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Sat, 23 Sep 2017 18:09:33 +0200
Subject: [PATCH] msgunfmt: Avoid heap buffer overrun

* gettext-tools/src/read-mo.c (get_sysdep_string): NUL-terminate the result.
* gettext-tools/tests/msgunfmt-3: Check no-nul-sysdep.mo.
* gettext-tools/tests/no-nul-sysdep.mo: New test data.
Reported by Jakub Wilk in:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876498
---
 gettext-tools/src/read-mo.c  |   3 ++-
 gettext-tools/tests/msgunfmt-3   |   4 ++--
 gettext-tools/tests/no-nul-sysdep.mo | Bin 0 -> 2805 bytes
 3 files changed, 4 insertions(+), 3 deletions(-)
 create mode 100644 gettext-tools/tests/no-nul-sysdep.mo

diff --git a/gettext-tools/src/read-mo.c b/gettext-tools/src/read-mo.c
index 9ddd6b2d2..33d7a5828 100644
--- a/gettext-tools/src/read-mo.c
+++ b/gettext-tools/src/read-mo.c
@@ -194,7 +194,8 @@ get_sysdep_string (const struct binary_mo_file *bfp, size_t offset,
 }
 
   /* Allocate and fill the string.  */
-  string = XNMALLOC (length, char);
+  string = XNMALLOC (length + 1, char);
+  string[length] = '\0';
   p = string;
   s_offset = get_uint32 (bfp, offset);
   for (i = 4; ; i += 8)
diff --git a/gettext-tools/tests/msgunfmt-3 b/gettext-tools/tests/msgunfmt-3
index 42dc1cc55..3d06d1c52 100755
--- a/gettext-tools/tests/msgunfmt-3
+++ b/gettext-tools/tests/msgunfmt-3
@@ -5,8 +5,8 @@
 
 : ${MSGUNFMT=msgunfmt}
 
-for n in 1 2 3 4 5 6; do
-  LANGUAGE= LC_ALL=C ${MSGUNFMT} "$abs_srcdir"/overflow-$n.mo 2>mu-3.err >/dev/null
+for f in "$abs_srcdir"/overflow-*.mo "$abs_srcdir"/no-nul-sysdep.mo; do
+  LANGUAGE= LC_ALL=C ${MSGUNFMT} $f 2>mu-3.err >/dev/null
   test $? != 0 || Exit 1
   grep ' is truncated' mu-3.err >/dev/null || Exit 1
 done
diff --git a/gettext-tools/tests/no-nul-sysdep.mo b/gettext-tools/tests/no-nul-sysdep.mo
new file mode 100644
index ..6bcaa510535cc77b4b1bd48ecad9741bd4549021
GIT binary patch
literal 2805
zcmca7#4^>ufB_5);D7

Re: [bug-gettext] Debian Bug#872869: msgfmt: trailing null bytes in header's msgstr

[Daiki Ueno]

Bruno Haible  writes:

>>   $ msgunfmt /usr/share/locale/pl/LC_MESSAGES/gettext-runtime.mo | msgfmt -
>>   $ i18nspector messages.mo
>>   E: messages.mo: invalid-mo-file unexpected null byte in msgstr
>
> Thanks for this report! Fixed through
> http://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commitdiff;h=2bad4d89684303fe884410ab0ae53770df6a6093

Although it was reported against an unreleased version, can we have a
test case for this to prevent similar issue happening in the future?

Regards,
-- 
Daiki Ueno
>From 01d82c6ed523b0c44ad4ae4e2935e4d0072abb86 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Tue, 5 Sep 2017 16:14:03 +0200
Subject: [PATCH] Add test for "Avoid extraneous NUL bytes in .mo files" change

* gettext-tools/src/read-mo.c: Include "msgunfmt.h".
(get_string): Print warning if a string is followed by an extraneous
NUL character.
* gettext-tools/tests/msgfmt-19: Check if extraneous NUL character is
not included in the output.
---
 gettext-tools/src/read-mo.c   | 5 +
 gettext-tools/tests/msgfmt-19 | 5 +
 2 files changed, 10 insertions(+)

diff --git a/gettext-tools/src/read-mo.c b/gettext-tools/src/read-mo.c
index 9ddd6b2d2..37d7ca291 100644
--- a/gettext-tools/src/read-mo.c
+++ b/gettext-tools/src/read-mo.c
@@ -29,6 +29,8 @@
 #include 
 #include 
 
+#include "msgunfmt.h"
+
 /* This include file describes the main part of binary .mo format.  */
 #include "gmo.h"
 
@@ -131,6 +133,9 @@ get_string (const struct binary_mo_file *bfp, size_t offset, size_t *lengthp)
 error (EXIT_FAILURE, 0,
_("file \"%s\" contains a not NUL terminated string"),
bfp->filename);
+  if (s_length > 0 && bfp->data[s_offset + s_length - 1] == '\0' && verbose)
+error (0, 0, _("file \"%s\" contains an extraneous NUL character"),
+   bfp->filename);
 
   *lengthp = s_length + 1;
   return bfp->data + s_offset;
diff --git a/gettext-tools/tests/msgfmt-19 b/gettext-tools/tests/msgfmt-19
index 76d4a739d..043456332 100755
--- a/gettext-tools/tests/msgfmt-19
+++ b/gettext-tools/tests/msgfmt-19
@@ -56,6 +56,11 @@ EOF
 ${MSGFMT} -o mf-19-1.mo mf-19-1.po 2>/dev/null
 test $? = 0 || { Exit 1; }
 
+: ${MSGUNFMT=msgunfmt}
+LC_ALL=C \
+${MSGUNFMT} -v -o mf-19-1.tmp mf-19-1.mo 2> mf-19.err > /dev/null || Exit 1
+grep -v 'contains an extraneous NUL character' mf-19.err && Exit 1
+
 : ${MSGFMT=msgfmt}
 ${MSGFMT} -o mf-19-2.mo mf-19-2.po 2>/dev/null
 test $? = 0 || { Exit 1; }
-- 
2.13.5

Re: [bug-gettext] replacing autotools with meson?

[Daiki Ueno]

Hello,

Bruno Haible  writes:

>> The larger reason is that meson as build system is a lot more pleasant 
>> to deal with, and many projects, particularly those affiliated with 
>> Gnome are switching to it. Gstreamer is perhaps the most well known.
>
> Well, I have seen several attempts at providing a good build system:
> SCons, CMake, quagmire, ninja, ... and none of them so far was attractive
> to the entire GNU community. (Although I do concede that ninja looks
> promising, because it follows the philosophy of "do one thing and do it
> right".)
>
> Gettext's build system is the GNU build system, because we want every
> GNU maintainer to be able to get familiar with Gettext's code when necessary.

I agree with this, but we don't need to switch over to the new build
system completely.  Several projects keep the GNU build system along
with the meson build system:

https://github.com/GStreamer/gstreamer
https://github.com/xkbcommon/libxkbcommon

Of course, it could be a burden if one had to maintain two build systems
and some might drop either of them in the future.

Personally I am interested in how it could help reduce the build time.
If anyone experiments it and provides the benchmark result, that would
be nice.

>> The immediate reason is that getting to the point where gettext can be 
>> built takes an unusually long time: several minutes, and for no good 
>> reason that I can see, just lots of trivial tests like whether 
>> definitions are macros etc.
>
> The main reason that it takes a long time is that it is composed of
> several directories, each doing its own configuration. Maybe you can
> speed it up by passing a --cache option that points to an (initially
> empty) file?

I remember someone proposed to switch to non-recursive make and
eliminate AC_CONFIG_SUBDIRS calls for performance reasons.  I suggested
the same and he told that it didn't help much.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] libpeas Plugin file support

[Daiki Ueno]

Iñigo Martínez  writes:

> Actually I wasn't aware of the issue of the plugin files translation.
> I came across it by chance when trying to port totem to meson[0]. Once
> I figured out what was going on, I tried to solve the issue also
> considering the options you mentioned[1], but I ended thinking about
> those different options as workarounds for the real solution. Being
> aware of the impact this issue might have considering the number of
> applications that would benefit from a proper solution, I decided to
> implement it.

I totally agree, though I would like to avoid the case where similar
treatment is required for other .desktop file variants.

Perhaps more generic approach would be to implement "per-file options"
feature in xgettext[1], which would allow user to specify different
options for each file within the same po directory.

Footnotes: 
[1]  http://lists.gnu.org/archive/html/bug-gettext/2015-06/msg6.html

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] libpeas Plugin file support

[Daiki Ueno]

Hello,

Iñigo Martínez  writes:

> Following the last mail, you can find attached three patches that add
> full support for libpeas plugin files support.

Thank you for the patches.  While the implementation looks sensible to
me, I am wondering if it could be worked around by using a separate po
directory for plugins, in a similar way gtk+ does for GObject property
blurbs:
https://git.gnome.org/browse/gtk+/tree/po-properties

Maybe you could have a po-plugins directory with:

  XGETTEXT_OPTIONS = -kName -kDescription ...

in Makevars and *.plugins in POTFILES.in.

This way, you wouldn't need to wait another gettext release.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #46111] ECMAScript templates are not extracted

[Daiki Ueno]

Update of bug #46111 (project gettext):

  Status:None => Duplicate  
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

Closing this as duplicate of:
https://savannah.gnu.org/bugs/?50920
where we seem to have practical use-cases.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #50408] AppData’s doesn’t get translated

[Daiki Ueno]

Follow-up Comment #2, bug #50408 (project gettext):

The gettext-provided file is solely for compatibility reasons: it is there not
to break the applications that expect the previous behavior, implementation in
C.

Shouldn't we suggest to install appstream instead?

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #49955] xgettext parses wrongly hex escapes in concatenated literals

[Daiki Ueno]

Follow-up Comment #1, bug #49955 (project gettext):

I think this is a duplicate of:
https://savannah.gnu.org/bugs/?46756

Thanks for the report and the analysis, anyway.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] JavaScript support status

[Daiki Ueno]

Hello,

Bruno Haible  writes:

> I'm trying to get an understanding of:
>   - What is the JavaScript community doing w.r.t. localization and PO and MO
> files?
>   - What can/should the GNU gettext package do in order to be useful for this
> community?
>
> Here's a set of packages and use situations I could find. It does not give a
> big picture to me. Can someone shed some light on anything of this, and
> clarify?

I doubt it would be useful to support for particular Web frameworks;
they still shoot up like mushrooms after a rain.

On the other hand, it would be useful to add a minimal xgettext support
for JSON, which is considered stable, through a generic mechanism like
ITS:
https://savannah.gnu.org/bugs/?48903#comment1

> - gjs (command-line, but linked to Mozilla's JavaScript) [who uses that?]

gjs is extensively used by GNOME; you can see the list of such modules
(search with "JavaScript"):
https://git.gnome.org/browse/

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #49809] xgettext cannot parse "\'"

[Daiki Ueno]

Update of bug #49809 (project gettext):

  Status:None => Need Info  

___

Follow-up Comment #1:

Sorry, you provide too little information.  I can't reproduce it with:


cat > a.sh
sed -e s/\'//g" or "tr -d \'
^D

xgettext -o - a.sh


We would at least need:

- What version of gettext did you use?
- What command line did you use?
- What behavior did you expect?
- What behavior did you actually see (what do you mean by "bail out")?

Thank you,

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [PATCH] intl: Fix that /@unixroot prefix is not working on OS/2 kLIBC

[Daiki Ueno]

KO Myung-Hun  writes:

> OS/2 kLIBC has a feature to rewrite some path components. For example,
> '/@unixroot' is replaced with a value of $UNIXROOT if it is.
>
> So prepending a drive letter to the path starting with '/' makes the
> path starting with '/@unixroot' to 'x:/@unixroot' which is unexpected.
>
> This will breaks the behavior of some programs depending on /@unixroot
> prefix.
>
> * gettext-runtime/intl/bindtextdom.c (BINDTEXTDOMAIN): Do not touch
> dirname if it is started with '/@unixroot'.
> * gettext-runtime/intl/relocatable.c (relocate): Do not touch pathname
> if it is started with '/@unixroot'.
> ---
>  gettext-runtime/intl/bindtextdom.c |  6 ++
>  gettext-runtime/intl/relocatable.c | 12 ++++
>  2 files changed, 18 insertions(+)

Thanks for the patch.  I have pushed it.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #49540] FAIL xgettext-its-2

[Daiki Ueno]

Update of bug #49540 (project gettext):

  Status:None => Fixed  
 Assigned to:None => haible 
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

Bruno pushed the fix:
http://lists.gnu.org/archive/html/bug-gettext/2016-12/msg2.html

Thanks.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] bug tracker: savannah vs. debbugs

[Daiki Ueno]

Bruno Haible  writes:

>> On a related note, I thought it might
>> make sense if we eventually move over to debbugs.gnu.org, as automake
>> and libtool do.  Though I haven't used it much, it apparently has a
>> concept of "user tags" which could be used as categories.
>
> My preference goes the other way around: I do prefer the Savannah tracker
> over debbugs.
>
> Rationale:
>
> 1) Generally speaking, web-based tools with a "comment"/discussion facility
> are more efficient to use than email-based tools when the discussion remains
> self-contained (not connected to other issues).
> Only when issues are not well understood, or when an issue is composed of
> several sub-issues, then email is better. Because only then there is a need
> for mails to be forwarded to different mailing lists or for threads to be
> split.
>
> 2) More in detail: When working with mail, a comment on lists.gnu.org and
> in my mailbox are two different things, but they represent the same entity.
> Therefore I need to spend time looking up the lists.gnu.org URL of a
> particular
> comment, as a reference. With debbugs the conversion is a bit simpler:
> xx...@debbugs.gnu.org <-->
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=x
> but it's still a conversion that's needed in many cases anyway.
>
> Sometimes also I don't have the mail in my mailbox, so I have to copy&paste
> from the lists.gnu.org page. All this wastes time.
>
> Also, with debbugs you have to spend time setting up the "To" and "CC" list.
> (Be careful not to omit the debbugs CC.)
>
> When working with mail, I have to move every mail to archive manually. So,
> send a mail to change a status AND manage my mailbox accordingly. With a
> web-based interface, I only have to change the status.
>
> Whereas in a web-based tracker, an issue is represented by one URL - there's
> no need to convert it to/from an email address, and in most cases it's just
> a hyperlink that I can click on.
>
> 3) The Savannah tracker has extensive per-project customizations. I fear
> that with debbugs less per-project customizations are available.
>
> 4) Submitting a bug through a web-based tool is easier for most people than
> through debbugs. IMO the barrier to giving feedback should be low.
>
> 5) debbugs has not many advocates: you can see in
> https://debbugs.gnu.org/Packages.html
> that aside from Jim, Paul, and Ludovic, it's mostly only Emacs, Automake,
> and Libtool that use it.
>
> There are not many advantages of debbugs:
>
> 1) It has a full-text search engine.

2) It can be accessed through the Emacs interface:
http://elpa.gnu.org/packages/debbugs.html

It works pretty nicely and could address some of your concerns.

I actually don't have strong opinion here.  However, having seen quite a
few people being confused that we have multiple places to report bugs
(this list and the tracker) and that discussions often scatter across
those places, I am wondering if we could unify them.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] categories in bug tracker

[Daiki Ueno]

Bruno Haible  writes:

> The bugs in our bug tracker https://savannah.gnu.org/bugs/?group=gettext
> can now be assigned to one of several "categories". The categories that I
> defined are:
[...]
> I did this because I find it comfortable to work on a group of bugs in the
> same area at once, and because some categories of bugs have higher priority
> than others. Maybe you feel the same way...

Thank you, that would be useful.  On a related note, I thought it might
make sense if we eventually move over to debbugs.gnu.org, as automake
and libtool do.  Though I haven't used it much, it apparently has a
concept of "user tags" which could be used as categories.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #49678] xgettext fails to extract strings from Perl code

[Daiki Ueno]

Update of bug #49678 (project gettext):

 Assigned to:None => gflohr 

___

Follow-up Comment #1:

Guido, could you take a look at this?

It might already have been fixed with your recent change.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #49677] xgettext fails to extract strings from Perl code

[Daiki Ueno]

Update of bug #49677 (project gettext):

  Status:None => Duplicate  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

This apparently is a duplicate of https://savannah.gnu.org/bugs/?49678. 
Closing.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #49654] msgfmt and build determinism

[Daiki Ueno]

Follow-up Comment #4, bug #49654 (project gettext):

I too fully agree with Bruno's suggestion.  It sounds like a good idea to
strip the timestamp when producing .mo files.  Bruno, are you willing to
provide a patch for this?

On the other hand, I think it could also be worked around by disabling the
auto-msgmerge rule in po/Makefile.in.in, as I did in p11-kit:
https://github.com/p11-glue/p11-kit/commit/4965a8b2f150ea6c8dadd7dd22aab718f2814591

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #49598] Allow ngettext to be used with non-English source language with more than two plural forms

[Daiki Ueno]

Update of bug #49598 (project gettext):

  Status:None => Not a Bug  
 Assigned to:None => haible 
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

Thanks for the follow up, Bruno.  Closing.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #49497] gettext needs automake 1.13

[Daiki Ueno]

Follow-up Comment #2, bug #49497 (project gettext):

> It'd be nice if `autogen.sh` would fail with a verbose error message and
give a hint that automake 1.13 needs to be installed rather than just complain
about the missing macro.

I thought it would be achieved by the AM_INIT_AUTOMAKE invocation in
configure.ac:
  AM_INIT_AUTOMAKE([1.13 silent-rules parallel-tests dist-xz dist-lzip])

Perhaps it might not work with older automake?

> IMO these comments should be moved to HACKING. And, if necessary, add the
required version numbers there.

Good idea, done:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=f837369264c119d0e96c9ba6e5960deba59eee73

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] What is tmp_dirname ?

[Daiki Ueno]

Egor Pugin  writes:

> I'm trying to build gettext on windows.
> On the line below I see  'tmp_dirname'. Where it comes from? And what
> value should it have? '/tmp' but for win32?
> http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/dcigettext.c#n637

I'm not sure but it looks like a typo: s/tmp_dirname/resolved_dirname/.
Do you see a compile error caused by this?

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] xgettext: input PO files w/duplicate plural forms cause core dump

[Daiki Ueno]

Hello,

Davlet Panech  writes:

> I'm getting a core dump in xgettext whenever I try to process PO files
> that contain duplicate plural form entries. Looks like a double-free()
> somewhere:

Thank you for reporting that.  I have pushed a fix as:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=dce3a16e

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #48921] Get po-mode.el ready for packaing

[Daiki Ueno]

Update of bug #48921 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Thanks, merged the patches.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48903] xgettext: Add support for ClutterScript

[Daiki Ueno]

Follow-up Comment #1, bug #48903 (project gettext):

While the patch looks reasonable, I would like to see JSON+ITS being
implemented at some point:
https://www.w3.org/International/its/wiki/JSON%2BITS

Do you know if there is any JSONPath implementation for json-c?

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48729] gettextize should not add config.rpath to EXTRA_DIST

[Daiki Ueno]

Update of bug #48729 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 
 Summary: gettetize should not add config.rpath to EXTRA_DIST
=> gettextize should not add config.rpath to EXTRA_DIST

___

Follow-up Comment #1:

Thanks, that makes sense.  The file is actually included in tarball
automatically.

I have pushed a fix:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=6445c3ad2573d2d8a29953a7a791e3ec460c675a

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48481] [PATCH] Allow CLDR plural rules lookup fallback

[Daiki Ueno]

Follow-up Comment #6, bug #48481 (project gettext):

Just adding some data points, you might want to look at the recent efforts to
adopt CLDR script names in glibc:
https://sourceware.org/ml/libc-locales/2016-q2/msg00034.html
and some *BSD variants:
https://wiki.freebsd.org/LocaleNewApproach

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] po-mode and emacs 25

[Daiki Ueno]

Göran Uddeborg  writes:

> After upgrading to emacs 25 (emacs-25.0.94-1.fc24.x86_64 on Fedora)
> the po-send-mail function in po-mode stopped working.  It creates a
> mail buffer, but it then base64 encodes the entire buffer, including
> the header.  Not surprisingly, the mail functions doesn't know what to
> do with that afterwards.

I can reproduce it on Emacs 25.  On the other hand, I doubt that it
worked as expected on earlier Emacs versions.  On Emacs 24, I got "The
mark is not set now, so there is no region" error and the content is not
uuencoded.

> I've tracked this down to the use of "(region-beginning)" near line
> 3525 in po-mode.el as included in emacs-gettext-0.19.8.1-1.fc24.  I
> guess something that used to set the mark in previous emacs version
> doesn't do so any more.  Or something like that.  I tried to change
> the save-excursion part to use the code below instead, and that seems
> to work.  It would be one possible way to do it, I'm not too
> proficient in emacs lisp, som I'm not sure if it is the preferred one.
>
> (save-excursion
>   (let
>   ((beginning (point)))
> (insert-buffer-substring buffer)
> (shell-command-on-region
>  beginning (region-end)
>  (concat po-gzip-uuencode-command " " name ".gz") t t)))

Thank you for tracking it down.  I would rather avoid the use of marker
at all, like the attached patch.

Regards,
-- 
Daiki Ueno
>From a0ecec28cdc4a7a69e499077ca62a0fee4180729 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Wed, 3 Aug 2016 14:10:13 +0200
Subject: [PATCH] po-mode: Fix po-send-mail behaviour on Emacs 25
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gettext-tools/misc/po-mode.el (po-send-mail): Don't rely on
region markers.
Reported by Göran Uddeborg in:
http://lists.gnu.org/archive/html/bug-gettext/2016-07/msg00027.html
---
 gettext-tools/misc/po-mode.el | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/gettext-tools/misc/po-mode.el b/gettext-tools/misc/po-mode.el
index 9189cb3..b0eed21 100644
--- a/gettext-tools/misc/po-mode.el
+++ b/gettext-tools/misc/po-mode.el
@@ -3518,10 +3518,12 @@ Write to your team?  ('n' if writing to the Translation Project robot) ")))
 (re-search-forward
  (concat "^" (regexp-quote mail-header-separator) "\n"))
 (save-excursion
-  (insert-buffer-substring buffer)
-  (shell-command-on-region
-   (region-beginning) (region-end)
-   (concat po-gzip-uuencode-command " " name ".gz") t t))
+  (save-restriction
+(narrow-to-region (point) (point))
+(insert-buffer-substring buffer)
+(shell-command-on-region
+ (point-min) (point-max)
+ (concat po-gzip-uuencode-command " " name ".gz") t t)))
   (message ""))
 
 (defun po-confirm-and-quit ()
-- 
2.7.4

[bug-gettext] [bug #48684] [PATCH] Search for 'parent' ids too when looking for plural rules

[Daiki Ueno]

Update of bug #48684 (project gettext):

  Status:None => Duplicate  
 Open/Closed:Open => Closed 


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48481] [PATCH] Allow CLDR plural rules lookup fallback

[Daiki Ueno]

Follow-up Comment #3, bug #48481 (project gettext):

Yes, that's what I understand from your code, and I wonder if it is a
reasonable behaviour; shouldn't it signal an error or stop processing when it
sees a '@' twice?

By the way, I think there should be a reliable conversion between CLDR locale
names and gettext locale names.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48481] [PATCH] Allow CLDR plural rules lookup fallback

[Daiki Ueno]

Follow-up Comment #1, bug #48481 (project gettext):

Thank you for the patch, it makes sense.

Regarding the implementation, though I haven't tested the patch, I wonder if
it accepts bogus locales like: foo@bar@baz?  Also doesn't it stop at the
second '-' if the locale looks like: it-IT.UTF-8?

I would like to make it more strictly conforming to the existing logic:
http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/finddomain.c#n71

Would you mind checking the above cases, and if possible add a similar comment
there?

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48405] Can't compile gettext 0.19.8.1 for Windows

[Daiki Ueno]

Follow-up Comment #5, bug #48405 (project gettext):

Thanks for tracking it down.  I have updated the gnulib submodule to the
latest so the next release will hopefully fix the issue:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=a6f88433b397cbcbd226457fcf9791790291e63e

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48462] [PATCH] Better description of GETTEXTCLDRDIR for msginit

[Daiki Ueno]

Update of bug #48462 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Thanks, applied the patch.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [PATCH] Add support for msgmerge --previous

[Daiki Ueno]

Stanislav Brabec  writes:

> msgmerge --previous is a very useful feature that makes adjusting of
> translation much easier when small changes in the source code are done.
> msgmerge supports it for 10 years, but it is not used by most projects,
> as Makefile.in.in never added support for it.
>
> Use msgmerge --previous as default on all systems with gettext >= 0.16.
>
> ---
>  NEWS  |  3 +++
>  gettext-runtime/po/Makefile.in.in | 16 ++--
>  2 files changed, 13 insertions(+), 6 deletions(-)

Thanks, applied.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [PATCH] Parallel build fixes

[Daiki Ueno]

Hello,

Thanks for the patch, and sorry for not responding earlier.

Andrew Stormont  writes:

> diff --git a/gettext-tools/doc/Makefile.am b/gettext-tools/doc/Makefile.am
> index 3768dbe..a3e1e72 100644
> --- a/gettext-tools/doc/Makefile.am
> +++ b/gettext-tools/doc/Makefile.am
> @@ -69,21 +69,7 @@ iso-3166.texi: ISO_3166 iso-3166.sed
>   mv iso-3166.tmp $(srcdir)/iso-3166.texi
>  
>  # The dependencies of stamp-vti generated by automake are incomplete.
> -# So we have to duplicate the entire rule which would otherwise be generated
> -# by automake.

I was pondering on this part; is the above comment no longer true with
the latest automake?  If so, do you know which automake version fixed
the issue?

>  po-gram-gen.h: po-gram-gen.c
> +cldr-plural.h: cldr-plural.c

This part looks correct.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #48384] Memory error in desktop_parse

[Daiki Ueno]

Update of bug #48384 (project gettext):

  Status:None => Duplicate  
 Open/Closed:Open => Closed 


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #48384] Memory error in desktop_parse

[Daiki Ueno]

Follow-up Comment #1, bug #48384 (project gettext):

Which gettext version did you use for testing?  The line numbers shown in the
log indicates that you are still using 0.19.7 or earlier.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] musl + gettext 0.19.8.1

[Daiki Ueno]

Hello,

Kylie McClain  writes:

> I was under the impression that with the new gettext version, gettext
> should be properly detected as being in the libc now; however, even upon
> regenerating ./configure scripts of projects using the autoconf gettext 
> macros,
> the detection doesn't get updated and it still gets detected as not being in
> the libc, as with previous gettext versions. Is there something I'm doing 
> wrong?

I suspect that your copied gettext m4 files are overwritten by
autoreconf, when regenerating ./configure.  To avoid that, you could do:

- skip the autopoint invocation by autoreconf, by:
  AUTOPOINT=true ./autoreconf ..., or
- update AM_GNU_GETTEXT_VERSION to 0.19.8, either by hand or by
  gettextize

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #48233] libintl still built even when passed --without-included-gettext

[Daiki Ueno]

Follow-up Comment #1, bug #48233 (project gettext):

I haven't looked into the detail, but I suspect that this is intentional so
that we can test the included libintl functionality in our test suite:
http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-tools/tests/gettext-3-prg.c#n28

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] gettext-0.19.8.1 released

[Daiki Ueno]

This is the announcement of a new release of GNU gettext.

GNU gettext allows programs to produce messages in the user's native
language. It consists of
  - runtime libraries for C, C++, Java, C#, Shell programs.
  - tools for programmers and translators.

This release fixes unintentional soname bumps in the included libraries
in the 0.19.8 release.  Sorry for the inconvinience.

Here are the compressed sources:
  ftp://ftp.gnu.org/gettext/gettext-0.19.8.1.tar.gz   (19MB)
  ftp://ftp.gnu.org/gettext/gettext-0.19.8.1.tar.lz   (6.4MB)
  ftp://ftp.gnu.org/gettext/gettext-0.19.8.1.tar.xz   (6.9MB)

Here are the GPG detached signatures[*]:
  ftp://ftp.gnu.org/gettext/gettext-0.19.8.1.tar.gz.sig
  ftp://ftp.gnu.org/gettext/gettext-0.19.8.1.tar.lz.sig
  ftp://ftp.gnu.org/gettext/gettext-0.19.8.1.tar.xz.sig

Use a mirror for higher download bandwidth:
  http://www.gnu.org/order/ftp.html

Here are the MD5 and SHA1 checksums:

97e034cf8ce5ba73a28ff6c3c0638092  gettext-0.19.8.1.tar.gz
d838d2c4144261d0c5fbab4a0aceb5c1  gettext-0.19.8.1.tar.lz
df3f5690eaa30fd228537b00cb7b7590  gettext-0.19.8.1.tar.xz
b5d24ba2958c91fc5cc0058165837c99a0f58784  gettext-0.19.8.1.tar.gz
404e072c455f79be4a2458863c19fb55a217771e  gettext-0.19.8.1.tar.lz
e0fe90ede22f7f16bbde7bdea791a835f2773fc9  gettext-0.19.8.1.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify gettext-0.19.8.1.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys D7E69871

and rerun the 'gpg --verify' command.

Regards,
-- 
Daiki Ueno


signature.asc
Description: PGP signature

Re: [bug-gettext] gettext from Git fails on Solaris

[Daiki Ueno]

Dagobert Michelsen  writes:

> Cannot open load file: No such file or directory, bytecomp

I too encountered this while building the RC version on a machine in the
opencsw build farm, but ignored it because it seemed like a
site-specific issue.

I even get:

  Error in post-command-hook (global-font-lock-mode-check-buffers): (file-error 
"Cannot open load file" "no such file or directory" "time-date")

when launching Emacs with "emacs -Q" there.

Regards,
-- 
Daiki Ueno

[bug-gettext] gettext-0.19.8 released

[Daiki Ueno]

This is the announcement of a new release of GNU gettext.

GNU gettext allows programs to produce messages in the user's native
language. It consists of
  - runtime libraries for C, C++, Java, C#, Shell programs.
  - tools for programmers and translators.

Here are the compressed sources:
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.8.tar.gz   (20MB)
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.8.tar.lz   (6.3MB)
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.8.tar.xz   (6.9MB)

Here are the GPG detached signatures[*]:
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.8.tar.gz.sig
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.8.tar.lz.sig
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.8.tar.xz.sig

Use a mirror for higher download bandwidth:
  http://www.gnu.org/order/ftp.html

Here are the MD5 and SHA1 checksums:

7ad5c90e32ac6828de955a0432ab4a7c  gettext-0.19.8.tar.gz
e83becf5dc4c29519a5ec5335d769764  gettext-0.19.8.tar.lz
3e7ed1ac886b9b1a4e23e71113da6358  gettext-0.19.8.tar.xz
4b3dc947a729fac8a14bde473454730919382b23  gettext-0.19.8.tar.gz
3d6ffc59141cef7ba7d7a4415da5214091164a3a  gettext-0.19.8.tar.lz
dc551d4783edf691c1f0095ca927d3128b5093e8  gettext-0.19.8.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify gettext-0.19.8.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys D7E69871

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.15
  Libtool 2.4.6
  Bison 3.0.4
  Gnulib v0.1-778-g6f9206d

Noteworthy changes in 0.19.8:

* Support for reproducible builds:
  - msgfmt now produces little-endian .mo files by default.

* Programming languages support:
  - XML:
xgettext and msgfmt now look for .its files in directories
supplied through the GETTEXTDATADIRS or XDG_DATA_DIRS environment
variable.
  - JavaScript:
xgettext and msgfmt now recognize numbered arguments in format
strings.

* Portability:
  - Improve OS/2 kLIBC support.
  - Fix libintl compilation issue with pre-C99 compilers.  It was a
regression since 0.19.5.
  - The AM_GNU_GETTEXT Autoconf macro can now detect musl-libc's
gettext as a compatible implementation.

Thanks,
-- 
Daiki Ueno


signature.asc
Description: PGP signature

[bug-gettext] gettext 0.19.8-rc1

[Daiki Ueno]

Hello,

The first rc of gettext 0.19.8 is available from:

http://alpha.gnu.org/gnu/gettext/gettext-0.19.8-rc1.tar.xz
http://alpha.gnu.org/gnu/gettext/gettext-0.19.8-rc1.tar.xz.sig

Since this release is mainly for bug fixes, I plan to release it soon
unless there is any critical issue.

Please send feedback to bug-gettext@gnu.org.

Here is a tentative NEWS entry for this release:

Version 0.19.8 - June 2016

* Support for reproducible builds:
  - msgfmt now produces little-endian .mo files by default.

* Programming languages support:
  - XML:
xgettext and msgfmt now look for .its files in directories
supplied through the GETTEXTDATADIRS or XDG_DATA_DIRS environment
variable.
  - JavaScript:
xgettext and msgfmt now recognize numbered arguments in format
strings.

* Portability:
  - Improve OS/2 kLIBC support.
  - Fix libintl compilation issue with pre-C99 compilers.  It was a
regression since 0.19.5.
  - The AM_GNU_GETTEXT macro can now detect musl-libc's gettext
implementation.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [PATCH] intl: Add API to get/set language precedence list

[Daiki Ueno]

Daiki Ueno  writes:

> So, I would propose:

For what it's worth I have filed a bug at glibc bugzilla:
https://sourceware.org/bugzilla/show_bug.cgi?id=20184
Let's see how they think about it.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #46844] gettext 0.19.7 fails to compile, OS X 10.7 & 10.8

[Daiki Ueno]

Follow-up Comment #3, bug #46844 (project gettext):

Thanks for the analysis and the patch.  I have pushed it as:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=7a8f93d8de8f24a303c5aec30ceb696700a503ec

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #38941] XML support

[Daiki Ueno]

Update of bug #38941 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

I consider this as fixed now, as ITS support.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #44098] Check strings for common problems

[Daiki Ueno]

Update of bug #44098 (project gettext):

  Status: In Progress => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #10:

This was added in 0.19.5.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [sr #108888] tags in AppData files

[Daiki Ueno]

Update of sr #10 (project gettext):

  Status:None => Done   
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

It was fixed in 0.19.7.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #46436] gettext.m4 does not detect musl-libc's gettext implementation

[Daiki Ueno]

Update of bug #46436 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=b67399b40bc5bf3165b09e6a095ec941d4b30a97

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [musl] Re: AM_GNU_GETTEXT without referring internal symbols?

[Daiki Ueno]

Rich Felker  writes:

> On Thu, May 26, 2016 at 09:53:49PM +0900, Masanori Ogino wrote:
>> 
>> It looks essentially good to me. You can remove the "if test
>> $gt_api_version -ge 3; then ... fi" part before where you modified too
>> if it is not used anywhere else, I guess.

I think it is still needed when AM_GNU_GETTEXT is invoked as:

  AM_GNU_GETTEXT([external], [need-formatstring-macros])

> I haven't tested it but the concept looks good to me too.

Thanks both for looking at it.  I have pushed it as:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=b67399b40bc5bf3165b09e6a095ec941d4b30a97

For the record, I have tested it on the Alpine Linux image obtained from
http://www.nongnu.org/pretest/ using a test package created with:

$ cp .../gettext.m4 gnulib/m4
$ ./gnulib/gnulib-tool --create-testdir --dir=t gettext

(I had to copy libintl.h to the VM as Alpine doesn't install it)

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [musl] Re: AM_GNU_GETTEXT without referring internal symbols?

[Daiki Ueno]

Hello,

Masanori Ogino  writes:

> 2016-04-07 11:26 GMT+09:00 Daiki Ueno :
>> Masanori Ogino  writes:
>>> That is why I proposed to have a blacklist of "broken" implementations
>>> as an option.
>>>
>>> AFAIK there have already been some blacklisting in autotools e.g.
>>> checking the version of glibc to reject specific broken implementation
>>> of a function. Thus, I think it's acceptable to use a blacklist. What
>>> do you think about it?
>>
>> Yes, that sounds like a good idea.  But I guess we then need to collect
>> information about incompatible implementations.  In this regard I'm
>> actually not sure if the gettext-tools test coverage can be used as an
>> indicator of compatibility.
>
> Indeed.

I was wondering if there is anything could be done in the upcoming
gettext release.  Let's go back to the original explanation by Bruno:
https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html
where he states two things:

1. The purpose of the checks are excluding incompatible implementations,
   e.g., NetBSD (around 1.5?) and Solaris 7

2. The __GNU_GETTEXT_SUPPORTED_REVISION macro is a recent addition

In that case, I guess we could bypass the symbol checks if
__GNU_GETTEXT_SUPPORTED_REVISION is defined, as long as broken
implementations do not define it.

How about the attached patch?

Regards,
-- 
Daiki Ueno
>From 77a71921d4925db2abc53a4610e89799e19fb113 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Thu, 26 May 2016 13:38:57 +0900
Subject: [PATCH] m4: Rely less on internal symbols

* gettext-runtime/m4/gettext.m4 (AM_GNU_GETTEXT): Skip checks for the
internal symbols _nl_msg_cat_cntr, _nl_domain_bindings, and
_nl_expand_alias, if __GNU_GETTEXT_SUPPORTED_REVISION is defined.
---
 gettext-runtime/m4/gettext.m4 | 29 ++---
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/gettext-runtime/m4/gettext.m4 b/gettext-runtime/m4/gettext.m4
index 1f6979a..32ff263 100644
--- a/gettext-runtime/m4/gettext.m4
+++ b/gettext-runtime/m4/gettext.m4
@@ -1,4 +1,4 @@
-# gettext.m4 serial 67 (gettext-0.19.6)
+# gettext.m4 serial 68 (gettext-0.19.6)
 dnl Copyright (C) 1995-2014, 2016 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -161,13 +161,18 @@ changequote([,])dnl
 [AC_LANG_PROGRAM(
[[
 #include 
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
 extern int _nl_msg_cat_cntr;
 extern int *_nl_domain_bindings;
+#define _GT_COMPAT (_nl_msg_cat_cntr + *_nl_domain_bindings)
+#else
+#define _GT_COMPAT 0
+#endif
+$gt_revision_test_code
]],
[[
 bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_bindings
+return * gettext ("")$gt_expression_test_code + _GT_COMPAT
]])],
 [eval "$gt_func_gnugettext_libc=yes"],
 [eval "$gt_func_gnugettext_libc=no"])])
@@ -193,17 +198,22 @@ return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_b
   [AC_LANG_PROGRAM(
  [[
 #include 
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
 extern int _nl_msg_cat_cntr;
 extern
 #ifdef __cplusplus
 "C"
 #endif
 const char *_nl_expand_alias (const char *);
+#define _GT_COMPAT (_nl_msg_cat_cntr + *_nl_expand_alias (""))
+#else
+#define _GT_COMPAT 0
+#endif
+$gt_revision_test_code
  ]],
  [[
 bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("")
+return * gettext ("")$gt_expression_test_code + _GT_COMPAT
  ]])],
   [eval "$gt_func_gnugettext_libintl=yes"],
   [eval "$gt_func_gnugettext_libintl=no"])
@@ -214,17 +224,22 @@ return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_a
 [AC_LANG_PROGRAM(
[[
 #include 
-$gt_revision_test_code
+#ifndef __GNU_GETTEXT_SUPPORTED_REVISION
 extern int _nl_msg_cat_cntr;
 extern
 #ifdef __cplusplus
 "C"
 #endif
 const char *_nl_expand_alias (const char *);
+#define _GT_COMPAT (_nl_msg_cat_cntr + *_nl_expand_alias (""))
+#else
+#define _GT_COMPAT 0
+#endif
+$gt_revision_test_code
]],
[[
 bindtextdomain ("", "");
-return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("")
+return * gettext ("")$gt_expression_test_code + _GT_COMPAT
]])],
 [LIBINTL="$LIBINTL $LIBICONV"
  LTLIBINTL="$LTLIBINTL $LTLIBICONV"
-- 
2.5.5

[bug-gettext] [sr #108743] Inconsistent definition of libintl_gettext_germanic_plural

[Daiki Ueno]

Update of sr #108743 (project gettext):

  Status:None => Done   
 Open/Closed:Open => Closed 

___

Follow-up Comment #4:

pan7: you are right, this is a regression after commit d5c2a516.
I have pushed a fix as:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=77dde02ef05e63e0ae2b113546e8adb26a962e29

Sorry for not responding earlier and thanks for the investigation.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] Failed Tests

[Daiki Ueno]

"Mulcihy, David D. (JSC-OD)[THE BOEING COMPANY]"
 writes:

Thanks for the report.

> 1)  Compile time issue with SunStudio 12.4 compiler. Apparently it is 
> more pendantic the gcc.
> The redeclared function is libintl_gettext_germanic_plural. Line 71 in 
> plural-exp.c has it as struct,
> Line 111 in plural-exp.h has it as const struct.

This is the same issue as:
https://savannah.gnu.org/support/?108743
where I suggested to use CFLAGS="-xc99=all,lib -Xc" as a work around.

However, it seems that the C compiler of SunStudio 12.4 actually
supports the needed C99 feature (struct/union initialiazers) by default.
Perhaps it might make sense to enable it like the attached patch.

> 2)  3 Failed tests 
> FAIL: test-mbrtowc3.sh
> FAIL: test-mbrtowc4.sh
> FAIL: test-mbsrtowcs4.sh

I suppose those are:
https://lists.gnu.org/archive/html/bug-gnulib/2014-11/msg00012.html
where the problem is caused by missing locale packages.

Regards,
-- 
Daiki Ueno
>From 8407ae13aa28a53dd29d14ab16220ef82e2437e0 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Tue, 24 May 2016 12:18:07 +0900
Subject: [PATCH] intl: Pacify SunStudio C compiler

* gettext-runtime/intl/plural-exp.c [__SUNPRO_C]: Use named struct/union
initializers.
---
 gettext-runtime/intl/plural-exp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gettext-runtime/intl/plural-exp.c b/gettext-runtime/intl/plural-exp.c
index 7f37e54..bfb228d 100644
--- a/gettext-runtime/intl/plural-exp.c
+++ b/gettext-runtime/intl/plural-exp.c
@@ -25,7 +25,8 @@
 
 #include "plural-exp.h"
 
-#if (defined __GNUC__ && !(defined __APPLE_CC_ && __APPLE_CC__ > 1) \
+#if ((defined __GNUC__ || defined __SUNPRO_C)   \
+ && !(defined __APPLE_CC_ && __APPLE_CC__ > 1)  \
  && !defined __cplusplus)   \
 || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L)
 
-- 
2.5.5

[bug-gettext] [bug #47991] memory errors in desktop_lex

[Daiki Ueno]

Update of bug #47991 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

Thanks for pointing that out.  I have pushed the patch with the same treatment
for 'locale':
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=68ab0dafa99f1941b3ebb47b7cf969381e7310f4

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [PATCH] intl: Add API to get/set language precedence list

[Daiki Ueno]

Daiki Ueno  writes:

> To do that, I think we need a good name of the public header.

After some considerations, I came up with , because those
functions are about user configuration of languages.

I also do not really like the name {set,get}_i18n_language(), because:

- "i18n" followed by "language" sounds redundant

- "language" is a bit misleading, because the document says it processes
  language precedence list

So, I would propose:

  /* Sets the language precedence list for the program to LANGUAGES.

 LANGUAGES may be a colon-separated list of language codes.
 If LANGUAGES is NULL, use the one inferred from the environment
 variable 'LANGUAGE'.

 The effect of the language precedence list is application
 dependent.  However, supporting applications should give priority
 to the language precedence list over the locale settings.

 This returns an opaque string that corresponds to the current
 language precedence list.  */
  const char *
  set_preferred_languages (const char *languages);

  /* Returns the language precedence list for the program.

 This returns NULL if the list is not set yet, or an opaque string
 that corresponds to the current language precedence list.  */
  const char *
  get_preferred_languages (void);

I also considered the following:

- {set,get}_gettext_language{,s}(): This is intuitive as long as the
  language precedence list is only used by gettext().  Otherwise it
  doesn't make sense, and it might make the border between the layer (B)
  and (C) vague.

- PREFIX_{set,get}_language{,s}(): Where PREFIX is "i18n", "nl", or
  something.

and here are some open questions:

- If this API is open to public, then how should the lock variable be
  exposed?

- Would it make any sense if those functions take a category, like
  LC_MESSAGES, to limit the effect?

Anyway, the above is still just a proposal, and better ideas are
welcome.

For the implementation, I'm attaching a couple of patches: one is
against gnulib master and the other one is against gettext master.

To test:

  $ cd .../gnulib
  $ patch -p1 < ...

  $ cd .../gettext
  $ patch -p1 < ...
  $ GNULIB_SRCDIR=`pwd`/../gnulib ./autogen.sh --no-git
  $ ./configure && make && make check

Regards,
-- 
Daiki Ueno
>From b5497a8b602a3b652d9fdbb7aa8bcc05fb158de4 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Wed, 18 May 2016 16:36:36 +0900
Subject: [PATCH v2] langconf, preferred-languages: new module

This adds functions to manipulate language precedence list, which
supersedes LANGUAGE envvar, used by gettext() or applications.
Suggested by Bruno Haible in:
https://lists.gnu.org/archive/html/bug-gettext/2016-05/msg9.html
* modules/langconf: New file.
* modules/preferred-languages: New file.
* modules/preferred-languages-tests: New file.
* m4/langconf_h.m4: New file.
* lib/preferred-languages.c: New file.
* lib/langconf.in.h: New file.
* tests/test-preferred-languages.c: New file.
---
 ChangeLog |  13 
 lib/langconf.in.h |  69 +++
 lib/preferred-languages.c | 135 ++
 m4/langconf_h.m4  |  58 
 modules/langconf  |  46 +
 modules/preferred-languages   |  27 
 modules/preferred-languages-tests |  14 
 tests/test-preferred-languages.c  |  60 +
 8 files changed, 422 insertions(+)
 create mode 100644 lib/langconf.in.h
 create mode 100644 lib/preferred-languages.c
 create mode 100644 m4/langconf_h.m4
 create mode 100644 modules/langconf
 create mode 100644 modules/preferred-languages
 create mode 100644 modules/preferred-languages-tests
 create mode 100644 tests/test-preferred-languages.c

diff --git a/ChangeLog b/ChangeLog
index 02a0f6f..2fec295 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2016-05-18  Daiki Ueno  
+
+	langconf, preferred-languages: new module
+	This adds functions to manipulate language precedence list, which
+	supersedes LANGUAGE envvar, used by gettext() or applications.
+	* modules/langconf: New file.
+	* modules/preferred-languages: New file.
+	* modules/preferred-languages-tests: New file.
+	* m4/langconf_h.m4: New file.
+	* lib/preferred-languages.c: New file.
+	* lib/langconf.in.h: New file.
+	* tests/test-preferred-languages.c: New file.
+
 2016-05-17  Paul Eggert  
 
 	manywarnings: update for GCC 6.1
diff --git a/lib/langconf.in.h b/lib/langconf.in.h
new file mode 100644
index 000..7ab2b51
--- /dev/null
+++ b/lib/langconf.in.h
@@ -0,0 +1,69 @@
+/* Functions to manipulate language precedence list.
+   Copyright (C) 2016 Free Software Foundation, Inc.
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU Lesser General Public License as published by
+   the Free Software Foundation; either version 2.1 of the

[bug-gettext] [bug #47123] reading ITS files

[Daiki Ueno]

Update of bug #47123 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Thanks for the suggestion.  I have added a check for that envvar and also
GETTEXTDATADIRS:

http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=e26b846fe273243b446cc02a027cc1a715734d1f

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [PATCH] intl: Add API to get/set language precedence list

[Daiki Ueno]

Bruno Haible  writes:

> The patch looks already nearly perfect to me. Just a few (minor) points:

Thanks for the review.  I have locally fixed those except the last one:

> - Probably I would also create a new .h file for these two functions,
>   because they are in layer (B) - like  is also different from
>   . Yes I know gnulib has the techniques for overriding
>when it needs to add two more function declarations to it;
>   but nevertheless.

To do that, I think we need a good name of the public header.
Tentatively, I have chosen , which sounds a bit ambiguous to
me.  Do you have any suggestions?

Also, splitting libgnuintl.in.h is a bit of a task since it requires to
copy all the helper macros (for redefining functions) into the new file.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #45405] VPATH build failure due to missing libintl.h

[Daiki Ueno]

Update of bug #45405 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #3:

Closing this, as the fix shipped in 0.19.5.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #45305] Add support for AppData specifications

[Daiki Ueno]

Update of bug #45305 (project gettext):

  Status: In Progress => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #10:

Closing this, as the fix shipped in 0.19.7.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #47674] gettext test suite fails with grep 2.24

[Daiki Ueno]

Update of bug #47674 (project gettext):

  Status:None => Invalid
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

According to the announcement, I assume this is no longer the case with grep
2.25:
https://lists.gnu.org/archive/html/info-gnu/2016-04/msg00010.html

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #47847] Undefined behavior [use-after-free] possible in libgettext

[Daiki Ueno]

Update of bug #47847 (project gettext):

  Status:None => Fixed  
 Privacy: Private => Public 
 Open/Closed:Open => Closed 

___

Follow-up Comment #4:

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=d8cd505c692ec0eb647135fef23f7f2b037f7571

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [PATCH] intl: Add API to get/set language precedence list

[Daiki Ueno]

Due to a potential getenv() call for LANGUAGE envvar, multi-threaded
programs may crash in gettext(), when there is another thread calling
setenv().  To mitigate this, add a new API set_i18n_language() to allow
setting language precedence list programatically.
Suggested by Bruno Haible in:
https://lists.gnu.org/archive/html/bug-gettext/2016-05/msg9.html
* gettext-runtime/intl/dcigettext.c (_nl_default_i18n_language)
(_nl_current_i18n_language): New variable.
(GET_I18N_LANGUAGE, SET_I18N_LANGUAGE): New functions.
(guess_category_value): Check LANGUAGE envvar only when
_nl_current_i18n_language is not set.
* gettext-runtime/intl/libgnuintl.in.h (get_i18n_language)
(set_i18n_language): New function declaration.
* gettext-tools/tests/Makefile.am (TESTS): Add test for
set_i18n_language().
* gettext-tools/tests/gettext-9: New test.
* gettext-tools/tests/gettext-9-prg.c: New test program.
* gettext-tools/tests/.gitignore: Ignore gettext-9-prg.
---
 gettext-runtime/intl/dcigettext.c| 54 ++--
 gettext-runtime/intl/libgnuintl.in.h | 34 +++
 gettext-tools/tests/.gitignore   |  1 +
 gettext-tools/tests/Makefile.am  |  4 +-
 gettext-tools/tests/gettext-9| 56 
 gettext-tools/tests/gettext-9-prg.c  | 82 
 6 files changed, 226 insertions(+), 5 deletions(-)
 create mode 100755 gettext-tools/tests/gettext-9
 create mode 100644 gettext-tools/tests/gettext-9-prg.c

diff --git a/gettext-runtime/intl/dcigettext.c 
b/gettext-runtime/intl/dcigettext.c
index 83bd775..f474786 100644
--- a/gettext-runtime/intl/dcigettext.c
+++ b/gettext-runtime/intl/dcigettext.c
@@ -342,6 +342,9 @@ libc_hidden_data_def (_nl_default_dirname)
 struct binding *_nl_domain_bindings;
 #endif
 
+static char _nl_default_i18n_language[] = "";
+static char *_nl_current_i18n_language = _nl_default_i18n_language;
+
 /* Prototypes for local functions.  */
 static char *plural_lookup (struct loaded_l10nfile *domain,
unsigned long int n,
@@ -430,8 +433,12 @@ typedef unsigned char transmem_block_t;
prefix.  So we have to make a difference here.  */
 #ifdef _LIBC
 # define DCIGETTEXT __dcigettext
+# define GET_I18N_LANGUAGE __get_i18n_language
+# define SET_I18N_LANGUAGE __set_i18n_language
 #else
 # define DCIGETTEXT libintl_dcigettext
+# define GET_I18N_LANGUAGE libintl_get_i18n_language
+# define SET_I18N_LANGUAGE libintl_set_i18n_language
 #endif
 
 /* Lock variable to protect the global data in the gettext implementation.  */
@@ -875,6 +882,43 @@ DCIGETTEXT (const char *domainname, const char *msgid1, 
const char *msgid2,
 }
 
 
+const char *
+GET_I18N_LANGUAGE (void)
+{
+  return _nl_current_i18n_language;
+}
+
+
+const char *
+SET_I18N_LANGUAGE (const char *language)
+{
+  gl_rwlock_wrlock (_nl_state_lock);
+
+  if (language == NULL)
+language = getenv ("LANGUAGE");
+
+  if (language != NULL && language[0] != '\0')
+{
+  char *new_language;
+
+  new_language = strdup (language);
+  if (__builtin_expect (new_language == NULL, 0))
+{
+  gl_rwlock_unlock (_nl_state_lock);
+  return NULL;
+}
+
+  if (_nl_current_i18n_language != _nl_default_i18n_language)
+free (_nl_current_i18n_language);
+  _nl_current_i18n_language = new_language;
+}
+
+  gl_rwlock_unlock (_nl_state_lock);
+
+  return _nl_current_i18n_language;
+}
+
+
 /* Look up the translation of msgid within DOMAIN_FILE and DOMAINBINDING.
Return it if found.  Return NULL if not found or in case of a conversion
failure (problem in the particular message catalog).  Return (char *) -1
@@ -1582,9 +1626,13 @@ guess_category_value (int category, const char 
*categoryname)
   if (strcmp (locale, "C") == 0)
 return locale;
 
-  /* The highest priority value is the value of the 'LANGUAGE' environment
- variable.  */
-  language = getenv ("LANGUAGE");
+  /* The highest priority value is the language precedence list
+ supplied either by calling set_i18n_language() or the value of
+ the 'LANGUAGE' environment variable.  If the former is used, the
+ value is stored in _nl_current_i18n_language.  */
+  language = _nl_current_i18n_language;
+  if (language == _nl_default_i18n_language)
+language = getenv ("LANGUAGE");
   if (language != NULL && language[0] != '\0')
 return language;
 #if !defined IN_LIBGLOCALE && !defined _LIBC
diff --git a/gettext-runtime/intl/libgnuintl.in.h 
b/gettext-runtime/intl/libgnuintl.in.h
index 4fb1514..3526555 100644
--- a/gettext-runtime/intl/libgnuintl.in.h
+++ b/gettext-runtime/intl/libgnuintl.in.h
@@ -302,6 +302,40 @@ extern char *bind_textdomain_codeset (const char 
*__domainname,
 
 #endif /* IN_LIBGLOCALE */
 
+#ifndef IN_LIBGLOCALE
+
+/* Returns the language precedence list for the program.  */
+#ifdef _INTL_REDIRECT_INLINE
+extern const char *libintl_get_i18n_language (void);
+static inline const char *get_i18n_language (void)
+{
+  

Re: [bug-gettext] [bug #47847] Undefined behavior [use-after-free] possible in libgettext

[Daiki Ueno]

Michael Pyne  writes:

> On Wed, May 11, 2016 11:32:29 Daiki Ueno wrote:
>> Michael Pyne  writes:
>> > This is important because 6.2.4 specifies that the value of a pointer is
>> > only valid as long as the lifetime of the pointed-to object.
>> 
>> Can you quote the sentence please?  Also note that 6.2.4 leaves the
>> description about allocated storage in 7.22.3:
>
> It was quoted below in the email. And don't read too much into the reference 
> to 7.22.3. Just because the standard mentions that allocated storage 
> functions 
> are described elsewhere doesn't make those allocated storage functions 
> relevant to the other 3 storage durations mentioned in that section...

Yes, that's true, and reading it so, then the argument makes sense.
Sorry for the confusion and thanks for your time to explain that.

For the record, here is the reasoning:

- in 3.15, an object is defined as: "region of data storage in the
  execution environment, the contents of which can represent values"
  
- 7.22.3 states: "The lifetime of an allocated object extends from the
  allocation until the deallocation"

- 6.2.4 states, as you quoted, "The value of a pointer becomes
  indeterminate when the object it points to (or just past) reaches the
  end of its lifetime"

> But if the C standard's legalese will not work, see also 
> http://kristerw.blogspot.com/2016/03/c-pointers-are-not-hardware-pointers.html
>
> One of the comments to that post gives a justification for *why* the value is 
> allowed to become indeterminate (sourced as from the C99 standard, but I 
> haven't verified it myself): "Consider a hypothetical segmented architecture 
> on which pointers comprise a segment descriptor and an offset. Suppose that 
> segments are relatively small so that large arrays are allocated in multiple 
> segments. While the segments are valid (allocated, mapped to real memory), 
> the 
> hardware, operating system, or C implementation can make these multiple 
> segments behave like a single object: pointer arithmetic and relational 
> operators use the defined mapping to impose the proper order on the elements 
> of the array. Once the memory is deallocated, the mapping is no longer 
> guaranteed to exist. Use of the segment descriptor might now cause an 
> exception, or the hardware addressing logic might return meaningless data."
>
> Note here that the registers being described are notionally separate from 
> arithmetic registers -- the idea is that referencing a now-invalid memory 
> address (even to compare to a valid memory address) would cause a hardware 
> trap due to the use of a segment description register that no longer has a 
> valid mapping assigned.
>
> C permits this sort of thing in order to support the widest range of hardware 
> architectures, and there are *very weird* architectures out there.

Yes, this well explains the rationale.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] RFC: move LANGUAGE check out of gettext()

[Daiki Ueno]

Hi Bruno,

Thank you very much for the detailed explanation.

Bruno Haible  writes:

> Why is this being reported for the LANGUAGE environment variable but not
> for the LANG and LC_ALL environment variables? Because for LANG and LC_*
> we have an architecture composed of three functionalities:
>
>   (A) environment variables: getenv(), setenv()
>
>   (B) locales: setlocale(), newlocale(), uselocale().
>
>   (C) gettext() and friends.
>
> (A) is the bottom-most layer. But it has the limitation that multi-threaded
> programs must not call setenv().
>
> (B) is a layer that fetches the initial values from (A), and that allows
> mutators (setlocale(), uselocale()) in multi-threaded programs.
> So that multi-threaded applications can modify the program's locale after
> startup, there is the setlocale() function.
> So that multi-threaded programs can have a locale per thread, there is a
> uselocale() function.
>
> (C) is an application layer that happens to be in Glibc for convenience
> reasons. It is based on the layer (B).
>
>
> Back to the LANGUAGE environment variable. The problem is that here we
> have the layers (A) and (C), but (B) is missing. The solution ought to
> be to introduce a layer (B) for LANGUAGE. LANGUAGE is not specified by
> POSIX and does not perfectly fit into the locale system, therefore I
> believe it is best treated separately.
>
> So, what I imagine is a layer (B) with an API like this:
>
>   /* Returns the language precedence list for the program. */
>   const char *get_i18n_language (void);
>
>   /* Sets the language precedence list for the program.
>  NULL means to use the one inferred from the environment variable. */
>   void set_i18n_language (const char *);
>
> or - if you want to have a language per thread -:
>
>   /* Returns the language precedence list for the current thread. */
>   const char *get_i18n_language (void);
>
>   /* Sets the language precedence list for the program.
>  NULL means to use the one inferred from the environment variable. */
>   void set_i18n_language (const char *);
>
>   /* Sets the language precedence list for the current thread.
>  NULL means to use the one for the program or, if not set,
>  the one inferred from the environment variable. */
>   void set_thread_i18n_language (const char *);
>
> You can protect the implementation of these functions with locks
> (functions/macros gl_rwlock_*).
>
>
> With this approach,
>   - Multithread program can change the i18n language in a thread-safe
> way, without using setenv().
>   - The setlocale() code is left alone.

The approach makes a lot of sense.  I will prepare a patch along those
lines.

>   * It modifies the code of setlocale() for a purpose that is unrelated to
> the locale system.
>
> The fact that glibc's locale/setlocale.c has to increment _nl_msg_cat_cntr
> (notification from layer (B) to layer (C)) is already bad enough; it
> exists because there is no standardized API for being notified of
> locale changes. It forces us to override setlocale on non-glibc systems,
> using gnulibology patterns.
>
> But adding yet another call from layer (B) to layer (C) is even more of a
> hack.

Yes.  The intention of this hack was to save existing programs without
modification (for the new API).  However, for the majority of programs,
it could be mitigated in a general-purpose library like GLib, by calling
set_i18n_language() during initialization of the library.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [bug #47847] Undefined behavior [use-after-free] possible in libgettext

[Daiki Ueno]

Michael Pyne  writes:

>> Could you point me to the actual sentence which you think is the case?
>> 
>> - The value of a pointer to an object whose lifetime has ended is used
>> (6.2.4).
>
> This sentence or, alternately,
>
>> - The value of a pointer that refers to space deallocated by a call to
>>   the free or realloc function is used (7.22.3).
>
> The section is really 7.20.3 (at least in the standard I have available) but 
> either way this text is talking about object lifetime, not passing a pointer 
> back to free (7.20.3.2) or realloc (7.20.3.4).

I'm reading the N1570 draft:
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
which might be different from the final standard, or earlier versions.

> This is important because 6.2.4 specifies that the value of a pointer is only 
> valid as long as the lifetime of the pointed-to object.

Can you quote the sentence please?  Also note that 6.2.4 leaves the
description about allocated storage in 7.22.3:

"There are four storage durations: static, thread, automatic, and
allocated. Allocated storage is described in 7.22.3."

So, I think all the discussions regarding allocated memory are
irrelevant here.

> 6.2.4 further says that "The value of a pointer becomes indeterminate
> when the object it points to reaches the end of its lifetime."
> (i.e. after free() is called).

It is unbelievable to me that free() makes a pointer indeterminate, in
addition to the pointed memory.  Why do you think so, do you have any
reference?

Unlike 'delete' in C++, free() is implemented as a library function,
with the signature of:

  void free(void *ptr);

See described in 7.22.3.3.  Which means, without compiler support, the
implementation is not able to modify PTR.

GCC even provides 'cleanup' attribute which brings the automatic
behavior to allocated storage:
https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#Common-Variable-Attributes
where the cleanup_function takes a pointer to the pointer variable
(i.e. TYPE **ptr, not TYPE *ptr) to properly invalidate the pointer
variable.

Regards,
-- 
Daiki Ueno

[bug-gettext] RFC: move LANGUAGE check out of gettext()

[Daiki Ueno]

Hello,

A while ago Matthias Clasen pointed me to a bug that is caused by a race
condition between a getenv() call in gettext() and a setenv() call in
another thread:
https://bugzilla.gnome.org/show_bug.cgi?id=754951

The direct cause of this bug is that gettext() tries to check LANGUAGE
envvar, while the string content returned by getenv() can be overwritten
by setenv() before being used.

To mitigate this, I was wondering if it would be possible to move the
getenv() call from gettext() to setlocale(), which is typically called
at program startup, and cache the value in a static variable.

The attached patch is an experiment implementing that in libintl, though
in practice it would require a change in glibc's setlocale
implementation.

Comments appreciated.

Regards,
-- 
Daiki Ueno
>From df259036a888e89d3fae7f4b4781b015776852db Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Tue, 10 May 2016 17:53:42 +0900
Subject: [PATCH] intl: Check LANGUAGE envvar in setlocale

Problem reported by Matthias Clasen in:
https://bugzilla.gnome.org/show_bug.cgi?id=754951
* gettext-runtime/intl/dcigettext.c (_nl_default_overriding_languages):
New static variable.
(_nl_overriding_languages): New variable to cache the value of LANGUAGE
envvar.
(_nl_locale_name_override): New function, split from
guess_category_value.  Set _nl_overriding_languages once LANGUAGE envvar
is successfully retrieved.
(guess_category_value): Call _nl_locale_name_override instead of
directly calling getenv.
* gettext-runtime/intl/gettextP.h (_nl_locale_name_override): New
function declaration.
(_nl_overriding_languages): New variable declaration.
* gettext-runtime/intl/setlocale.c (libintl_setlocale)
(libintl_newlocale): Call _nl_locale_name_override to ensure
_nl_overriding_languages.
---
 gettext-runtime/intl/dcigettext.c | 33 +++--
 gettext-runtime/intl/gettextP.h   | 12 
 gettext-runtime/intl/setlocale.c  | 12 ++--
 3 files changed, 53 insertions(+), 4 deletions(-)

diff --git a/gettext-runtime/intl/dcigettext.c b/gettext-runtime/intl/dcigettext.c
index 83bd775..318c27b 100644
--- a/gettext-runtime/intl/dcigettext.c
+++ b/gettext-runtime/intl/dcigettext.c
@@ -342,6 +342,12 @@ libc_hidden_data_def (_nl_default_dirname)
 struct binding *_nl_domain_bindings;
 #endif
 
+/* Default value of the 'LANGUAGE' environment variable used by
+   gettext(3) prior any call to setlocale(3) or gettext(3).  The
+   default value for this is "".  */
+static const char _nl_default_overriding_languages[] = "";
+const char *_nl_overriding_languages = _nl_default_overriding_languages;
+
 /* Prototypes for local functions.  */
 static char *plural_lookup (struct loaded_l10nfile *domain,
 			unsigned long int n,
@@ -1510,6 +1516,28 @@ category_to_name (int category)
 }
 #endif
 
+const char *
+internal_function
+_nl_locale_name_override (void)
+{
+  const char *language;
+
+  /* The value is stored in _nl_overriding_languages to avoid race
+ when setenv and gettext are called in separate threads. */
+  if (_nl_overriding_languages != _nl_default_overriding_languages)
+return _nl_overriding_languages;
+
+  language = getenv ("LANGUAGE");
+
+  if (language != NULL && language[0] != '\0')
+_nl_overriding_languages = strdup (language);
+  else
+_nl_overriding_languages = NULL;
+  language = _nl_overriding_languages;
+
+  return language;
+}
+
 /* Guess value of current locale from value of the environment variables
or system-dependent defaults.  */
 static const char *
@@ -1584,9 +1612,10 @@ guess_category_value (int category, const char *categoryname)
 
   /* The highest priority value is the value of the 'LANGUAGE' environment
  variable.  */
-  language = getenv ("LANGUAGE");
-  if (language != NULL && language[0] != '\0')
+  language = _nl_locale_name_override ();
+  if (language != NULL)
 return language;
+
 #if !defined IN_LIBGLOCALE && !defined _LIBC
   /* The next priority value is the locale name, if not defaulted.  */
   if (locale_defaulted)
diff --git a/gettext-runtime/intl/gettextP.h b/gettext-runtime/intl/gettextP.h
index 52be2fc..59c6cf1 100644
--- a/gettext-runtime/intl/gettextP.h
+++ b/gettext-runtime/intl/gettextP.h
@@ -254,6 +254,7 @@ extern const char *_nl_locale_name_default (void);
 # define gl_locale_name _nl_locale_name
 /* extern const char *_nl_locale_name (int category,
    const char *categoryname); */
+extern const char *_nl_locale_name_override (void);
 #endif
 
 struct loaded_l10nfile *_nl_find_domain (const char *__dirname, char *__locale,
@@ -308,6 +309,17 @@ extern const char _nl_default_default_domain[] attribute_hidden;
 /* Default text domain in which entries for gettext(3) are to be found.  */
 extern const char *_nl_current_default_domain attribute_hidden;
 
+/* The internal variables in the standalone libintl.a must have different
+   n

Re: [bug-gettext] [bug #47847] Undefined behavior [use-after-free] possible in libgettext

[Daiki Ueno]

Michael Pyne  writes:

> On Mon, May 9, 2016 08:51:56 Daiki Ueno wrote:
>> Follow-up Comment #3, bug #47847 (project gettext):
>> 
>> Thanks for the comment, Bruno.
>> 
>> The reasoning sounds convincing, but I'm a bit confused that there is no
>> such path in the original code.  ISO C 6.2.4 also says: "The result of
>> attempting to indirectly access an object with automatic storage duration
>> from a thread other than the one with which the object is associated is
>> implementation-defined", but I neither see a possibility of this.
>> 
>> So far, the more I think of this, the more it seems like a false positive
>> (and if so, perhaps we could add an annotation instead to suppress the
>> warning).
>
> It is absolutely not a false positive. It is warning because the pointer is 
> referred to after it has been free()'d. This is a class of behavior which is 
> explicitly described as undefined behavior in the C standard (Annex J.2 lists 
> the conditions if you have the reference handy, as does the CERT link I 
> provided in the bug link).

Could you point me to the actual sentence which you think is the case?

The Annex J collects portability issues in the C standard itself,
referring to the corresponding sections.  In J.2, the following seem to
be related:

- An object is referred to outside of its lifetime (6.2.4).

- The value of a pointer to an object whose lifetime has ended is used (6.2.4).

- The value of an object with automatic storage duration is used while
  it is indeterminate (6.2.4, 6.7.9, 6.8).

- The value of a pointer that refers to space deallocated by a call to
  the free or realloc function is used (7.22.3).

However, looking at the cited section, I see the word "used" is used in
a stricter meaning.  For example, the last one refers to 7.22.3, where
the undefined behavior is defined as:

"[...] if the argument does not match a pointer earlier returned by
a memory management function, or if the space has been deallocated by a
call to free or realloc, the behavior is undefined."

So, if I read it correctly, the behavior is undefined only if the
pointer value is used as an argument of free or realloc, not in a
general expression.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #47847] Undefined behavior [use-after-free] possible in libgettext

[Daiki Ueno]

Follow-up Comment #3, bug #47847 (project gettext):

Thanks for the comment, Bruno.

The reasoning sounds convincing, but I'm a bit confused that there is no such
path in the original code.  ISO C 6.2.4 also says: "The result of attempting
to indirectly access an object with automatic storage duration from a thread
other than the one with which the object is associated is
implementation-defined", but I neither see a possibility of this.

So far, the more I think of this, the more it seems like a false positive (and
if so, perhaps we could add an annotation instead to suppress the warning).

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #47847] Undefined behavior [use-after-free] possible in libgettext

[Daiki Ueno]

Follow-up Comment #1, bug #47847 (project gettext):

Thanks for pointing that out.

> A reasonable fix is to convert the pointers to be compared to uintptr while
both pointers are still valid

It sounds like a good idea, but I doubt we can always assume that uintptr_t is
available.  So, I have created a patch to fix it by postponing free:
https://lists.gnu.org/archive/html/bug-gnulib/2016-05/msg00016.html

Does it make sense?

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] Misspelled: "massage" instead of "message"

[Daiki Ueno]

Vitalie Ghelbert  writes:

> See attached screen shot.
> URL: http://www.gnu.org/software/gettext/

Thanks for the report.  However, I suppose the term "massage" here is
used in the meaning of "tweak" or "adjust".  If we replaced it with
"message", the sentence wouldn't make sense.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #47697] js-format needs to be updated for gjs

[Daiki Ueno]

Update of bug #47697 (project gettext):

  Status:None => Duplicate  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Thanks for the report, even though this was already reported and fixed in git
master:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=9b9ebf8f96dd3b142e4202ca4a60feac9db0820e

Maybe we should make a new stable release soon.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #47674] gettext test suite fails with grep 2.24

[Daiki Ueno]

Follow-up Comment #1, bug #47674 (project gettext):

I'd expect it will be fixed once the next version of grep is released, with a
fix in gnulib incorporated:
https://lists.gnu.org/archive/html/bug-gnulib/2016-04/msg9.html

We would better wait it rather than adding a workaround relying on the non
portable -a option.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [musl] Re: AM_GNU_GETTEXT without referring internal symbols?

[Daiki Ueno]

Masanori Ogino  writes:

>> So I suppose the only feasible option here is to somehow whitelist the
>> implementations by checking macros or symbols.  Does musl provides
>> anything like that[1]?
>
> No, it doesn't on purpose. Here is the entry on this topic in the FAQ:
> http://wiki.musl-libc.org/wiki/FAQ#Q:_why_is_there_no_MUSL_macro_.3F
>
> Also, I'd like to point out some detailed explanations in a recent discussion:
> http://www.openwall.com/lists/musl/2016/03/23/6
> http://www.openwall.com/lists/musl/2016/03/23/7
> http://www.openwall.com/lists/musl/2016/03/23/9

Thanks for the information.

> That is why I proposed to have a blacklist of "broken" implementations
> as an option.
>
> AFAIK there have already been some blacklisting in autotools e.g.
> checking the version of glibc to reject specific broken implementation
> of a function. Thus, I think it's acceptable to use a blacklist. What
> do you think about it?

Yes, that sounds like a good idea.  But I guess we then need to collect
information about incompatible implementations.  In this regard I'm
actually not sure if the gettext-tools test coverage can be used as an
indicator of compatibility.

By the way, musl defines __GNU_GETTEXT_SUPPORTED_REVISION in the same
way as glibc:

  #define __GNU_GETTEXT_SUPPORTED_REVISION(major) ((major) == 0 ? 1 : -1)

Is major = 1 + minor = 1 actually supported in musl?

After briefly checking Solaris 11 variants have:

  #define __GNU_GETTEXT_SUPPORTED_REVISION(m) \
  m) == 0) || ((m) == 1)) ? 1 : -1)

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols?

[Daiki Ueno]

Hello,

Masanori Ogino  writes:

> Now AM_GNU_GETTEXT uses _nl_msg_cat_cntr and _nl_expand_alias to check
> whether the implementation is compatible with GNU gettext. However,
> the symbols don't appear in libintl.h so it seems that they are not
> part of the public API.
>
> Actually, musl libc implements libintl features and the score of
> gettext-tools' testsuite is equal to that with the internal libintl,
> using a modified AM_GNU_GETTEXT.
>
> The musl's libintl.h even defines __USE_GNU_GETTEXT and
> __GNU_GETTEXT_SUPPORTED_REVISION, but it does not imitate private
> symbols.
>
> I had checked the archive and I've found some discussions:
> https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html
> http://lists.gnu.org/archive/html/bug-gettext/2015-11/msg00015.html
>
> So, if the goal of the macro is check if the implementation is
> compatible with GNU gettext, why don't we check the public API rather
> than using internal symbols? Is it possible to check if the
> implementation is not one of known "broken" implementations and/or it
> is really compatible?

I agree that it would be desirable, but doubt that it is possible (at
least reliably), because:

- For some reason, there is no public API to directly load arbitrary MO
  files and we need to mimic the behavior of translated applications:
  prepare a directory structure (e.g. DIR/fr/domain.mo), call
  bindtextdomain() for the directory, and finally call gettext().

- That requires that at least one non-POSIX locale is available on the
  system, to pick the translation.  However, even if the system is glibc
  based, not all locales might be available thanks to sub-packaging
  (Fedora) or user configuration (Debian).

So I suppose the only feasible option here is to somehow whitelist the
implementations by checking macros or symbols.  Does musl provides
anything like that[1]?

Regards,

Footnotes: 
[1]  https://sourceforge.net/p/predef/wiki/Libraries/

-- 
Daiki Ueno

[bug-gettext] [bug #47531] One byte heap buffer overflow in x-lua.c (triggered by make check)

[Daiki Ueno]

Update of bug #47531 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Thanks, I have pushed the patch:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=0571be87e09c6db11be8c56c773b563c2b4c99a4

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] "msgfmt -c" bug

[Daiki Ueno]

Paul Franklin  writes:

> To trigger this bug, make an xx.po file (with a correct
> header) with this stanza in it:
>
> #: ../gramps/gen/datehandler/_datedisplay.py:168
> #, python-brace-format
> msgid "{long_month} {year}"
> msgstr "{long_month.f[D]} {year}"
>
> and when you run run "msgfmt -c -v xx.po" on it, it is
> flagged as having an error:
>
> xx.po:21: 'msgstr' is not a valid Python brace format string, unlike
> 'msgid'. Reason: In the directive number 0, there is an unterminated
> format directive.
> msgfmt: found 1 fatal error

Thanks for the report.  I didn't realize that PEP3101 allows such
"chained" invocations of '.' and '[..]'.  I have pushed the attached
patch to fix this.

Regards,
-- 
Daiki Ueno
>From b4220c509a90186931a69575981c1b0e80ffc1f6 Mon Sep 17 00:00:00 2001
From: Daiki Ueno 
Date: Wed, 23 Mar 2016 15:17:21 +0900
Subject: [PATCH] format-python-brace: Support chained expression

* gettext-tools/src/format-python-brace.c (parse_directive): Recognize
chained getattr/getitem expressions.
* gettext-tools/tests/format-python-brace-1: Add test for the case where
both getattr and getitem are used.
Reported by Paul Franklin in:
https://lists.gnu.org/archive/html/bug-gettext/2016-03/msg00017.html
---
 gettext-tools/src/format-python-brace.c   | 11 +--
 gettext-tools/tests/format-python-brace-1 |  2 ++
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/gettext-tools/src/format-python-brace.c b/gettext-tools/src/format-python-brace.c
index a212707..c76088c 100644
--- a/gettext-tools/src/format-python-brace.c
+++ b/gettext-tools/src/format-python-brace.c
@@ -140,7 +140,13 @@ parse_directive (struct spec *spec,
   return false;
 }
 
+  /* Parse '.' (getattr) or '[..]' (getitem) operators followed by a
+ name.  If must not recurse, but can be specifed in a chain, such
+ as "foo.bar.baz[0]".  */
+  for (;;)
+{
   c = *format;
+
   if (c == '.')
 {
   format++;
@@ -152,7 +158,6 @@ parse_directive (struct spec *spec,
   FDI_SET (format, FMTDIR_ERROR);
   return false;
 }
-  c = *format;
 }
   else if (c == '[')
 {
@@ -175,7 +180,9 @@ parse_directive (struct spec *spec,
   FDI_SET (format, FMTDIR_ERROR);
   return false;
 }
-  c = *format;
+}
+  else
+break;
 }
 
   if (c == ':')
diff --git a/gettext-tools/tests/format-python-brace-1 b/gettext-tools/tests/format-python-brace-1
index 601b023..3a1f9ea 100755
--- a/gettext-tools/tests/format-python-brace-1
+++ b/gettext-tools/tests/format-python-brace-1
@@ -30,6 +30,8 @@ cat <<\EOF > f-pyb-1.data
 "abc{value[0}"
 # Invalid: unknown character in getitem operator
 "abc{value[!]}"
+# Valid: use of both getattr and getitem operators
+"abc{value.v[name]}"
 # Valid: format specifier
 "abc{value:0}"
 # Valid: standard format specifier
-- 
2.5.0

Re: [bug-gettext] GNU gettext plural forms manual page uses incorrect '%$2d' instead of correct '%2$d'

[Daiki Ueno]

Jonathan Leffler  writes:

> In the GNU gettext utilities manual at:
> https://www.gnu.org/software/gettext/manual/html_node/Plural-forms.html, 
> there is the suggested
> function call:
>
> printf (ngettext ("%$2d file removed from directory %$1s",
>   "%$2d files removed from directory %$1s",
>   n),
> dir, n);

Thank you.  I have fixed it in git:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=009c1218593a5f5a6d132b3694e4e679572eff9f

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #47492] Testsuite summary for gettext-tools 0.19.7.17-298b-dirty

[Daiki Ueno]

Follow-up Comment #2, bug #47492 (project gettext):

> FAIL: format-javascript-1

Thanks.  Among others, this is a good catch.  I've pushed a fix for this:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=2328b5b5a7e227ab1fb4dc1bccb39948e721aa76

I'll look into other failures soon.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #44588] Swedish translations not working on Windows systems

[Daiki Ueno]

Update of bug #44588 (project gettext):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Thanks Geert for the patch.  I have pushed it to gnulib repo so gettext will
use the code from the next sync.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #47484] make fails under OSX 10.11 - plural.c not found

[Daiki Ueno]

Follow-up Comment #5, bug #47484 (project gettext):

Building from git means that you need more tools installed on the system, such
as gperf, bison, as described in the HACKING file.

In this particular case, as plural.c is generated by bison, bison might be
missing on your system.

For "make check" failures, yes, please open a new bug.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] [bug #47484] make fails under OSX 10.11 - plural.c not found

[Daiki Ueno]

Update of bug #47484 (project gettext):

  Status:None => Need Info  

___

Follow-up Comment #2:

I'm a bit confused that you mention "gettext 0.19.7" in the first comment and
"git clone" in the second comment.

What did you do exactly?  It would be helpful if you provide more information,
e.g. how you get the source code, and which command line is supplied to
./configure, etc.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] Errors in 'make check-TESTS' in gettext-tools/gnulib-tests

[Daiki Ueno]

Peter Dyballa  writes:

> On PPC Mac OS X 10.4.11 ("Tiger") this happens:
>
>   make[4]: Leaving directory
> `/sw/build.build/gettext-tools-0.19.7-1l/gettext-0.19.7/gettext-tools/gnulib-tests'

As the failures are in gnulib tests (not gettext-tools itself), that's
not too bad IMO.

>   ../../build-aux/test-driver: line 107: 21788 Abort trap
> (core dumped) "$@" >$log_file 2>&1
>   FAIL: test-float
>   ../../build-aux/test-driver: line 107: 22543 Segmentation
> fault  (core dumped) "$@" >$log_file 2>&1
>   FAIL: test-localename

Are you able to reproduce those failures individually with the procedure
described in:
http://www.gnu.org/software/gnulib/manual/html_node/How-to-add-a-new-module.html#How-to-add-a-new-module

For the above cases, you could do:

  $ ./gnulib-tool --create-testdir --dir=/tmp/testdir float

or:

  $ ./gnulib-tool --create-testdir --dir=/tmp/testdir localename

and then "configure && make && make check" in /tmp/testdir.

>   configure: WARNING: gettext-tools will be built without ACL support.
>
> IMO invoking the ACL tests should not have happened.

Actually, the ACL tests are skipped.  From your log:

>   SKIP: test-file-has-acl.sh
>   SKIP: test-file-has-acl-1.sh
>   SKIP: test-file-has-acl-2.sh
>   ../../build-aux/test-driver: line 107: 21788 Abort trap
> (core dumped) "$@" >$log_file 2>&1
>   FAIL: test-float

The failing test is test-float.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] javascript-format fails to recognize string reordering

[Daiki Ueno]

Sean Burke  writes:

> Given a string "%s foo %s", the gjs JavaScript implementation allows these to 
> be reordered in a
> fashion similar to C, that is "%2$s foo %1$s". However, `msgfmt -vc' will 
> error on encountering
> this in a translation when the msgid is marked javascript-format, giving the 
> following:
>
> 'msgstr' is not a valid JavaScript format string, unlike 'msgid'. Reason: In 
> the directive number 1,
> the character '$' is not a valid conversion specifier.
>
> javascript-format should be made to understand this form of string 
> formatting, as c-format does.

Thanks for the report (and sorry for the shameless delay).

I have pushed a fix for that:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=9b9ebf8f

Numbered arguments in the width and precision part are not supported
yet, but I guess it wouldn't be a problem for gettext as it is ignored
at extraction phase and 'javascript-format' flag is not emitted.

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #42615] Feature request: Support internationalization of PolicyKit XML files

[Daiki Ueno]

Follow-up Comment #3, bug #42615 (project gettext):

Now that gettext 0.19.7 has a built-in support for ITS, I would suggest
PolicyKit to install *.its files by themselves, like this:

https://github.com/hughsie/appstream-glib/commit/2f98b73d42fc65e9c1c3feaefe0ba1fe0fd1b66e
or:
https://git.gnome.org/browse/glib/commit/?id=50645b724a3b43767fd57e4af53365d0cd270382

I am more than happy to help, but I don't really know how those PolicyKit
translations are consumed.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] [PATCH v3 0/4] OS/2 patches

[Daiki Ueno]

KO Myung-Hun  writes:

> [PATCH v3 1/4] moop: Replace CR as well as LF with a space
> [PATCH v3 2/4] intl: Support UNIXROOT in BINDTEXTDOMAIN () on EMX
> [PATCH v3 3/4] libasprintf: Build a shared library on OS/2
> [PATCH v3 4/4] gettext-tools: Use a short name for DLLs on OS/2

Thanks, merged them.  Sorry for the delay.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] Reproducible builds and .mo file endianness

[Daiki Ueno]

Bruno Haible  writes:

>> Would you please reconsider about msgfmt default behaviour?
>
> Yes. And since the industry trend of endianness goes in favour of little-
> endian CPUs (even PowerPC and ARM now exist in little-endian flavour),
> the default behaviour should be to generate little-endian .mo file.
>
> In 2012, in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=671257,
> I wrote "Users of PowerPC systems have the same right to optimized
> MO files as users of x86 systems. Therefore I would like to have the
> issue solved in Debian's build system." But now the goal is to make
> reproducible builds not only possible, but also easy, and this has
> higher importance than a few CPU cycles on big-endian PowerPC or ARM.

Thanks, well said and the patch looks good.  Could you install it?

Regards,
-- 
Daiki Ueno

[bug-gettext] [bug #46844] gettext 0.19.7 fails to compile, OS X 10.7 & 10.8

[Daiki Ueno]

Follow-up Comment #1, bug #46844 (project gettext):

Thanks for reporting that.  We should probably make the libxml2 check tighter,
so that it fails when the required functions are not compiled in the library.

Anyway, I guess a work around would be adding --with-included-libxml to
configure.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug-gettext] gettext source contains help2man. Is this really needed?

[Daiki Ueno]

Santiago Vila  writes:

> While fixing the previous bug I noticed that the build process
> modifies msgfmt.1. This is because we modify msgfmt so that it always
> output little endian.
>
> But I don't really want msgfmt.1 to be modified just because of this.
>
> In the past I used to insert a suitable "touch" command somewhere in
> debian/rules so that msgfmt.1 was not considered "outdated". Is there
> any better way?

A possible idea is to compare the output of help2man with the existing
one, and don't overwrite it if they are identical (except the date), in
a similar way to the msgmerge call in po/Makefile.in.in.  I personally
feel it as a bit over-engineering, though.

> Should help2man be really part of the gettext source tarball? I would
> think that it is on the same level as automake or autoconf.

I don't know, but some other projects (e.g. coreutils) also ships a copy
of help2man for some reasons.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [h...@realh.co.uk: Bug#809846: gettext: Does not recognise glade]

[Daiki Ueno]

Santiago Vila  writes:

> Simple question: Would it make sense to modify the "PACKAGING" file
> accordingly? (See attached patch for an example).

Thanks, applied the patch with a slight modification.

> I don't quite understand why this gettext-VERSION is needed.

It is merely needed to distinguish user-supplied *.its files, which
shall be installed in /usr/share/gettext/its, from our own.

This is analogous to the use of /usr/share/aclocal-VERSION in Automake
or /usr/share/vala-VERSION/vapi in Vala.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [h...@realh.co.uk: Bug#809846: gettext: Does not recognise glade]

[Daiki Ueno]

Santiago Vila  writes:

> Package: gettext
> Version: 0.19.7-1
> Severity: normal
>
> Dear Maintainer,
>
> The current version of xgettext seems to be unable to process glade
> files, warning that it doesn't recognise the extension '.glade' on the
> input file, and ignoring -L Glade and --language=Glade options. No
> output .pot file is produced, but it returns a 0 exit code. For example:
>
> $ /usr/bin/xgettext --package-name=roxterm -L Glade -o
> /home/tony/Dev/roxterm/build/po/glade.pot ../src/roxterm-config.glade
> /usr/bin/xgettext: warning: file '../src/roxterm-config.glade'
> extension 'glade' is unknown; will try C

This is a packaging problem; the gettext package now needs to install
version specific data files from /usr/share/gettext-0.19.7 directory,
like the attached patch.

Regards,
-- 
Daiki Ueno
diff -Nru gettext-0.19.7/debian/changelog gettext-0.19.7/debian/changelog
--- gettext-0.19.7/debian/changelog	2015-12-30 02:27:12.0 +0900
+++ gettext-0.19.7/debian/changelog	2016-01-05 05:45:09.0 +0900
@@ -1,3 +1,9 @@
+gettext (0.19.7-2) unstable; urgency=medium
+
+  * Install version specific data files.
+
+ -- Daiki Ueno   Tue, 05 Jan 2016 05:45:06 +0900
+
 gettext (0.19.7-1) unstable; urgency=medium
 
   * New upstream release.
diff -Nru gettext-0.19.7/debian/rules gettext-0.19.7/debian/rules
--- gettext-0.19.7/debian/rules	2015-12-30 01:00:00.0 +0900
+++ gettext-0.19.7/debian/rules	2016-01-05 05:47:50.0 +0900
@@ -149,7 +149,7 @@
 	cp -a debian/tmp/usr/lib debian/$@/usr
 	cp -a debian/tmp/usr/share/info debian/$@/usr/share
 	cp -a debian/tmp/usr/share/aclocal debian/$@/usr/share
-	cp -a debian/tmp/usr/share/gettext debian/$@/usr/share
+	cp -a debian/tmp/usr/share/gettext* debian/$@/usr/share
 	rm -f debian/$@/usr/share/gettext/libintl.jar
 ifeq (,$(findstring %$(DEB_HOST_ARCH)%,$(NOJAVA_ARCHS)))
 ifeq ($(filter stage1,$(DEB_BUILD_PROFILES)),)

[bug-gettext] gettext-0.19.7 released

[Daiki Ueno]

This is the announcement of a new release of GNU gettext.

GNU gettext allows programs to produce messages in the user's native
language. It consists of
  - runtime libraries for C, C++, Java, C#, Shell programs.
  - tools for programmers and translators.

Here are the compressed sources:
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.7.tar.gz   (19MB)
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.7.tar.xz   (6.9MB)
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.7.tar.lz   (6.3MB)

Here are the GPG detached signatures[*]:
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.7.tar.gz.sig
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.7.tar.xz.sig
  ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.7.tar.lz.sig

Use a mirror for higher download bandwidth:
  http://www.gnu.org/order/ftp.html

Here are the MD5 and SHA1 checksums:

87c4ab267c4dce8a75db5d057bb3c92f  gettext-0.19.7.tar.gz
f81e50556da41b44c1d59ac93474dca5  gettext-0.19.7.tar.xz
0008c0ac4958eb9749362fbac4bdf750  gettext-0.19.7.tar.lz
d8fc932196cc78b83ca1b63c8687ec3d513b40b6  gettext-0.19.7.tar.gz
4b2574b76d14c98270bf607a2a62f033524d8e8c  gettext-0.19.7.tar.xz
e89d7147c69b3d94ea04d3be2e833f50fa85f321  gettext-0.19.7.tar.lz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify gettext-0.19.7.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys D7E69871

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.15
  Libtool 2.4.2
  Bison 3.0.4
  Gnulib v0.1-645-g1f63650

Noteworthy changes in 0.19.7:

* Programming languages support:
  - XML:
xgettext can now load custom string extraction rules supplied by
consumer projects.  The rules are written in XML, conforming the
Internationalization Tag Set (ITS) standard.  All the existing
XML-based language scanners (Glade, GSettings, and AppData) are
rewritten using ITS.  In addition, msgfmt now has --xml option to
merge translations back to the original XML document.

* Portability:
  - Improve OS/2 kLIBC support (still not complete)
  - Remove dependency on expat

Thanks,
-- 
Daiki Ueno


signature.asc
Description: PGP signature

[bug-gettext] [bug #46756] xgettext 0.19.6 mangles C escapes; causes po file corruption

[Daiki Ueno]

Update of bug #46756 (project gettext):

  Status:None => Confirmed  

___

Follow-up Comment #1:

Yes, this is certainly a regression and the character escapes should be
converted before concatenating adjacent strings.

However, it is not straightforward to fix this, now that the \u escape is
introduced.  I guess it would be ideally fixed by consolidating encoding
conversion stuff (currently in xgettext.c) into character escape handling
(x-c.c) and do that before string concatenation.

In any case, an easy work around is to use octal escapes.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug-gettext] gettext 0.19.7-rc1

[Daiki Ueno]

Hello,

The first rc of gettext 0.19.7 is available from:

http://alpha.gnu.org/gnu/gettext/gettext-0.19.7-rc1.tar.xz
http://alpha.gnu.org/gnu/gettext/gettext-0.19.7-rc1.tar.xz.sig

Please send feedback to bug-gettext@gnu.org.

Dear TP coordinator, this includes a string change in gettext-tools.pot
from the previous pretest versions:

 - "The root element <%s> is not allowed in a valid CLDR file"
 + "The root element must be <%s>"

I tried to build it using the VMs from the pretest project, and got
"make check" failures on some platforms (namely CentOS 6.5 & 7, and
FreeBSD 10.1), but I doubt we can fix it for this release.

The issues on CentOS seem to be related to the system locale settings.

The issues on FreeBSD can be avoided with the --enable-threads=pthreads
configure option, and an update of a test data file:
http://git.sv.gnu.org/cgit/gettext.git/commit/?id=8054ef11

Unless there is any major problem, I will release it as 0.19.7 in a few
days.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [PATCH] OS/2 patches

[Daiki Ueno]

Sorry for late response.

KO Myung-Hun  writes:

>>> [PATCH 1/8] moop: Replace CR as well as NL with a space
>> 
>> I'm not sure who adds CR here.  Can't we assume that all source files
>> use NL line termination?  Or some other tools add CR on OS/2?  Also, I
>> worry about the portability of '\r' as an argument of tr.
>
> sed. On OS/2, many utils such as sed, coreutils and sh use CR+LF as a
> EOL marker instead of LF only.
>
> I replaced '\r' with '\015' in according to John Cowan

Thanks, but I'm still not sure if this is the right solution.

I generally avoid special handling of CR in scripts, to make the
maintenance easier.

So, couldn't the corresponding code:

  methods=`sed -e "$sed_remove_comments" < "$file" | sed -e 
"$sed_extract_methods" | tr '\n' ' ' | tr ';' '\n' | sed -e 's,[]*$,,'`

be translated into:

  methods=`sed -e "$sed_remove_comments" < "$file" | sed -e 
"$sed_extract_methods" | tr -d ';' | sed -e 's,[]*$,,'`

?

The results should be equivalent except empty lines, which will be
removed later on.

>>> [PATCH 5/8] intl: Support UNIXROOT in BINDTEXTDOMAIN () on EMX
>> 
>> Doesn't this conflict with the UNIXROOT resolution in os2compat.c?
>
> I removed the dependencies on os2compat.[ch] from kLIBC code paths at
> With [PATCH 3/8]. So this will not conflict with os2compat.c
>
> Even if EMX is used, it will be no problem because this patch check the
> condition strictly. That is, once os2compat.c resolved UNIXROOT, this
> patch does not prepend UNIXROOT.

That's good.  Maybe it would be worth mentioning as a comment?

>>> [PATCH 7/8] gettext-tools: Use a short name for DLLs on OS/2

+AM_CONDITIONAL([OS2], [test "${host_os#os2}" != "${host_os}"])

I guess it would be a little more portable to use a 'case "$host_os"
...' statement (like WOE32), instead of ${VAR#WORD} substitution.

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] [platform-testers] gettext pretest 0.19.6.44-a2e0a

[Daiki Ueno]

Assaf Gordon  writes:

> On 12/14/2015 12:44 AM, Daiki Ueno wrote:
>> I have updated the included libxml2 to the latest release 2.9.3 and
>> uploaded a new tarball:
>> http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.44-a2e0a.tar.xz
>> http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.44-a2e0a.tar.xz.sig
>
> Linking now fails in various ways:

Thanks for testing.  I was too lazy and only tested it with
--with-included-libxml on my work environment.

The latter 3 failures are due to missing -lm, which I'm going to address
with this commit:
http://git.sv.gnu.org/cgit/gettext.git/commit/?id=19ce4c30

I have no idea about the first one, as the relevant code hasn't been
touched for a while.  Does it happen only on ARM?

Regards,
-- 
Daiki Ueno

[bug-gettext] gettext pretest 0.19.6.44-a2e0a

[Daiki Ueno]

Hello,

It turned out that the previous pretest release failed to build on some
environments where libxml2 is not installed (in that case the included
copy of libxml2 is used, but it was way too old).

I have updated the included libxml2 to the latest release 2.9.3 and
uploaded a new tarball:
http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.44-a2e0a.tar.xz
http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.44-a2e0a.tar.xz.sig

Thanks,
-- 
Daiki Ueno

Re: [bug-gettext] [platform-testers] gettext pretest 0.19.6.43-6e6f6

[Daiki Ueno]

Assaf Gordon  writes:

> Hello,

Hi Assaf,

> On 12/11/2015 04:04 AM, Daiki Ueno wrote:
>>
>> The first pretest of gettext 0.19.7 is now available at:
>> http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.43-6e6f6.tar.xz
>>
>> I would appreciate if you can download and test it.  I plan to release
>> it in this month.
>
> I'm getting compilation failures on various environments (centos 6.5,
> centos 7, Debian 8.1, OpenBSD, but *not* on Ubuntu 14.04),
> and they all seem related to similar XML issue of missing function
> definitions (perhaps a wrong version of some XML library is
> installed?).

Thanks for quick testing.  On those environments the included libxml2 is
used and I overlooked that it doesn't have XPath support.

Unfortunately, it seems not sufficient to edit
gettext-tools/gnulib-lib/libxml/xmlversion.in.h.  I'll try to work out a
way of enabling XPath with the included libxml2 (perhaps it might need
an update of it to a newer release).

Regards,
-- 
Daiki Ueno

[bug-gettext] gettext pretest 0.19.6.43-6e6f6

[Daiki Ueno]

Hello,

The first pretest of gettext 0.19.7 is now available at:
http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.43-6e6f6.tar.xz
http://alpha.gnu.org/gnu/gettext/gettext-0.19.6.43-6e6f6.tar.xz.sig

I would appreciate if you can download and test it.  I plan to release
it in this month.

Please send feedback to bug-gettext@gnu.org.

Here is the current summary for NEWS.

* Programming languages support:
  - XML:
xgettext can now load custom string extraction rules supplied by
consumer projects.  The rules are written in XML, utilizing the
Internationalization Tag Set (ITS) standard.  All the existing
XML-based language scanners (Glade, GSettings, and AppData) are
rewritten using ITS.  In addition, msgfmt now has --xml option to
merge translations back to the original XML document.

* Portability:
  - Improve OS/2 kLIBC support (still not complete)
  - Remove dependency on expat

Thanks,
-- 
Daiki Ueno

[bug-gettext] removing expat dependency

[Daiki Ueno]

KO Myung-Hun  writes:

>> After merging the ITS branch (wip/ueno/its2), I plan to replace the
>> libexpat dependencies with libxml2 (for tools) and a hand written XML
>> parser (for libgettextpo, maybe ported from GLib's gmarkup.c).  So this
>> shouldn't be necessary.
>
> Then, @LTLIBEXPAT@ is removed from libgettextsrc, too ?

I did that (also merged ITS branch).  The relevant commit is:
http://git.sv.gnu.org/cgit/gettext.git/commit/?id=898e184a

Let me know if this change causes any compatibility problems.

The main obstacle was format-kde-kuit.c, which is shared between
xgettext and libgettexpo; while we can assume that libxml2 is always
usable from xgettext (through libgettextlib), it may not be usable from
libgettexpo.  So, I ported GLib's gmarkup.c as a local Gnulib module:
markup.  Currently, the use of the module is limited for libgettexpo for
some reasons (copyright, etc).

Regards,
-- 
Daiki Ueno

Re: [bug-gettext] gettext doc minor errors and typos

[Daiki Ueno]

Phil Davis  writes:

> I have attached a bunch of patches for various doc grammar and typos.

Thanks, will look into the patch soon.

> I couldn't see how to submit anything directly to the gettext.git repo
> for review. The submission and review process is so easy for projects
> that have repos hosted on Github - just fork/clone, edit away then
> press the Pull Request button.
>
> Is there a similar way for mere mortals to submit change requests for
> review to the repos in git.sv.gnu.org ?

No, gettext uses a traditional email-based workflow, like many other
free software projects, so you are doing it the right way.  I personally
don't think it is too much hurdle for contributing.

More importantly, we might need a copyright assignment, if the patch
contains legally significant changes:
http://www.gnu.org/prep/maintain/maintain.html#Legally-Significant

I will let you know if the paperwork is necessary.

Regards,
-- 
Daiki Ueno