Before Glenn finalizes updating message localization support in libast
we
should come to some consensus on how we expect this support to work.
As far as I can tell (correct me if I am wrong) there never really has
been
a real discussion on this list about this issue.
As I see it there are three scenarios:
- Support for localization of messages in compiled applications
which use AST libraries.
- Support for localization of ksh93 internal messages.
- Support for localization of messages in ksh93 scripts.
I believe that the third scenario is the one that is of current interest
to
myself and Wendy.
Besides the approach taken to message localization by ksh93 there are
a number of other approaches to handing shell script message
localization and I will briefly describe them as some people may not be
aware of them.
1. The catgets methodology as used in CDE's dtksh. It uses NLS message
catalogs. Here is a simple example of how it works.
# open message catalog
catopen CAT_ID "myCatalog.cat"
# get message 14 from set 1
catgets TEMP_STRING $CAT_ID 1 14 "Please check printer"
# and output the message
echo "$TEMP_STRING"
# Close the message catalog, when no longer needed
catclose $CAT_ID
Tru64 UNIX used a similar approach, i.e. dspmsg(1) as does AIX.
2. Bash uses gettext together with PO message catalogs whose layout
is totally different from NLS message catalogs. The following is a
simplified example of the layout a PO file.
white-space
# translator-comments
#. extracted-comments
#: reference...
#, flag...
#| msgid previous-untranslated-string
msgid1 untranslated-string
msgstr1 translated-string
......
msgidN untranslated-string
msgstrN translated-string
So what do I want of ksh93 as far as message localization is concerned?
- Understands and honors the $"..." syntax (Already does)
- Understands and honors the current locale environment as defined
by IEEE Std 1003.1-2008 (Already does)
- Honors all specified message sets - not just set 3. (Recently
fixed)
- Uses NLS message catalogs generated by the platform gencat utility.
This makes production of message catalogs much easier and enables
message catalog translators to work with existing toolsets.
- An additional message catalog search path ($PWD/locale) so I can
store localization script message catalogs under the directory
where
I store the shell scripts. (This can be done using NLSPATH but it
would be nice if it were on the default message catalog search
path)
- Accurate and complete documentation. No use having a feature in
the
shell if it is not properly documented!
I am sure others have different requirements. I suggest now is the time
to put them on the table.
- Finnbarr
> Date: Mon, 17 May 2010 21:56:07 +0200
> Subject: Re: [ast-users] Re: Re: [ast-developers] Seeking help with
> $"string", ksh -D, gencat and NLSPATH
> From: [email protected]
> To: [email protected]
> CC: [email protected]; [email protected]
>
> I can make a patch but it will be incomplete compared to what Glenn
> has done. If you read Glenn's last mail in this thread you'll see he
> has much more done than I could fix quickly...
>
> Olga
>
> On Mon, May 17, 2010 at 8:35 PM, Wendy Lin <[email protected]> wrote:
> > On 7 May 2010 07:14, Wendy Lin <[email protected]> wrote:
> >> Glenn, Olga, I would be very grateful if one of you could fix this for
> >> the next ksh93t release or make a patch available. Working, built in
> >> localization support in ksh93 would be a great benefit.
> >>
> >> On 3 May 2010 22:05, Glenn Fowler <[email protected]> wrote:
> >>>
> >>> Olga, thanks for looking into this and provding a link to Finbarr's blog
> >>> sometimes the todo list is too long to track down and reproduce every
> >>> problem
> >>>
> >>> there were a few problems that were bugs in libast:
> >>>
> >>> * $set==3 (ast) vs $set==1 (default) mismatches noted by Finbarr
> >>>
> >>> this is not necessarily a bug, but a strict ast convention that ast
> >>> commands and scripts only look for messages in $set==3; this broke the
> >>> principle of least surprise so ...
> >>>
> >>> this was fixed by translate() first checking for $set==3 and then $set==1
> >>>
> >>> this allows ast messages ($set==3) to be included in non-ast message
> >>> files
> >>> but also allows the "default" $set==1 to be found in files that do not
> >>> have a $set==3 message set
> >>>
> >>> * an underlying problem where ast may recognize a locale that the
> >>> native setlocale()+catopen() do not
> >>>
> >>> in particular ast with ast-locale installed recognizes "fr" as a valid
> >>> locale even though the native setlocale() requires territory
> >>> qualification
> >>> (e.g. "fr_FR")
> >>>
> >>> this was fixed by changing the ast local locale vs native standard
> >>> locale logic
> >>>
> >>> the reason for the locale vs native distinction is that ast provides
> >>> "local"
> >>> locales on systems that may not otherwise support that locale; e.g., on
> >>> a system
> >>> that does not have the "fr_FR" locale installed you can still get
> >>> ast-specific
> >>> "fr" locale message translation from the catalogs in the ast-locale
> >>> package
> >>>
> >>> ast also provides a "debug" local locale for debugging locale settings
> >>> and
> >>> message translation; running Finbarr's demo with LC_ALL=debug produces
> >>>
> >>> ksh93 message translation example
> >>> (demo,3,1)
> >>> (demo,3,2)
> >>> (demo,3,3)(debug,demo,libshell,No Translation available)
> >>>
> >>> (demo,3,1) means message 1 in set 3 in the demo catalog; 3 is the lookup
> >>> set,
> >>> if not found then set 1 will be used per the fix above
> >>>
> >>> (debug,demo,libshell,No Translation available) is for messages that are
> >>> not
> >>> in the C locale catalogs: search the demo catalog first, then the
> >>> libshell catalog,
> >>> for the message "No Translation available"
> >>>
> >>> * some other bugs were fixed to allow intermingling of standard catgen(1)
> >>> and
> >>> ast msggen(1) compiled catalogs (these are the file types that catopen()
> >>> must find)
> >>>
> >>> the reason for msggen(1) is the msggen(1) output is portable to all
> >>> machine architectures
> >>> this is how one ast-locale package can work on all hosttypes; similar
> >>> guarantees
> >>> cannot be made for catgen(1) output from different hosttypes
> >>>
> >>> the final misconception (and this is a results of ast documentation and
> >>> coding bugs)
> >>> in Finbarr's example and patch is the locations where ast looks for
> >>> message catalogs
> >>>
> >>> in ast just about every command-related data file lookup is done via the
> >>> ast pathpath()
> >>> pathpath() lookups use only the PATH environment variable to search for
> >>> related files
> >>> e.g.,
> >>> ../lib/file/magic for the file(1) magic file
> >>> ../lib/<plugin-class>/<plugin-shared-library> for runtime plugins
> >>> the idea being, if you have the PATH properly set up to find the std and
> >>> your own ast
> >>> commands/scripts, then the related file lookups will work
> >>>
> >>> the message catalog search order is (using NLSPATH notation):
> >>> getenv("NLSPATH")
> >>> pathpath("share/lib/locale/%l/%C/%N")
> >>> pathpath("share/locale/%l/%C/%N")
> >>> pathpath("lib/locale/%l/%C/%N")
> >>>
> >>> so to package a script susbsystem using ksh93 (with the libast fixes
> >>> mentioned above)
> >>> put scripts in $PACKAGEROOT/bin
> >>> put catgen(1) or msggen(1) catalogs in
> >>> $PACKAGEROOT/share/lib/locale/%l/LC_MESSAGES/<script-name>
> >>> (or share/locale/... or lib/locale/...)
> >>> put $PACKAGEROOT/bin in PATH
> >>> set LC_* to a supported locale
> >>>
> >>> if you use catgen(1) then you may have to provide os/architecture specific
> >>> catalogs in your package distribution; msggen(1) should work on all
> >>> os/architectures
> >>>
> >>> On Mon, 3 May 2010 02:14:50 +0200 =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?=
> >>> wrote:
> >>>> David, the problem is the name of the catalog. Instead of using the
> >>>> script name it always uses 'libshell' as the catalog name.
> >>>> I try to make a patch soon.
> >>>
> >>>> Olga
> >>>
> >>>> On Wed, Aug 12, 2009 at 4:44 PM, David Korn <[email protected]>
> >>>> wrote:
> >>>> > cc: [email protected]
> >>>> > Subject: Re: Re: [ast-developers] Seeking help with $"string", ksh -D,
> >>>> > gencat and NLSPATH
> >>>> > --------
> >>>> >
> >>>> >> On 06/07/2009, Wendy Lin <[email protected]> wrote:
> >>>> >> > On 22/05/2009, Finnbarr Murphy <[email protected]> wrote:
> >>>> >> > > In short, as far as I can see, ksh93 does not appear to support
> >>>> >> > localized
> >>>> >> > > message strings at present.
> >>>> >> >
> >>>> >> >
> >>>> >> > Can this be fixed for the next release, please?
> >>>> >
> >>>> >> After reading your blog at
> >>>> >> http://blog.fpmurphy.com/2009/05/ksh93-message-localization.html I
> >>>> >> feel very disappointed about ksh.
> >>>> >> If bash 4.0 would support fp math we would immediately migrate to it
> >>>> >> because it has the message localisation working.
> >>>> >>
> >>>> >
> >>>> >
> >>>> > ksh93 does support localized messages strings in scripts.
> >>>> >
> >>>> > Any string within a script that needs to be localized has to be
> >>>> > written as
> >>>> > $"string"
> >>>> > In the C or POSIX locale this will be treated as "string".
> >>>> >
> >>>> > If you run
> >>>> > ksh -D script
> >>>> > it will generate the list of strings that need localization.
> >>>> >
> >>>> > These can be put into a catalog for the C local.
> >>>> >
> >>>> > The catalogs for each locale can be created.
> >>>
> >>> _______________________________________________
> >>> ast-developers mailing list
> >>> [email protected]
> >>> https://mailman.research.att.com/mailman/listinfo/ast-developers
> >
> > Glenn, Olga, could you make a patch or a new alpha, beta or gamma
> > release which fixes this bug? I *beg* you to fix this as soon as
> > humanly possible.
> >
> > Wendy
> >
>
>
>
> --
> , _ _ ,
> { \/`o;====- Olga Kryzhanovska -====;o`\/ }
> .----'-/`-/ [email protected] \-`\-'----.
> `'-..-| / http://twitter.com/fleyta \ |-..-'`
> /\/\ Solaris/BSD//C/C++ programmer /\/\
> `--` `--`
> _______________________________________________
> ast-developers mailing list
> [email protected]
> https://mailman.research.att.com/mailman/listinfo/ast-developers
_______________________________________________
ast-users mailing list
[email protected]
https://mailman.research.att.com/mailman/listinfo/ast-users
