Re: [GNC-dev] Early locale initialization
Hi, Am 20.03.21 um 17:43 schrieb John Ralls: > You're right about the order, but doesn't guile call setlocale before gtk > does? That's significant only because Guile spawns a bunch of threads and > those keep the locale generated by its setlocale call. The main thread that > GnuCash and Gtk use is governed by the last setlocale call in the main thread > which is generally the one in gtk_init. I think that is the reason, why I get when testing in C > echo $LANG de_DE.UTF-8 > echo $LANGUAGE > echo $LC_ALL > LANG=C gnucash --nofile Diese Version befindet sich noch in Entwicklung. Sie kann funktionieren, muss aber nicht. Fehler und andere Probleme werden auf gnucash-devel@gnucash.org diskutiert. Fehlerberichte können hier eingesehen und erstellt werden: https://bugs.gnucash.org Um die letzte stabile Version zu finden, sehen Sie bitte hier nach: https://www.gnucash.org/ Found Finance::Quote version 1.49. (all in German) and the the GUI in en_US Regards Frank ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Early locale initialization
> On Mar 20, 2021, at 3:10 AM, Geert Janssens > wrote: > > Op vrijdag 19 maart 2021 02:19:46 CET schreef John Ralls: >>> On Mar 18, 2021, at 6:08 AM, Geert Janssens >>> wrote:> >>> Op donderdag 18 maart 2021 13:56:35 CET schreef Derek Atkins: On Thu, March 18, 2021 7:58 am, Geert Janssens wrote: [snip] > 3. While all three allow to tweak some environment parameters, only > linux > allows the user to do so via the environment file. For Windows and Macos > that > file is only read after locale is set. Whether that's a good thing > depends > on > the answer to question 2. Are you saying that /etc/gnucash/environment (or the equivalent thereof) is not read/used on Windows or Mac? >>> >>> No, it is used on all platforms. But it's only read after locale >>> initialization on mac/windows has been done. So for locale initialization >>> it only will affect linux. The other parameters in there (like >>> LD_LIBRARY_PATH tweaks and so on) will still affect all platforms. >> >> No, it's read after the other initializations so that it will override them. >> As long as setlocale is called after loading the environment then the last >> computed locale is what GnuCash starts up with. I've tested that pretty >> extensively on both macOS and Win32, though not since last fall. >> > Running this through a debugger I see setlocale gets called indeed afterwards > but more by chance than by choice (the first time during gtk's > initialization). > > To zoom in on this: if I get you right the idea is effectively to allow users > to set LC_ALL in the environment file and make it override whatever system > locale was found (or fallen back to). > > The code and comment in https://github.com/Gnucash/gnucash/blob/maint/gnucash/ > gnucash-core-app.cpp#L424 certainly don't suggest that. For Macos and Windows > it explicitly excludes the call to setlocale, with the suggestion it was > already done. That's contrary to that intent. > > I guess the part that complicates this area is that on both Windows and Macos > there are two locales to take into account - the posix locale and the > platform > native one. > > The windows branch of this code explicitly tries to set both to the same > locale. If it finds LC_ALL, LC_MESSAGES or LANG it will use that for posix > and > then set the same for the Windows locale. In the absence of any of these it > will start from the Windows locate and use that to also set the posix locale. > To me that suggests the environment overrides should happen before calling > this code. If you do it afterwards like we do now you may end up with out of > sync Windows and Posix locales. I have no idea if that would have > side-effects > but I can imagine this could result in some obscure locale related corner > case > bugs. > > If I read the mac branch of the code correctly, it always starts from the > Macos locale and tries to derive a posix locale from it. A user can > afterwards > override that posix locale via settings in the environment file. The mac > locale will never be updated to match any posix locale though. Again I don't > know if that could be cause for issues. > > Do you think a reverse mapping as in the Windows environment would be > possible > ? I mean if users set LC_ALL (or related), can we use that to set the Macos > locale for the gnucash process space ? I don't know enough of the platform to > know whether Macos allows an application to change locales. Geert, You're right about the order, but doesn't guile call setlocale before gtk does? That's significant only because Guile spawns a bunch of threads and those keep the locale generated by its setlocale call. The main thread that GnuCash and Gtk use is governed by the last setlocale call in the main thread which is generally the one in gtk_init. GnuCash uses libc localization and at that level there's no difference at all between Linux and macOS: They're both flavors of unix. macOS has two "native" locales: The low-level standard C one and a more nuanced implemented via (wrapped and renamed) ICU locales. The macOS implementations of bash, csh, tcsh, and zsh set the LC_FOO environment variables from those more nuanced values but since GnuCash doesn't invoke a shell we have to do likewise in set_mac_locale in order to get the libc locale set up to match the user's preferences. Of course the user can also run GnuCash from the command line so we have to respect the environment passed in from the shell, and then there's the environment file. Windows is a bit different because while it implements the standard C localization functions it does it its own way. Guile complicates it a bit because they call (IIRC via GnuLib, haven't looked at their code in a while) the Win32 function SetThreadLocale() instead of the standard C setlocale so set_win32_thread_locale has to make sure that both are set up to the same locale or things get we
Re: [GNC-dev] Early locale initialization
Op vrijdag 19 maart 2021 02:19:46 CET schreef John Ralls: > > On Mar 18, 2021, at 6:08 AM, Geert Janssens > > wrote:> > > Op donderdag 18 maart 2021 13:56:35 CET schreef Derek Atkins: > >> On Thu, March 18, 2021 7:58 am, Geert Janssens wrote: > >> [snip] > >> > >>> 3. While all three allow to tweak some environment parameters, only > >>> linux > >>> allows the user to do so via the environment file. For Windows and Macos > >>> that > >>> file is only read after locale is set. Whether that's a good thing > >>> depends > >>> on > >>> the answer to question 2. > >> > >> Are you saying that /etc/gnucash/environment (or the equivalent thereof) > >> is not read/used on Windows or Mac? > > > > No, it is used on all platforms. But it's only read after locale > > initialization on mac/windows has been done. So for locale initialization > > it only will affect linux. The other parameters in there (like > > LD_LIBRARY_PATH tweaks and so on) will still affect all platforms. > > No, it's read after the other initializations so that it will override them. > As long as setlocale is called after loading the environment then the last > computed locale is what GnuCash starts up with. I've tested that pretty > extensively on both macOS and Win32, though not since last fall. > Running this through a debugger I see setlocale gets called indeed afterwards but more by chance than by choice (the first time during gtk's initialization). To zoom in on this: if I get you right the idea is effectively to allow users to set LC_ALL in the environment file and make it override whatever system locale was found (or fallen back to). The code and comment in https://github.com/Gnucash/gnucash/blob/maint/gnucash/ gnucash-core-app.cpp#L424 certainly don't suggest that. For Macos and Windows it explicitly excludes the call to setlocale, with the suggestion it was already done. That's contrary to that intent. I guess the part that complicates this area is that on both Windows and Macos there are two locales to take into account - the posix locale and the platform native one. The windows branch of this code explicitly tries to set both to the same locale. If it finds LC_ALL, LC_MESSAGES or LANG it will use that for posix and then set the same for the Windows locale. In the absence of any of these it will start from the Windows locate and use that to also set the posix locale. To me that suggests the environment overrides should happen before calling this code. If you do it afterwards like we do now you may end up with out of sync Windows and Posix locales. I have no idea if that would have side-effects but I can imagine this could result in some obscure locale related corner case bugs. If I read the mac branch of the code correctly, it always starts from the Macos locale and tries to derive a posix locale from it. A user can afterwards override that posix locale via settings in the environment file. The mac locale will never be updated to match any posix locale though. Again I don't know if that could be cause for issues. Do you think a reverse mapping as in the Windows environment would be possible ? I mean if users set LC_ALL (or related), can we use that to set the Macos locale for the gnucash process space ? I don't know enough of the platform to know whether Macos allows an application to change locales. > The one thing that doesn't work is passing LANG/LANGUAGES in the Win32 CMD > shell because that shell doesn't understand that. It should be possible on > Powershell with something like $Env:LANG=nl_BE "c:\Program Files > (x86)\gnucash\bin\gnucash.exe"; I've never actually tried that. > Well, you can do the same in a Win32 CMD shell, but not on a single line. set LANG=nl_BE "c:\Program Files (x86)\gnucash\bin\gnucash.exe" Will do it. > On macOS the language fallback is used because Apple's localization can > result in locales that libc thinks aren't legal, e.g. en_BE so we have to > find something close or the user gets "C". That's done only when getting > the locale from system preferences (aka "defaults"). I consider that a very nice refinement. Regards, Geert ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Early locale initialization
> On Mar 18, 2021, at 6:08 AM, Geert Janssens > wrote: > > Op donderdag 18 maart 2021 13:56:35 CET schreef Derek Atkins: >> On Thu, March 18, 2021 7:58 am, Geert Janssens wrote: >> [snip] >> >>> 3. While all three allow to tweak some environment parameters, only linux >>> allows the user to do so via the environment file. For Windows and Macos >>> that >>> file is only read after locale is set. Whether that's a good thing depends >>> on >>> the answer to question 2. >> >> Are you saying that /etc/gnucash/environment (or the equivalent thereof) >> is not read/used on Windows or Mac? >> > No, it is used on all platforms. But it's only read after locale > initialization on mac/windows has been done. So for locale initialization it > only will affect linux. The other parameters in there (like LD_LIBRARY_PATH > tweaks and so on) will still affect all platforms. > No, it's read after the other initializations so that it will override them. As long as setlocale is called after loading the environment then the last computed locale is what GnuCash starts up with. I've tested that pretty extensively on both macOS and Win32, though not since last fall. The one thing that doesn't work is passing LANG/LANGUAGES in the Win32 CMD shell because that shell doesn't understand that. It should be possible on Powershell with something like $Env:LANG=nl_BE "c:\Program Files (x86)\gnucash\bin\gnucash.exe"; I've never actually tried that. On macOS the language fallback is used because Apple's localization can result in locales that libc thinks aren't legal, e.g. en_BE so we have to find something close or the user gets "C". That's done only when getting the locale from system preferences (aka "defaults"). Regards, John Ralls ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Early locale initialization
Op donderdag 18 maart 2021 13:56:35 CET schreef Derek Atkins: > On Thu, March 18, 2021 7:58 am, Geert Janssens wrote: > [snip] > > > 3. While all three allow to tweak some environment parameters, only linux > > allows the user to do so via the environment file. For Windows and Macos > > that > > file is only read after locale is set. Whether that's a good thing depends > > on > > the answer to question 2. > > Are you saying that /etc/gnucash/environment (or the equivalent thereof) > is not read/used on Windows or Mac? > No, it is used on all platforms. But it's only read after locale initialization on mac/windows has been done. So for locale initialization it only will affect linux. The other parameters in there (like LD_LIBRARY_PATH tweaks and so on) will still affect all platforms. Geert ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Early locale initialization
On Thu, March 18, 2021 7:58 am, Geert Janssens wrote: [snip] > 3. While all three allow to tweak some environment parameters, only linux > allows the user to do so via the environment file. For Windows and Macos > that > file is only read after locale is set. Whether that's a good thing depends > on > the answer to question 2. Are you saying that /etc/gnucash/environment (or the equivalent thereof) is not read/used on Windows or Mac? > Regards, > > Geert -derek -- Derek Atkins 617-623-3745 de...@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Early locale initialization
Op donderdag 18 maart 2021 12:58:28 CET schreef Geert Janssens: > On Windows the code will first look for environment variables "LC_ALL", > "LC_MESSAGES" or "LANG" (in that order). If any of those exists it will try > to convert that to a locale. If that fails it will fall back to the system > default locale and if that fails it falls through to our main code which > then defaults to "C". This is not even true. The way it is currently implemented it won't reach the fallback-to-C code at all. Geert ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Early locale initialization
In addition, the Macos code doesn't return the final locale string that was set. As a result the PINFO message to log the locale details to the trace file is partly bogus. Granted this was written with the Windows locale issues in mind, but again, it would be nice if this could be made consistent. Regards, Geert Op donderdag 18 maart 2021 12:58:28 CET schreef Geert Janssens: > In a moment of distraction I started looking at the locale initialization > code in gnucash-core-app.cpp. > > If I understand the actual code properly it's not exactly working as I had > intended. So with this thread, I'd like to clear this out. > > For reference I'm talking about the code that starts here: > https://github.com/Gnucash/gnucash/blob/ > 1756f76079221f77904ccb9c9a504f9ebca2eb0e/gnucash/gnucash-core-app.cpp#L418 > > Locale initialization is strongly influenced by platform, so there are > different code paths to set locale. > 1. Macos will use set_mac_locale > 2. Windows will use set_win32_thread_locale > 3. Linux will fall through and use setlocale > > So far so good. > > Looking deeper, the linux method will search for an environment variables > LC_ALL or LANG in that order and uses its value as locale. If that fails, it > will fall back to the C locale. There is also code in place to allow the > user to override any of these variables by setting them in the gnucash > environment file. > > On Macos the code will first try to use the Macos proper locale setup code. > If that fails it falls back to locale "en_US.utf-8". Then it sets the LANG > variable to the same locale and proceeds with calling gnc_localeconv which > will fill in the locale details that will be used throughout the code. > > On Windows the code will first look for environment variables "LC_ALL", > "LC_MESSAGES" or "LANG" (in that order). If any of those exists it will try > to convert that to a locale. If that fails it will fall back to the system > default locale and if that fails it falls through to our main code which > then defaults to "C". > > > Putting these next to each other there are a few differences: > 1. the final fallback locale. On Macos that will be "en_US.utf-8", on the > other platforms it will be "C". Should we use the same default on all or > rather not ? > 2. Priority of locale source. On linux and Windows adding variables in the > environment will allow the user to override the default system locale. On > Macos it will only allow to set LANG or LANGUAGE variables, not locale. Do > we want to allow the user to override system locale via environment > variables or do we only want a fallback mechanism ? Do we want to allow the > user specify the locale to use for that fallback ? > I'll note the Macos code has a neat fallback trick in that it tries to find > a related locale in case the default doesn't work. Related in the sense > that it may be another language in same country or if that fails the > language without country. Downside is that in case of multiple fallback > options the one chosen may not be the one the user actually would have > preferred. > 3. While all three allow to tweak some environment parameters, only linux > allows the user to do so via the environment file. For Windows and Macos > that file is only read after locale is set. Whether that's a good thing > depends on the answer to question 2. > > Regards, > > Geert > > > ___ > gnucash-devel mailing list > gnucash-devel@gnucash.org > https://lists.gnucash.org/mailman/listinfo/gnucash-devel ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
