Re: [DIY BUG] bash wants en_US locale
Justin R. Knierim wrote: Nevermind, I just tried with the 6.2-pre3 LiveCD and I can't reproduce the error either. Sorry for the noise, I can't figure it out. The issue is probably that the old root password contained some non-ASCII character, and someone recently changed the password to pure ASCII. -- Alexander E. Patrakov -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Randy McMurchy wrote: Justin R. Knierim wrote these words on 02/09/06 09:50 CST: IMO he isn't blaming BLFS for problems with UTF-8. Then you and I read plain English differently. IMO, the following statement directly conflicts with what you say above: But it is indeed better to say no to UTF-8 support now, because of BLFS issues and because UTF-8 support is a property of the whole system, not just its base. I do not blame BLFS. I am just saying that it is a huge (but doable!) job for me to verify all BLFS packages. My point is that if we release both LFS and BLFS now, the reader will have the choice: either don't use UTF-8 based locales (then why have 100K patches in LFS?), or don't use BLFS as it is (then why have BLFS?). Released versions of LFS always expected the reader to be able to continue in BLFS, and my patch broke that. My opinion is that known 99.9% working BLFS has greater value than UTF-8 support in the base system. Thus, it is a good idea to postpone UTF-8 support in LFS trunk to the 6.3 release (even though it is nearly ready by itself, minus political issues), move it to a branch, and work with BLFS problems right now in the wiki. -- Alexander E. Patrakov -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Alexander E. Patrakov wrote: Justin R. Knierim wrote: Nevermind, I just tried with the 6.2-pre3 LiveCD and I can't reproduce the error either. Sorry for the noise, I can't figure it out. The issue is probably that the old root password contained some non-ASCII character, and someone recently changed the password to pure ASCII. Nope, nothing has changed on the system since the last tests. It is the server for lfs-matrix.net I logged into, only 3 people have access to it, and the passwords are the same. :-/ Oh well. Maybe it was a PEBKAC problem before, wouldn't be the first time. Justin -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Alexander E. Patrakov wrote these words on 02/08/06 22:52 CST: But it is indeed better to say no to UTF-8 support now, because of BLFS issues and because UTF-8 support is a property of the whole system, not just its base. The blacklist approach (If a package is not listed here, it means there are no known locale specific issues or problems with that package on http://www.linuxfromscratch.org/blfs/view/svn/introduction/locale-issues.html) effectively defeats the purpose of the page because the blackilst is always incomplete. Yes, this does mean that UTF-8 support in LFS is next to useless now. I don't remember reading any mail where you would prefer the text to be changed. Could you provide that URL for me? If I missed it, I apologize and you should have reminded me (someone). And, are you round-about (or perhaps even directly) saying that the UTF-8 support in LFS is useless because of BLFS issues? If so, may I suggest that you update the Wiki with all the information that it would take so that BLFS is not to blame for UTF-8 uselessness. Alexander, if BLFS is going to be your scapegoat, and excuse-bed, for UTF-8 ineptness, you should have thought of this before you implemented it into LFS. I realize you wanted to remove it after the fact, but it was too late then. *You* had already implemented it. Granted, I do remember you sending in something that never got to the book (if I remember correctly), but that was an update for one package, not to fix the sentence you quote above. If it *was* a fix for the sentence above and it was neglected, you need to start entering these issues into Trac, so there is a record of them. Understand, I am not taking it personal for you blaming BLFS for UTF-8's problems, I just wish you wouldn't do it, as it reflects poorly on BLFS. There's got to be a better way to tell someone that UTF-8 is not ready other than BLFS is broken, so don't use UTF-8. If that is truly the case, then *you* need to update the Wiki with information to fix it, and until then, please say to people instead of because of BLFS issues, something like I just haven't had time to update the BLFS Wiki with information to make our UTF-8 implementation complete as the reason for it being broke. -- Randy rmlscsi: [GNU ld version 2.15.94.0.2 20041220] [gcc (GCC) 3.4.3] [GNU C Library stable release version 2.3.4] [Linux 2.6.10 i686] 05:45:00 up 137 days, 15:09, 3 users, load average: 0.05, 0.10, 0.18 -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Randy McMurchy wrote: snip IMO he isn't blaming BLFS for problems with UTF-8. Alexander has publically stated that after adding UTF-8 to LFS, that maybe it really isn't ready for the main book. Even asking to revert it or put it in an experimental branch. http://archives.linuxfromscratch.org/mail-archives/lfs-dev/2006-January/055503.html http://archives.linuxfromscratch.org/mail-archives/lfs-dev/2006-January/055501.html Just FYI. So while there might be issues in BLFS to be solved, he isn't just putting the blame solely on you guys. /me goes back to lurking. :) Justin -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Justin R. Knierim wrote these words on 02/09/06 09:50 CST: IMO he isn't blaming BLFS for problems with UTF-8. Then you and I read plain English differently. IMO, the following statement directly conflicts with what you say above: But it is indeed better to say no to UTF-8 support now, because of BLFS issues and because UTF-8 support is a property of the whole system, not just its base. -- Randy rmlscsi: [GNU ld version 2.15.94.0.2 20041220] [gcc (GCC) 3.4.3] [GNU C Library stable release version 2.3.4] [Linux 2.6.10 i686] 10:06:01 up 137 days, 19:30, 3 users, load average: 0.99, 0.68, 0.32 -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Justin Knierim wrote: There are other problems, such as people not wanting the bloat, english speakers not seeing the need, etc. Just to clarify: As an English (only) speaker, I really don't want UTF-8 on *my* system. As a BLFS developer, I certainly understand the desire and need for UTF-8 in the books. My only problem with LFS on the issue is that LFS traditionally has not offered choices. That is, there is only one true path through the book. This makes users who do not need or want UTF-8 proceed on their own. IMO, this is not desireable. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Joel Miller wrote: Jeremy Huntwork wrote: snip And to further clarify, an LFS machine as it currently is does *not* automatically use UTF-8. The changes to the LFS book are so that the LFS system *can* use UTF-8 properly if a user requires it. IMO, this is snip I'd like some clarification on one point then. IIRC, I read in a post by Alexander that the changes to the book made it so that ssh would no longer connect properly to non UTF-8 systems. In order to do that you had to run ssh through screen. This would be a *massive* inconvenience issue for me. Is this the case just by following the new book instructions, or is this only the case if you set your locale to a UTF-8 locale? This is only if you use a UTF-8 locale. AFAIK if you build a current LFS system, with UTF-8 support, you should see absolutely no difference if you use a non-UTF-8 locale. -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Joel Miller wrote: Jeremy Huntwork wrote: snip And to further clarify, an LFS machine as it currently is does *not* automatically use UTF-8. The changes to the LFS book are so that the LFS system *can* use UTF-8 properly if a user requires it. IMO, this is snip I'd like some clarification on one point then. IIRC, I read in a post by Alexander that the changes to the book made it so that ssh would no longer connect properly to non UTF-8 systems. In order to do that you had to run ssh through screen. This would be a *massive* inconvenience issue for me. Is this the case just by following the new book instructions, or is this only the case if you set your locale to a UTF-8 locale? Also, I recall him saying that the issue with ssh is not a bug - it has something to do with the fact that ssh transfers bytes - not characters (or something like that...) and is part of the basic nature of the workings of ssh. -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Jeremy Huntwork wrote: I tried looking for the error using a recently built LFS LiveCD. I've been using en_US.UTF-8 as a locale with it recently, and ssh both incoming and outgoing has worked fine for me. Logging in is fine. Once you are in, try su - root. Justin -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Justin R. Knierim wrote: Jeremy Huntwork wrote: I tried looking for the error using a recently built LFS LiveCD. I've been using en_US.UTF-8 as a locale with it recently, and ssh both incoming and outgoing has worked fine for me. Logging in is fine. Once you are in, try su - root. Worked fine here... I will note that, seeing that I'm using a german locale, my keys were a bit askew. The 'z' and the 'y' were reversed and the '-' was where the '/' key is. Other than that, it worked fine. -- JH -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Jeremy Huntwork wrote: Worked fine here... I will note that, seeing that I'm using a german locale, my keys were a bit askew. The 'z' and the 'y' were reversed and the '-' was where the '/' key is. Other than that, it worked fine. Hmm, strange. I wasn't using the German profile when I reported that problem, it was the normal en_US.UTF-8. Anyways, I'll try it again tonight and see what I can find. Justin -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Justin R. Knierim wrote: Hmm, strange. I wasn't using the German profile when I reported that problem, it was the normal en_US.UTF-8. Anyways, I'll try it again tonight and see what I can find. Nevermind, I just tried with the 6.2-pre3 LiveCD and I can't reproduce the error either. Sorry for the noise, I can't figure it out. P.S. Also i am sooo happy now that Xorg 7.0 works (as on 6.2-pre3 LiveCD) and doesn't freeze on my notebook video card! yay! :) Justin -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Alexander E. Patrakov wrote: some tme ago I imported from DIY linux the statement that none of the locales are really required. As it stands now, this statement is wrong. When checking whether the ctype macros accept non-ascii characters, Bash configure script attempts to set the en_US.ISO8859-1 locale. However, it may or may not be present, thus the result may vary. Correct. However, the problem is with Readline and not Bash. The configure check you refer to affects the define of CTYPE_NON_ASCII. This in turn affects chardefs.h: #if defined (CTYPE_NON_ASCII) # define NON_NEGATIVE(c) 1 #else # define NON_NEGATIVE(c) ((unsigned char)(c) == (c)) #endif Now grep the Readline source for NON_NEGATIVE and you can see how it makes a difference. Two solutions: 1) mark the en_US locale as required, don't change bash instructions 2) don't mark any locales as required, add bash_cv_func_ctype_nonascii=yes to the end of bash configure line My preference is (2). Please don't import this into HLFS, because I think that no would be the correct result for uClibc. I'll probably go for (2) in DIY (but of course in Readline, not Bash) Some other comments: - The configure check in question is skipped when cross compiling - The Bash testsuite will gain greater coverage if the locale en_US.UTF-8 is installed - If you're wise you will not rely on anything I (or DIY) say about internationalization. Me being a native English speaker I have no personal need for it and because I'm a selfish b*stard I'm not likely to become an expert anytime soon :-) - I'd appreciate it if you could report DIY issues to the DIY list or to me personally. Thanks. Regards Greg PS - Slightly OT but in case you haven't seen it, there is a reasonable (but possibly inaccurate) writeup of UTF-8 issues here: http://www.linux.com/article.pl?sid=06/01/26/210214 PPS - Some folks have asked me about UTF-8. In short, I have no clue. I admire the distros for forging ahead. I also admire the work Alex has done to try and get it working for the rest of us because it's clearly the way of the future. But for me right now, I feel it's simply too disruptive and 100K+ patches are not on my agenda. I'd rather wait for upstream packages to catch up before trying to support UTF-8. Yes, it might take a while.. -- http://www.diy-linux.org/ -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [DIY BUG] bash wants en_US locale
Greg Schafer wrote: Correct. However, the problem is with Readline and not Bash. You are of course right. Thanks for the correction. I noticed this in Chapter 5 as a difference between Glibc-based LFS and uClibc-based HLFS. That's why I thought it's bash. Now grep the Readline source for NON_NEGATIVE and you can see how it makes a difference. It seems that _rl_lowercase_p(c) macro uses this, and is used in vi_mode.c Some other comments: - The configure check in question is skipped when cross compiling Correct, and a pessimistic assumption is made then. - The Bash testsuite will gain greater coverage if the locale en_US.UTF-8 is installed Already covered in LFS, because of patched coreutils. Thanks anyway for the report. - If you're wise you will not rely on anything I (or DIY) say about internationalization. Me being a native English speaker I have no personal need for it and because I'm a selfish b*stard I'm not likely to become an expert anytime soon :-) Point taken. - I'd appreciate it if you could report DIY issues to the DIY list or to me personally. Thanks. Do I understand correctly that bugs which affect both books have to be reported to both lists? PS - Slightly OT but in case you haven't seen it, there is a reasonable (but possibly inaccurate) writeup of UTF-8 issues here: http://www.linux.com/article.pl?sid=06/01/26/210214 Indeed, incomplete and inaccurate. But still partially useful. Some of inaccuracies below are already mentioned in comments. 1) The article assumes a distro where a distro-maker has applied all UTF-8 patches 2) Option XkbLayout en_us.utf-8 is just wrong, UTF-8 and other people use the same X keyboard layout 3) Missing link to DejaVu fonts 4) export GTK_IM_MODULE=xim works, but is suboptimal if the input method program supports GTK2 natively. In such case (e.g. with SCIM), export GTK_IM_MODULE=scim yields better results, e.g. avoids locking the whole system if one program misbehaves. Setting up XIM itself (for non-GTK apps) is not described at all. 5) For touch typists used to a legacy English locale, probably the greatest annoyance is the need to press the apostrophe key followed by the space bar to type a straight quotation mark. Wrong, applies only to a wronly chosen XkbLayout which contains dead keys. Plain us layout (that doesn't contain dead keys) works as before. 6) enter extended HTML characters in decimal format. Wrong, you don't need it if you serve UTF-8 HTML pages with the correct content-type text/html; charset=UTF-8. 7) you may need to configure Mozilla to use xprint. Was true not too long ago. Wrong for Seamonkey-1.0, which features Freetype-based printing and Pango-based layout. 8) BASH can display UTF-8 characters, but cannot accept them as input. This limitation may be due to legacy system fonts, or to limitations in UTF-8 support in the kernel. This is the kernel issue, which is solved in the LFS book BTW. I was asked to submit the kernel patch for 2.6.17. || PPS - Some folks have asked me about UTF-8. In short, I have no clue. I admire the distros for forging ahead. I also admire the work Alex has done to try and get it working for the rest of us because it's clearly the way of the future. But for me right now, I feel it's simply too disruptive and 100K+ patches are not on my agenda. I'd rather wait for upstream packages to catch up before trying to support UTF-8. Yes, it might take a while.. You can as well implement (but see below) a minimal acceptable subset of what is in the LFS book: 1) ncursesw 2) grep bracket patch from RedHat 3) Pass +lang none to Man, say that it's the reader's responsibility to remove all non-ISO-8859-1 manual pages, don't install them yourself with Shadow. Point readers who want non-ISO-8859-1 manual pages to my man-i18n hint. 4) sysklogd 8-bit patch 5) mention that the contents of /etc/inittab is closely tied with bootscripts 6) Pass --enable-multibyte to vim 7) Recommend that the X Window System should be used instead of text console with UTF-8 locales due to a kernel bug related to keyboard input. As you see, 100K-patches are outside of the proposed steps :) But the fact is that (due to LSB) patched coreutils, grep and diffutils are tested better than non-patched ones. Rationale for not including other UTF-8 stuff that is in LFS: 1) 100K-patches: they are not a complete solution and don't stop me from abusing perl. 2) Glibc-libidn: it is optional because there are not many IDN sites 3) Removal of non-working Glibc locales: one may as well just not use them. 4) Debian Groff-1.18.1.1 + DB + Man-DB: Jim Gifford says it's overkill. Groff-1.19.2 + Man works fine for formatting ISO-8859-1 manuals. Hacks exist to make it show Russian. Japanese does require the Man setup in the LFS book. So does the LiveCD configuration where the need to reconfigure anything except $LANG is automatically a bug. 5) Kbd backspace fix: nice but
