Re: svn commit: r362148 - head/contrib/nvi/common
Gleb Smirnoff wrote: Yuri, Zhihao, this commit totally broke Russian input for me in nvi. After exiting edit mode, nvi immediately converts all text to ???. I don't have any special settings in my environment. All I have is "russian" class for my user which yields in these environment variables: declare -x LANG="ru_RU.UTF-8" declare -x MM_CHARSET="UTF-8" declare -x XTERM_LOCALE="ru_RU.UTF-8" I'm already digging into that problem, but may be you have a clue immediately. My bad, yes, I see the problem, looking into it. On Sat, Jun 13, 2020 at 02:11:02PM +, Yuri Pankov wrote: Y> Author: yuripv Y> Date: Sat Jun 13 14:11:02 2020 Y> New Revision: 362148 Y> URL: https://svnweb.freebsd.org/changeset/base/362148 Y> Y> Log: Y> nvi: fallback to ISO8859-1 as last resort Y> Y> Current logic of using user's locale encoding that is UTF-8 doesn't make Y> much sense if we already failed the looks_utf8() check and skipped Y> encoding set using "fileencoding" as being UTF-8 as well; fallback to Y> ISO8859-1 in that case. Y> Y> Reviewed by:Zhihao Yuan Y> Differential Revision: https://reviews.freebsd.org/D24919 Y> Y> Modified: Y> head/contrib/nvi/common/exf.c Y> Y> Modified: head/contrib/nvi/common/exf.c Y> == Y> --- head/contrib/nvi/common/exf.c Sat Jun 13 09:16:07 2020(r362147) Y> +++ head/contrib/nvi/common/exf.c Sat Jun 13 14:11:02 2020(r362148) Y> @@ -1237,7 +1237,10 @@ file_encinit(SCR *sp) Y> } Y> Y> /* Y> - * Detect UTF-8 and fallback to the locale/preset encoding. Y> + * 1. Check for valid UTF-8. Y> + * 2. Check if fallback fileencoding is set and is NOT UTF-8. Y> + * 3. Check if user locale's encoding is NOT UTF-8. Y> + * 4. Use ISO8859-1 as last resort. Y>* Y>* XXX Y>* A manually set O_FILEENCODING indicates the "fallback Y> @@ -1246,9 +1249,13 @@ file_encinit(SCR *sp) Y>*/ Y> if (looks_utf8(buf, blen) > 1) Y> o_set(sp, O_FILEENCODING, OS_STRDUP, "utf-8", 0); Y> - else if (!O_ISSET(sp, O_FILEENCODING) || Y> - !strcasecmp(O_STR(sp, O_FILEENCODING), "utf-8")) Y> + else if (O_ISSET(sp, O_FILEENCODING) && Y> + strcasecmp(O_STR(sp, O_FILEENCODING), "utf-8") != 0) Y> + /* Use fileencoding as is */ ; Y> + else if (strcasecmp(codeset(), "utf-8") != 0) Y> o_set(sp, O_FILEENCODING, OS_STRDUP, codeset(), 0); Y> + else Y> + o_set(sp, O_FILEENCODING, OS_STRDUP, "iso8859-1", 0); Y> Y> conv_enc(sp, O_FILEENCODING, 0); Y> #endif ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r362148 - head/contrib/nvi/common
Yuri, Zhihao, this commit totally broke Russian input for me in nvi. After exiting edit mode, nvi immediately converts all text to ???. I don't have any special settings in my environment. All I have is "russian" class for my user which yields in these environment variables: declare -x LANG="ru_RU.UTF-8" declare -x MM_CHARSET="UTF-8" declare -x XTERM_LOCALE="ru_RU.UTF-8" I'm already digging into that problem, but may be you have a clue immediately. On Sat, Jun 13, 2020 at 02:11:02PM +, Yuri Pankov wrote: Y> Author: yuripv Y> Date: Sat Jun 13 14:11:02 2020 Y> New Revision: 362148 Y> URL: https://svnweb.freebsd.org/changeset/base/362148 Y> Y> Log: Y> nvi: fallback to ISO8859-1 as last resort Y> Y> Current logic of using user's locale encoding that is UTF-8 doesn't make Y> much sense if we already failed the looks_utf8() check and skipped Y> encoding set using "fileencoding" as being UTF-8 as well; fallback to Y> ISO8859-1 in that case. Y> Y> Reviewed by: Zhihao Yuan Y> Differential Revision: https://reviews.freebsd.org/D24919 Y> Y> Modified: Y> head/contrib/nvi/common/exf.c Y> Y> Modified: head/contrib/nvi/common/exf.c Y> == Y> --- head/contrib/nvi/common/exf.cSat Jun 13 09:16:07 2020 (r362147) Y> +++ head/contrib/nvi/common/exf.cSat Jun 13 14:11:02 2020 (r362148) Y> @@ -1237,7 +1237,10 @@ file_encinit(SCR *sp) Y> } Y> Y> /* Y> - * Detect UTF-8 and fallback to the locale/preset encoding. Y> + * 1. Check for valid UTF-8. Y> + * 2. Check if fallback fileencoding is set and is NOT UTF-8. Y> + * 3. Check if user locale's encoding is NOT UTF-8. Y> + * 4. Use ISO8859-1 as last resort. Y> * Y> * XXX Y> * A manually set O_FILEENCODING indicates the "fallback Y> @@ -1246,9 +1249,13 @@ file_encinit(SCR *sp) Y> */ Y> if (looks_utf8(buf, blen) > 1) Y> o_set(sp, O_FILEENCODING, OS_STRDUP, "utf-8", 0); Y> -else if (!O_ISSET(sp, O_FILEENCODING) || Y> -!strcasecmp(O_STR(sp, O_FILEENCODING), "utf-8")) Y> +else if (O_ISSET(sp, O_FILEENCODING) && Y> +strcasecmp(O_STR(sp, O_FILEENCODING), "utf-8") != 0) Y> +/* Use fileencoding as is */ ; Y> +else if (strcasecmp(codeset(), "utf-8") != 0) Y> o_set(sp, O_FILEENCODING, OS_STRDUP, codeset(), 0); Y> +else Y> +o_set(sp, O_FILEENCODING, OS_STRDUP, "iso8859-1", 0); Y> Y> conv_enc(sp, O_FILEENCODING, 0); Y> #endif Y> ___ Y> svn-src-all@freebsd.org mailing list Y> https://lists.freebsd.org/mailman/listinfo/svn-src-all Y> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org" -- Gleb Smirnoff ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
