Nice. Thank you for that patch info.
-----Original Message----- From: sqlite-users [mailto:[email protected]] On Behalf Of Olivier Mascia Sent: Monday, June 05, 2017 6:20 PM To: SQLite mailing list Subject: Re: [sqlite] [SPAM] Fwd: How to correctly display unicode characters in Windows 10 / cmd.exe / sqlite3.exe? > Le 5 juin 2017 à 20:44, Shane Dev <[email protected]> a écrit : > > I am already using Lucida Console font. Without chcp 65001, sqlite3.exe > will display LATIN SMALL LETTER E WITH ACUTE (U+00E9) correctly. But I > can't figure out how to display EURO SIGN (U+20AC). Lucida Console > definitely has the glyph, otherwise my test "type utf8test.txt" would have > failed. Also REGISTERED SIGN (U+00AE) is displayed as "r" (i.e. not > enclosed in a circle). Simon Slavin pointed out that Windows Console > doesn’t support multibyte characters. Despite that, other console programs > I use are able to display these characters correctly with code page 65001 > active - even a simple c program with printf("héllo wo®ld€"); works. > > Does anyone know how to display characters like these in the sqlite3.exe > command line utility under Windows 10 (either cmd.exe or powershell.exe)? Patch shell.c to get it to output to the console using WriteConsoleW and not fputs(). For a starter, see shell.c (3.19.2) around line 372 and following, function utf8_printf(). The intent is right (Windows needs some help to output UTF8), the implementation is not. For proper unicode console output on Windows you need to use WriteConsoleW(), and not the C library. shell.c @@ -380,10 +380,12 @@ va_start(ap, zFormat); if( stdout_is_console && (out==stdout || out==stderr) ){ char *z1 = sqlite3_vmprintf(zFormat, ap); - char *z2 = sqlite3_win32_utf8_to_mbcs_v2(z1, 0); + WCHAR *z2 = sqlite3_win32_utf8_to_unicode(z1); sqlite3_free(z1); - fputs(z2, out); - sqlite3_free(z2); + DWORD sout; + WriteConsoleW(GetStdHandle((out == stdout) ? STD_OUTPUT_HANDLE : STD_ERROR_HANDLE), + z2, wcslen(z2), &sout, 0); + sqlite3_free(z2); }else{ vfprintf(out, zFormat, ap); } And there you go: sqlite> select char(0x20ac); char(0x20ac) € sqlite> The important thing here is that it will work as long as the font selected in your command-prompt knows the glyph to code point 20ac. No matter if your Windows is 'western-', 'eastern-', or whatever-based. Also the command CHCP has NO effect on output produced by WriteConsoleW(). (This is just a quick dirty patch, I find it ugly to call wcslen() to get the number of wide chars of z2 while this information was known by the api which converted from utf8 to wide and could have been preserved. The GetStdHandle() calls might also better be made once, and not at each call. But the simpler the patch, the better it helps get the idea and experiment for yourself.) Then for the input handling (keyboard), you'll need an alternate implementation of local_getline(), lines 509 and further. The current implementation use the C library to read the console and then attempts to convert whatever MBCS it thinks it is to UTF8. It really needs to be reworked around ReadConsoleW() when the standard input is detected to really be the console. With these kind of changes, assuming this content is properly copied as utf8 into a file (unicode.sql for instance): create table €cole(proverb text); insert into €cole values('鱼与熊掌'); These commands work as expected, as long as your console font is set to something more complete than the usual Consolas (here using nsimsun on Server 2016 for this purpose): sqlite3 école.db ".read unicode.sql" And then: sqlite3 école.db SQLite version 3.19.2 2017-05-25 16:50:27 Enter ".help" for usage hints. sqlite> select * from €cole; proverb 鱼与熊掌 sqlite> -- Best Regards, Meilleures salutations, Met vriendelijke groeten, Olivier Mascia, http://integral.software _______________________________________________ sqlite-users mailing list [email protected] http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list [email protected] http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
