ksc5601.h needs to be updated to be consistent with iconv. otherwise, charset_wctocs() can't find appropriate charset for those characters in EUC-KR locale.
-federic
Choe Hwanjin wrote:
I make utf8 text file for test. It contains utf8-encoded ksx1001(ksc5601) chars. When I convert it to EUC-KR with iconv, no conversion error occur.
On X environment, I made TextProperty from Xutf8TextListToTextProperty and I converted the generated text property to the TextList by calling XmbTextPropertyToTextList. On UTF-8 locale, there was no error. But on ko_KR.EUC-KR locale, some errors occured. Almost chars converted successfully, but below chars did not:
Unconvertable: U+00B7 [�(2)] Unconvertable: U+00A8 [�(2)] Unconvertable: U+00AD [�(2)] Unconvertable: U+00B1 [�(2)] Unconvertable: U+00D7 [�(2)] Unconvertable: U+00F7 [�(2)] Unconvertable: U+00B0 [�(2)] Unconvertable: U+00A7 [�(2)] Unconvertable: U+00B4 [�(2)] Unconvertable: U+00B8 [�(2)] Unconvertable: U+00A1 [�(2)] Unconvertable: U+00BF [�(2)] Unconvertable: U+00A4 [�(2)] Unconvertable: U+00B6 [�(2)] Unconvertable: U+20AC [�(3)] Unconvertable: U+00AE [�(2)] Unconvertable: U+00C6 [�(2)] Unconvertable: U+00D0 [�(2)] Unconvertable: U+00AA [�(2)] Unconvertable: U+00D8 [�(2)] Unconvertable: U+00BA [�(2)] Unconvertable: U+00DE [�(2)] Unconvertable: U+00BD [�(2)] Unconvertable: U+00BC [�(2)] Unconvertable: U+00BE [�(2)] Unconvertable: U+00E6 [�(2)] Unconvertable: U+00F0 [�(2)] Unconvertable: U+00F8 [�(2)] Unconvertable: U+00DF [�(2)] Unconvertable: U+00FE [�(2)] Unconvertable: U+00B9 [�(2)] Unconvertable: U+00B2 [�(2)] Unconvertable: U+00B3 [�(2)]
I think this is not desirable result. Above chars are define in ksx1001, so iconv converts them correctly. But on X, it fails. Is it right behavior?
So I made small test case code and attached those files. You can reproduce this problem by following this instructions:
$ gmake $ LANG=ko_KR.UTF-8 ./conversion < utf8.txt # no error $ LANG=ko_KR.euckr ./conversion < utf8.txt # error $ iconv -f utf8 -t euckr utf8.txt > /dev/null # no error
PS. I'm not a member of this mailing list, so please cc me.
------------------------------------------------------------------------
CFLAGS = -I/usr/X11R6/include $(shell pkg-config --cflags glib-2.0) LIBS = -L/usr/X11R6/lib -lX11 $(shell pkg-config --libs glib-2.0)
all:
gcc -Wall -ggdb $(CFLAGS) $(LIBS) -o conversion conversion.c
------------------------------------------------------------------------
#include <stdio.h> #include <string.h> #include <locale.h> #include <X11/Xlib.h> #include <X11/Xutil.h> #include <glib.h>
static Display *display;
void textpropertytest(char *utf8) { int n, ret; XTextProperty text_prop; char *list[2]; char **return_list = NULL;
list[0] = utf8; list[1] = 0; n = 1; ret = Xutf8TextListToTextProperty(display, list, n, XCompoundTextStyle, &text_prop); if (ret != Success) { fprintf(stderr, "Error on utf8 -> TextProperty (%d)\n", ret); } else { ret = XmbTextPropertyToTextList(display, &text_prop, &return_list, &n); if (ret != Success) { printf("Unconvertable: U+%04X [%s(%d)]\n", g_utf8_get_char(utf8), utf8, strlen(utf8)); } XFreeStringList(return_list); } }
int main(void) { char *locale_name; char buf[256]; char utf8char[10]; const gchar *end; gchar *ptr; gchar *strend;
locale_name = setlocale(LC_ALL, ""); fprintf(stderr, "Locale: %s\n", locale_name);
display = XOpenDisplay(NULL); if (display == NULL) { fprintf(stderr, "Error on XOpenDisplay\n"); return 1; }
while (!feof(stdin)) { fgets(buf, sizeof(buf), stdin); if (g_utf8_validate(buf, -1, &end)) { ptr = buf; strend = strchr(buf, '\0'); while (ptr < strend) { memset(utf8char, 0, sizeof(utf8char)); g_utf8_strncpy(utf8char, ptr, 1); if (strlen(utf8char) > 0) textpropertytest(utf8char); ptr = g_utf8_find_next_char(ptr, NULL); } } else { fprintf(stderr, "Invalid utf8 string: %s\n", end); } }
XCloseDisplay(display);
return 0; }
------------------------------------------------------------------------
A1A0:??�..��?�--?\?�� A1B0:��????????????�� A1C0:�?????�????????? A1D0:???????�???????? A1E0:???????????????? A1F0:??????????????? A2A0:????�~?????�?�� A2B0:????�?�????????? A2C0:???????????????? A2D0:??���??????????? A2E0:No?�??TEL�� A2F0: A3A0:!"#$%&'()*+,-./ A3B0:0123456789:;<=>? A3C0:@ABCDEFGHIJKLMNO A3D0:PQRSTUVWXYZ[?]^_ A3E0:`abcdefghijklmno A3F0:pqrstuvwxyz{|}? A4A0:??????????????? A4B0:???????????????? A4C0:???????????????? A4D0:???????????????? A4E0:???????????????? A4F0:??????????????? A5A0:iiiiiiivvviviiviiiixx A5B0:IIIIIIIVVVIVIIVIIIIXX A5C0:??????????????? A5D0:????????? A5E0:??????????????? A5F0:????????? A6A0:??????????????? A6B0:???????????????? A6C0:???????????????? A6D0:???????????????? A6E0:????? A6F0: A7A0:???l??????????? A7B0:???????????????? A7C0:???????????????? A7D0:???????????????? A7E0:???????????????? A7F0: A8A0:�Ъ?IJL.?،��?? A8B0:??????????????? A8C0:?????????????(a)(b)(c) A8D0:(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s) A8E0:(t)(u)(v)(w)(x)(y)(z)(1)(2)(3)(4)(5)(6)(7)(8)(9) A8F0:(10)(11)(12)(13)(14)(15)�1/32/3��1/83/85/87/8 A9A0:�?�??ij?l.?����?? A9B0:n??????????????? A9C0:?????????????(a)(b)(c) A9D0:(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s) A9E0:(t)(u)(v)(w)(x)(y)(z)(1)(2)(3)(4)(5)(6)(7)(8)(9) A9F0:(10)(11)(12)(13)(14)(15)���^(4)^(n)v(1)v(2)v(3)v(4) AAA0:??????????????? AAB0:???????????????? AAC0:???????????????? AAD0:???????????????? AAE0:???????????????? AAF0:???? ABA0:??????????????? ABB0:???????????????? ABC0:???????????????? ABD0:???????????????? ABE0:???????????????? ABF0:??????? ACA0:??????????????? ACB0:???????????????? ACC0:?? ACD0:??????????????? ACE0:???????????????? ACF0:?? ADA0: ADB0: ADC0: ADD0: ADE0: ADF0: AEA0: AEB0: AEC0: AED0: AEE0: AEF0: AFA0: AFB0: AFC0: AFD0: AFE0: AFF0: B0A0:??????????????? B0B0:???????????????? B0C0:???????????????? B0D0:???????????????? B0E0:???????????????? B0F0:??????????????? B1A0:??????????????? B1B0:???????????????? B1C0:???????????????? B1D0:???????????????? B1E0:???????????????? B1F0:??????????????? B2A0:??????????????? B2B0:???????????????? B2C0:???????????????? B2D0:???????????????? B2E0:???????????????? B2F0:??????????????? B3A0:??????????????? B3B0:???????????????? B3C0:???????????????? B3D0:???????????????? B3E0:???????????????? B3F0:??????????????? B4A0:??????????????? B4B0:???????????????? B4C0:???????????????? B4D0:???????????????? B4E0:???????????????? B4F0:??????????????? B5A0:??????????????? B5B0:???????????????? B5C0:???????????????? B5D0:???????????????? B5E0:???????????????? B5F0:??????????????? B6A0:??????????????? B6B0:???????????????? B6C0:???????????????? B6D0:???????????????? B6E0:???????????????? B6F0:??????????????? B7A0:??????????????? B7B0:???????????????? B7C0:???????????????? B7D0:???????????????? B7E0:???????????????? B7F0:??????????????? B8A0:??????????????? B8B0:???????????????? B8C0:???????????????? B8D0:???????????????? B8E0:???????????????? B8F0:??????????????? B9A0:??????????????? B9B0:???????????????? B9C0:???????????????? B9D0:???????????????? B9E0:???????????????? B9F0:??????????????? BAA0:??????????????? BAB0:???????????????? BAC0:???????????????? BAD0:???????????????? BAE0:???????????????? BAF0:??????????????? BBA0:??????????????? BBB0:???????????????? BBC0:???????????????? BBD0:???????????????? BBE0:???????????????? BBF0:??????????????? BCA0:??????????????? BCB0:???????????????? BCC0:???????????????? BCD0:???????????????? BCE0:???????????????? BCF0:??????????????? BDA0:??????????????? BDB0:???????????????? BDC0:???????????????? BDD0:???????????????? BDE0:???????????????? BDF0:??????????????? BEA0:??????????????? BEB0:???????????????? BEC0:???????????????? BED0:???????????????? BEE0:???????????????? BEF0:??????????????? BFA0:??????????????? BFB0:???????????????? BFC0:???????????????? BFD0:???????????????? BFE0:???????????????? BFF0:??????????????? C0A0:??????????????? C0B0:???????????????? C0C0:???????????????? C0D0:???????????????? C0E0:???????????????? C0F0:??????????????? C1A0:??????????????? C1B0:???????????????? C1C0:???????????????? C1D0:???????????????? C1E0:???????????????? C1F0:??????????????? C2A0:??????????????? C2B0:???????????????? C2C0:???????????????? C2D0:???????????????? C2E0:???????????????? C2F0:??????????????? C3A0:??????????????? C3B0:???????????????? C3C0:???????????????? C3D0:???????????????? C3E0:???????????????? C3F0:??????????????? C4A0:??????????????? C4B0:???????????????? C4C0:???????????????? C4D0:???????????????? C4E0:???????????????? C4F0:??????????????? C5A0:??????????????? C5B0:???????????????? C5C0:???????????????? C5D0:???????????????? C5E0:???????????????? C5F0:??????????????? C6A0:??????????????? C6B0:???????????????? C6C0:???????????????? C6D0:???????????????? C6E0:???????????????? C6F0:??????????????? C7A0:??????????????? C7B0:???????????????? C7C0:???????????????? C7D0:???????????????? C7E0:???????????????? C7F0:??????????????? C8A0:??????????????? C8B0:???????????????? C8C0:???????????????? C8D0:???????????????? C8E0:???????????????? C8F0:??????????????? C9A0:??I???????? C9B0:??????????????? C9C0:?????????????? C9D0:??????????????? C9E0:??????????????? C9F0:?????????????? CAA0:??????????????? CAB0:???????????????? CAC0:???????????????? CAD0:???????????????? CAE0:???????????????? CAF0:??????????????? CBA0:??????????????? CBB0:???????????????? CBC0:???????????????? CBD0:???????????????? CBE0:???????????????? CBF0:??????????????? CCA0:??????????????? CCB0:???????????????? CCC0:???????????????? CCD0:???????????????? CCE0:???????????????? CCF0:??????????????? CDA0:??????????????? CDB0:???????????????? CDC0:???????????????? CDD0:???????????????? CDE0:???????????????? CDF0:??????????????? CEA0:??????????????? CEB0:???????????????? CEC0:???????????????? CED0:???????????????? CEE0:???????????????? CEF0:??????????????? CFA0:??????????????? CFB0:???????????????? CFC0:???????????????? CFD0:???????????????? CFE0:???????????????? CFF0:??????????????? D0A0:??????????????? D0B0:???????????????? D0C0:???????????????? D0D0:???????????????? D0E0:???????????????? D0F0:??????????????? D1A0:??????????????? D1B0:???????????????? D1C0:???????????????? D1D0:???????????????? D1E0:???????????????? D1F0:??????????????? D2A0:??????????????? D2B0:???????????????? D2C0:???????????????? D2D0:???????????????? D2E0:???????????????? D2F0:??????????????? D3A0:??????????????? D3B0:???????????????? D3C0:???????????????? D3D0:???????????????? D3E0:???????????????? D3F0:??????????????? D4A0:??????????????? D4B0:???????????????? D4C0:???????????????? D4D0:???????????????? D4E0:???????????????? D4F0:??????????????? D5A0:??????????????? D5B0:???????????????? D5C0:???????????????? D5D0:???????????????? D5E0:???????????????? D5F0:??????????????? D6A0:??????????????? D6B0:???????????????? D6C0:???????????????? D6D0:???????????????? D6E0:???????????????? D6F0:??????????????? D7A0:??????????????? D7B0:???????????????? D7C0:???????????????? D7D0:???????????????? D7E0:???????????????? D7F0:??????????????? D8A0:??????????????? D8B0:???????????????? D8C0:???????????????? D8D0:???????????????? D8E0:???????????????? D8F0:??????????????? D9A0:??????????????? D9B0:???????????????? D9C0:???????????????? D9D0:???????????????? D9E0:???????????????? D9F0:??????????????? DAA0:??????????????? DAB0:???????????????? DAC0:???????????????? DAD0:???????????????? DAE0:???????????????? DAF0:??????????????? DBA0:??????????????? DBB0:???????????????? DBC0:???????????????? DBD0:???????????????? DBE0:???????????????? DBF0:??????????????? DCA0:??????????????? DCB0:???????????????? DCC0:???????????????? DCD0:???????????????? DCE0:???????????????? DCF0:??????????????? DDA0:??????????????? DDB0:???????????????? DDC0:???????????????? DDD0:???????????????? DDE0:???????????????? DDF0:??????????????? DEA0:??????????????? DEB0:???????????????? DEC0:???????????????? DED0:???????????????? DEE0:???????????????? DEF0:??????????????? DFA0:??????????????? DFB0:???????????????? DFC0:???????????????? DFD0:???????????????? DFE0:???????????????? DFF0:??????????????? E0A0:??????????????? E0B0:???????????????? E0C0:???????????????? E0D0:???????????????? E0E0:???????????????? E0F0:??????????????? E1A0:??????????????? E1B0:???????????????? E1C0:???????????????? E1D0:???????????????? E1E0:???????????????? E1F0:??????????????? E2A0:??????????????? E2B0:???????????????? E2C0:???????????????? E2D0:???????????????? E2E0:???????????????? E2F0:??????????????? E3A0:??????????????? E3B0:???????????????? E3C0:???????????????? E3D0:???????????????? E3E0:???????????????? E3F0:??????????????? E4A0:??????????????? E4B0:???????????????? E4C0:???????????????? E4D0:???????????????? E4E0:???????????????? E4F0:??????????????? E5A0:??????????????? E5B0:???????????????? E5C0:???????????????? E5D0:???????????????? E5E0:???????????????? E5F0:??????????????? E6A0:??????????????? E6B0:???????????????? E6C0:???????????????? E6D0:???????????????? E6E0:???????????????? E6F0:??????????????? E7A0:??????????????? E7B0:???????????????? E7C0:???????????????? E7D0:???????????????? E7E0:???????????????? E7F0:??????????????? E8A0:??????????????? E8B0:???????????????? E8C0:???????????????? E8D0:???????????????? E8E0:???????????????? E8F0:??????????????? E9A0:??????????????? E9B0:???????????????? E9C0:???????????????? E9D0:???????????????? E9E0:???????????????? E9F0:??????????????? EAA0:??????????????? EAB0:???????????????? EAC0:???????????????? EAD0:???????????????? EAE0:???????????????? EAF0:??????????????? EBA0:??????????????? EBB0:???????????????? EBC0:???????????????? EBD0:???????????????? EBE0:???????????????? EBF0:??????????????? ECA0:??????????????? ECB0:???????????????? ECC0:???????????????? ECD0:???????????????? ECE0:???????????????? ECF0:??????????????? EDA0:??????????????? EDB0:???????????????? EDC0:???????????????? EDD0:???????????????? EDE0:???????????????? EDF0:??????????????? EEA0:??????????????? EEB0:???????????????? EEC0:???????????????? EED0:???????????????? EEE0:???????????????? EEF0:??????????????? EFA0:??????????????? EFB0:???????????????? EFC0:???????????????? EFD0:???????????????? EFE0:???????????????? EFF0:??????????????? F0A0:??????????????? F0B0:???????????????? F0C0:???????????????? F0D0:???????????????? F0E0:???????????????? F0F0:??????????????? F1A0:??????????????? F1B0:???????????????? F1C0:???????????????? F1D0:???????????????? F1E0:???????????????? F1F0:??????????????? F2A0:??????????????? F2B0:???????????????? F2C0:???????????????? F2D0:???????????????? F2E0:???????????????? F2F0:??????????????? F3A0:??????????????? F3B0:???????????????? F3C0:???????????????? F3D0:???????????????? F3E0:???????????????? F3F0:??????????????? F4A0:??????????????? F4B0:???????????????? F4C0:???????????????? F4D0:???????????????? F4E0:???????????????? F4F0:??????????????? F5A0:??????????????? F5B0:???????????????? F5C0:???????????????? F5D0:???????????????? F5E0:???????????????? F5F0:??????????????? F6A0:??????????????? F6B0:???????????????? F6C0:???????????????? F6D0:???????????????? F6E0:???????????????? F6F0:??????????????? F7A0:??????????????? F7B0:???????????????? F7C0:???????????????? F7D0:???????????????? F7E0:???????????????? F7F0:??????????????? F8A0:??????????????? F8B0:???????????????? F8C0:???????????????? F8D0:???????????????? F8E0:???????????????? F8F0:??????????????? F9A0:??????????????? F9B0:???????????????? F9C0:???????????????? F9D0:???????????????? F9E0:???????????????? F9F0:??????????????? FAA0:??????????????? FAB0:???????????????? FAC0:???????????????? FAD0:???????????????? FAE0:???????????????? FAF0:??????????????? FBA0:??????????????? FBB0:???????????????? FBC0:???????????????? FBD0:???????????????? FBE0:???????????????? FBF0:??????????????? FCA0:??????????????? FCB0:???????????????? FCC0:???????????????? FCD0:???????????????? FCE0:???????????????? FCF0:??????????????? FDA0:??????????????? FDB0:???????????????? FDC0:???????????????? FDD0:???????????????? FDE0:???????????????? FDF0:??????????????? FEA0: FEB0: FEC0: FED0: FEE0: FEF0:
--
_______________________________________________ I18n mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/i18n
