Zhaojun Wu wrote:
> Hi, Yongwei,
>
> I am also realizing this issue during my recent research on Chinese encodings.
>
> As shown in the wikipedia, CP936 is just the encoding derived from
> Microsoft's Windows 95. Due to the popularity of Windows OS in
> mainland China, the Chinese government build the GBK (GB-2312
> extension) on the top of CP936 to cover more characters which are out
> of the old GB2312 standard.
> GB18030 actually is a Unicode-compatible standard and a superset of
> GB2312 and back-compatible with GBK.
>
> Thus, in this case, GB18030 could not be treated as CP936.
>
> If you can read Chinese, here is a brief and good introduction of
> various Chinese encoding systems.
> [http://www.eygle.com/digest/2007/01/zhs16gbk_char.html]
>
> For the encoding detection, I find another powerful tool "enca"
> (Extremely Naive Charset Analyser)
> [http://trific.ath.cx/software/enca/] to do the same job.
>
> It seems working better than your "tellenc". For example, your
> "tellenc" cannot identify the encoding of one Chinese utf-8 test file
> distributed in the "enca" package.
>
> So far, I haven't dig into the details of both of your
> implementations, but just enclosed that test file for you reference.
If I understand it correctly is GB18030 a multi-byte character set that
is mostly the same as cp936, but adds a number of 4-byte characters.
Vim does not support those 4-byte characters, thus setting 'encoding' to
gb18030 won't work.
But conversion between gb18030 and utf-8 should work, thus when
'encoding' is utf-8 it should be possible to use gb18030 in
'fileencodings' and 'fileencoding'. Perhaps you can check if that
works.
--
hundred-and-one symptoms of being an internet addict:
199. You read this entire list of symptoms, looking for something
that doesn't describe you.
/// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ download, build and distribute -- http://www.A-A-P.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
