On 8/13/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote:
[removed the Vim maillist, this is development only]
Edward L. Fox wrote:
> On 8/12/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote:
> > [...]
> > You may have uncovered a bug that went unnoticed so far. Please try to
> > discover what causes this problem. I can't guess why the last character
> > is messed up, looks strange.
>
> I think I solved the problem! That was caused by iconv.
>
> size_t iconv(iconv_t cd,
> char **inbuf, size_t *inbytesleft,
> char **outbuf, size_t *outbytesleft);
>
> The parameter "inbytesleft" and "outbytesleft" should all include the
> trailing '\0' byte. In the previous version of gvim, we passed the
> parameter as the length of the string, excluding the trailing '\0'. So
> it is 1 byte less than the correct value.
This is not quite so. iconv() does not require the terminating NUL (it
can also be used to convert part of a string). If it does require the
NUL then iconv() is broken. That's unlikely though.
I wrote a short piece of testing code to test iconv with Chinese
characters. The fact is, if the last character is a Chinese character,
it is always malformed after converting. So I think it should be
necessary to pass the length including the trailing '\0' to iconv.
--------------------8<--------------------
#include <iconv.h>
int main(void)
{
char inbuffer[256];
char outbuffer[256];
int fd;
fd = iconv_open("cp936", "euc-cn");
for (;;)
{
int inlength, outlength;
char *inptr, *outptr;
gets(inbuffer);
inlength = strlen(inbuffer);
outlength = 256;
if (inlength == 0)
break;
inptr = inbuffer;
outptr = outbuffer;
iconv(fd, &inptr, &inlength, &outptr, &outlength);
printf("%s\n", outbuffer);
}
iconv_close(fd);
return 0;
}
--------------------8<--------------------
Your change suggests that the length that is passed should be one more.
Thus only one byte of the last double-byte character is currently
converted. I can't quickly figure out where the wrong length is
computed or passed. You probably already know the call stack, please
have a look at where the length comes from. It's probably an off-by-one
error somewhere.
I traced the code again and again but nothing special happened. You
called string_convert and pass 0 as the length of the string, so in
string_convert_ext you calculates the length of the string with
STRLEN, then call iconv_string, last iconv. There is nothing wrong
with the length anywhere. So... Maybe it is still iconv's fault.
[...]
Regards,
Edward L. Fox
