Public bug reported:
Greetings!
First of all, as per the instructions below, I am using Ubuntu 14.04
with libx11-6 version 2:1.6.2-1ubuntu2 installed.
I'm trying to figure out how to develop a Unicode-based Xlib
application, and, after several days of frustration with my code
strangely not picking up characters in some cases where every other
program seemed to handle them correctly, I realized that the problem was
due to Xutf8LookupString not reporting overflows correctly in some
situations when a small buffer is passed. The workaround is very simple
(using a bigger buffer), but I thought I'd report the problem anyway
because none of the documentation I've found has said anything about a
minimum buffer size and I think that it ought to get fixed (or at least
mentioned) for the benefit of other new Xlib programmers or people
porting programs over from other Xlib implementations.
I've attached a small C program that can be used to demonstrate the
problem. The particular key that I tested with was the ñ key on the
Spanish keyboard layout, but it seems like the problem manifests itself
for any non-ASCII Latin-1 character with a direct key association (and
possibly other cases). As written, the attached program functions
perfectly: pressing ñ results in "got 2 bytes: ñ" appearing on the
terminal. However, if one changes line 116 from "const int
INITIAL_BUFFER_SIZE = 4;" down to "const int INITIAL_BUFFER_SIZE = 1;"
(or even down to 0!), pressing ñ instead results in the message "got 0
bytes: " because, instead of reporting the actual character length and a
buffer overflow as it should, Xutf8LookupString reports that the
character sequence doesn't exist. The bug also occurs if one uses
XmbLookupString instead of Xutf8LookupString for roughly the same
reason, but I'm focusing on Xutf8LookupString here instead since its
logic is somewhat simpler and because it's the one that will be more
useful for what I'm trying to do in my program.
So, how does this bug come about? To answer that question, I downloaded
the source package, fired up my debugger, traced the execution path
through all of the indirect function calls, and narrowed the problem
down to these three functions:
modules/im/ximcp/imDefLkup.c lines 1120-1181:
int
_XimProtoUtf8LookupString(
XIC xic,
XKeyEvent *ev,
char*buffer,
int bytes,
KeySym *keysym,
Status *state)
{
Xic ic = (Xic)xic;
Xim im = (Xim)ic->core.im;
int ret;
Status tmp_state;
XimCommitInfoinfo;
if (!IS_SERVER_CONNECTED(im))
return 0;
if (!state)
state = &tmp_state;
if (ev->type == KeyPress && ev->keycode == 0) { /* Filter function */
if (!(info = ic->private.proto.commit_info)) {
*state = XLookupNone;
return 0;
}
ret = im->methods->ctstoutf8((XIM)im, info->string,
info->string_len, buffer, bytes, state);
if (*state == XBufferOverflow)
return ret;
if (keysym && (info->keysym && *(info->keysym))) {
*keysym = *(info->keysym);
if (*state == XLookupChars)
*state = XLookupBoth;
else
*state = XLookupKeySym;
}
_XimUnregCommitInfo(ic);
} else if (ev->type == KeyPress) {
ret = _XimLookupUTF8Text(ic, ev, buffer, bytes, keysym, NULL);
if (ret > 0) {
if (ret > bytes)
*state = XBufferOverflow;
else if (keysym && *keysym != NoSymbol)
*state = XLookupBoth;
else
*state = XLookupChars;
} else {
if (keysym && *keysym != NoSymbol)
*state = XLookupKeySym;
else
*state = XLookupNone;
}
} else {
*state = XLookupNone;
ret = 0;
}
return ret;
}
src/imConv.c lines 300-356:
int
_XimLookupUTF8Text(
Xic ic,
XKeyEvent* event,
char* buffer,
int nbytes,
KeySym* keysym,
XComposeStatus* status)
{
int count;
KeySym symbol;
Status dummy;
Xim im = (Xim)ic->core.im;
XimCommonPrivateRec* private = &im->private.common;
unsigned char look[BUF_SIZE];
ucs4_t ucs4;
/* force a latin-1 lookup for compatibility */
count = XLOOKUPSTRING(event, (char *)buffer, nbytes, &symbol, status);
if (keysym != NULL) *keysym = symbol;
if ((nbytes == 0) || (symbol == NoSymbol)) return count;
if (count > 1) {
memcpy(look, (char *)buffer,count);
look[count] = '\0';
if ((count = im->methods->ctstoutf8(ic->core.im,
(char*) look, count,
buffer, nbytes, &dummy)) < 0) {
count = 0;
}
