Hello, simple patch which corrects dix/inpututils.c:generate_modkeymap() function is attached. Current generate_modkeymap function is broken and returns BadAlloc when keyboard has no modifiers (i.e. max_keys_per_mod is zero).
Although this bug has no effect for Xorg server ifself, it has effect for Xvnc VNC server. I've also checked parts of Xorg source which might be affected by this change (because returned modkeymap can be NULL with the patch) and this change doesn't cause any problem. Patch applies to server-1.9-branch but apply it also to server-1.8-branch and server-1.7-branch branches, please. Regards, Adam -- Adam Tkac, Red Hat, Inc.
>From 3fae47581a47613f1117b8794ae37b75ace73f3e Mon Sep 17 00:00:00 2001 From: Adam Tkac <[email protected]> Date: Tue, 24 Aug 2010 17:21:30 +0200 Subject: [PATCH] Return Success from generate_modkeymap() when max_keys_per_mod is zero. max_keys_per_mod equal to zero is a valid situation so generate_modkeymap should not return BadAlloc in this case. Signed-off-by: Adam Tkac <[email protected]> --- dix/inpututils.c | 24 +++++++++++++----------- 1 files changed, 13 insertions(+), 11 deletions(-) diff --git a/dix/inpututils.c b/dix/inpututils.c index 8ec80b5..6693c67 100644 --- a/dix/inpututils.c +++ b/dix/inpututils.c @@ -286,7 +286,7 @@ int generate_modkeymap(ClientPtr client, DeviceIntPtr dev, { CARD8 keys_per_mod[8]; int max_keys_per_mod; - KeyCode *modkeymap; + KeyCode *modkeymap = NULL; int i, j, ret; ret = XaceHook(XACE_DEVICE_ACCESS, client, dev, DixGetAttrAccess); @@ -310,18 +310,20 @@ int generate_modkeymap(ClientPtr client, DeviceIntPtr dev, } } - modkeymap = calloc(max_keys_per_mod * 8, sizeof(KeyCode)); - if (!modkeymap) - return BadAlloc; + if (max_keys_per_mod != 0) { + modkeymap = calloc(max_keys_per_mod * 8, sizeof(KeyCode)); + if (!modkeymap) + return BadAlloc; - for (i = 0; i < 8; i++) - keys_per_mod[i] = 0; + for (i = 0; i < 8; i++) + keys_per_mod[i] = 0; - for (i = 8; i < MAP_LENGTH; i++) { - for (j = 0; j < 8; j++) { - if (dev->key->xkbInfo->desc->map->modmap[i] & (1 << j)) { - modkeymap[(j * max_keys_per_mod) + keys_per_mod[j]] = i; - keys_per_mod[j]++; + for (i = 8; i < MAP_LENGTH; i++) { + for (j = 0; j < 8; j++) { + if (dev->key->xkbInfo->desc->map->modmap[i] & (1 << j)) { + modkeymap[(j * max_keys_per_mod) + keys_per_mod[j]] = i; + keys_per_mod[j]++; + } } } } -- 1.7.2.2
_______________________________________________ [email protected]: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: http://lists.x.org/mailman/listinfo/xorg-devel
