Re: Moving xkbcomp into the server
On Tue, Nov 18, 2008 at 5:19 PM, Daniel Stone [EMAIL PROTECTED] wrote:
Hi,
On Mon, Nov 17, 2008 at 11:25:25AM -0800, Dan Nicholson wrote:
I decided to take a crack at moving xkbcomp into the server so it's
not popen'd whenever a keymap is loaded. For the first crack, I'm
trying to just leave xkbcomp pretty much unchanged except for the
interface. What's causing me the most difficulty is converting to
server API. One snag I've hit is XStringToKeysym. Here's an example
usage in the xkbcomp parser:
int
LookupKeysym(char *str, KeySym * sym_rtrn)
{
[...]
}
Is there an equivalent API in the server to do this conversion?
Is this crazy/am I going about this the wrong way?
Any general suggestions for working on this?
No, yes, and break it out into a convenience library, respectively. I
toyed with creating a libxkbcommon ages ago for exactly this reason, but
never ended up finishing it.
A server convenience library (in the libtool sense) or a low level
shared library for the server and client? If the second, that means
use of Xlib is prohibited, right?
OK, I suppose that's the right way to do it. Do you have any code I
could look at? I don't care if it's ancient/broken/lacking history,
I'd just like to see what you had in mind since you obviously have
infinitely more experience here than I do.
--
Dan
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Mon, Nov 17, 2008 at 8:54 PM, Peter Hutterer [EMAIL PROTECTED] wrote: On Mon, Nov 17, 2008 at 11:25:25AM -0800, Dan Nicholson wrote: I decided to take a crack at moving xkbcomp into the server so it's not popen'd whenever a keymap is loaded. For the first crack, I'm trying to just leave xkbcomp pretty much unchanged except for the interface. What's causing me the most difficulty is converting to server API. One snag I've hit is XStringToKeysym. Here's an example usage in the xkbcomp parser: As much as I'd like to see it in the server - is the popen the painful bit? AFAIU, the current approach goes from RMLVO to Kkcstg to xkb to xkm, every time we call InitKeyboardDeviceStruct. I agree completely. As soon as I looked at the path taken in XkbDDXNamesFromRules, I realized how insane it was that there were all these conversions. I'm just moving one step at a time here, with the first one being: leave the retarded conversion path in place, but move the converter into the server. The next step would be to actually start removing parts of the conversion process, but that will take a little more effort. Ideally, we'd like to cache and re-use as much as possible. Usually, all keyboards come up with the same map anyway and compiling it again is redundant. Just doing that might already save a significant chunk of time. This should also be much easier to achieve, and if it provides a relevant speedup it would be great as interim solution. I'll try to look at that. So the path is XkbInitKeyboardDeviceStruct:xkb/xkbInit.c - XkbDDXNamesFromRules:xkb/ddxLoad.c this is where all the rules parsing happens, skipping that may save time. - XkbDDXLoadKeymapByNames:xkb/ddxLoad.c this is where xkbcomp is called with the Kcstg format. xkbcomp now parses that into an xkm format - XkmReadFile:xkb/xkmread.c here we read in the compiled keymap and basically copy it into the internal structs. Right. So, ideally what would happen is: 1. Skip parsing completely if the rules haven't changed. 2. Go directly from RMLVO-internal structs. Or at least make the intra-conversion states not involve reading/writing/parsing files. Seem reasonable? -- Dan ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Mon, Nov 17, 2008 at 9:17 PM, Alan Coopersmith [EMAIL PROTECTED] wrote: Dan Nicholson wrote: One snag I've hit is XStringToKeysym. Is there an equivalent API in the server to do this conversion? I haven't checked if there's one added now, but I know our Xsun pre-xkb keytable parser linked in a copy of the ks_tables.h file built in the libX11 build and included a static copy of the XStringToKeysym function to do the lookups in it. Cool, that's what I thought of a little later. However, bringing XStringToKeysym right in would also mean the Xrm hash table used for the keysymdb. That seemed a little to nasty. Then I thought it might work to construct a static table from XKeysymDB and then dump it into some other lookup structure already in the server. Can the hash table API in dix/resources.c be used, or does that tie into the resources system more deeply. I wonder if going forward, moving XStringToKeysym into a separate library, or putting equivalent functionality in something like libxcb-keysyms wouldn't be a better way, to reduce duplication of this data/parsing code needed by users of both libX11 libxcb, and the Xserver itself. Not a bad idea. Nearly everything I've done so far has introduced duplication from the client side, but I figured I'd try to get it working before moving orthogonally. It's certainly easier to implement everything internally to the server rather than introducing a bunch of external API that might be wrong and can't be removed later. -- Dan ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Tue, Nov 18, 2008 at 03:44:32AM -0800, Dan Nicholson wrote: I agree completely. As soon as I looked at the path taken in XkbDDXNamesFromRules, I realized how insane it was that there were all these conversions. I'm just moving one step at a time here, with the first one being: leave the retarded conversion path in place, but move the converter into the server. The next step would be to actually start removing parts of the conversion process, but that will take a little more effort. I think it'd be less effort to leave the converter as-is and remove the need for calling it, but that's a guess only too. So the path is XkbInitKeyboardDeviceStruct:xkb/xkbInit.c - XkbDDXNamesFromRules:xkb/ddxLoad.c this is where all the rules parsing happens, skipping that may save time. - XkbDDXLoadKeymapByNames:xkb/ddxLoad.c this is where xkbcomp is called with the Kcstg format. xkbcomp now parses that into an xkm format - XkmReadFile:xkb/xkmread.c here we read in the compiled keymap and basically copy it into the internal structs. Right. So, ideally what would happen is: 1. Skip parsing completely if the rules haven't changed. 2. Go directly from RMLVO-internal structs. Or at least make the intra-conversion states not involve reading/writing/parsing files. right, you'd go from oh, same RMLVO to xkmRead() directly. In theory, you could just memcpy the structs from another device but that's not a task for the faint-hearted. Cheers, Peter ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
Peter Hutterer wrote: On Tue, Nov 18, 2008 at 03:44:32AM -0800, Dan Nicholson wrote: I agree completely. As soon as I looked at the path taken in XkbDDXNamesFromRules, I realized how insane it was that there were all these conversions. I'm just moving one step at a time here, with the first one being: leave the retarded conversion path in place, but move the converter into the server. The next step would be to actually start removing parts of the conversion process, but that will take a little more effort. A bit offtopic, but I think xkb really lacks a tool like xkeycaps http://www.jwz.org/xkeycaps/ Xkb configuration is not something trivial, and a program like that would be very useful. But I think the right path should be not move xkbcomp to the X Server, at least not without a major xkb redesign, instead, have an external tool (xkbcomp) generate pre compiled keymaps based on a brief description. I think it'd be less effort to leave the converter as-is and remove the need for calling it, but that's a guess only too. So the path is XkbInitKeyboardDeviceStruct:xkb/xkbInit.c - XkbDDXNamesFromRules:xkb/ddxLoad.c this is where all the rules parsing happens, skipping that may save time. - XkbDDXLoadKeymapByNames:xkb/ddxLoad.c this is where xkbcomp is called with the Kcstg format. xkbcomp now parses that into an xkm format - XkmReadFile:xkb/xkmread.c here we read in the compiled keymap and basically copy it into the internal structs. Right. So, ideally what would happen is: 1. Skip parsing completely if the rules haven't changed. 2. Go directly from RMLVO-internal structs. Or at least make the intra-conversion states not involve reading/writing/parsing files. right, you'd go from oh, same RMLVO to xkmRead() directly. In theory, you could just memcpy the structs from another device but that's not a task for the faint-hearted. Cheers, Peter Paulo ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
Le mardi 18 novembre 2008 à 13:36 -0200, Paulo Cesar Pereira de Andrade A bit offtopic, but I think xkb really lacks a tool like xkeycaps http://www.jwz.org/xkeycaps/ Xkb configuration is not something trivial, and a program like that would be very useful. Like this? http://simos.info/blog/archives/747 Interested people can help Simos polish his summer of code project. -- Nicolas Mailhot signature.asc Description: Ceci est une partie de message numériquement signée ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Tue, Nov 18, 2008 at 4:28 AM, Peter Hutterer [EMAIL PROTECTED] wrote: On Tue, Nov 18, 2008 at 03:44:32AM -0800, Dan Nicholson wrote: I agree completely. As soon as I looked at the path taken in XkbDDXNamesFromRules, I realized how insane it was that there were all these conversions. I'm just moving one step at a time here, with the first one being: leave the retarded conversion path in place, but move the converter into the server. The next step would be to actually start removing parts of the conversion process, but that will take a little more effort. I think it'd be less effort to leave the converter as-is and remove the need for calling it, but that's a guess only too. So, I took a look at this, and it was fairly easy to write an (untested) patch that checks if RMLVO has been changed and bailing out early from XkbInitKeyboardDeviceStruct if it hasn't. However, I think this still leaves you with two xkbcomps during the startup in the common configuration. XkbInitKeyboardDeviceStruct will be called first via InitCoreDevices with the default ruleset (base, pc105, us, NULL, NULL). Then the driver will call XkbInitKeyboardDeviceStruct again with a ruleset that is probably different (most likely rules == evdev). So, you're going to get two keymap conversions in most cases, anyway, but at least totally gratuitous conversions can be removed. I'll send that patch after I test it a bit. So the path is XkbInitKeyboardDeviceStruct:xkb/xkbInit.c - XkbDDXNamesFromRules:xkb/ddxLoad.c this is where all the rules parsing happens, skipping that may save time. - XkbDDXLoadKeymapByNames:xkb/ddxLoad.c this is where xkbcomp is called with the Kcstg format. xkbcomp now parses that into an xkm format - XkmReadFile:xkb/xkmread.c here we read in the compiled keymap and basically copy it into the internal structs. Right. So, ideally what would happen is: 1. Skip parsing completely if the rules haven't changed. 2. Go directly from RMLVO-internal structs. Or at least make the intra-conversion states not involve reading/writing/parsing files. right, you'd go from oh, same RMLVO to xkmRead() directly. In theory, you could just memcpy the structs from another device but that's not a task for the faint-hearted. But, why not oh, same RMLVO, do nothing? Oh, that's because you have to do it for each device. I guess then you probably want to keep the xkm file in that case, and only unlink during CloseDownDevices or something. Unfortunately, the xkm file is immediately deleted right now. This really only helps if you have multiple keyboards or you're hotplugging them, though. -- Dan ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Tue, Nov 18, 2008 at 2:21 PM, Peter Hutterer
[EMAIL PROTECTED] wrote:
On Tue, Nov 18, 2008 at 11:08:36AM -0800, Dan Nicholson wrote:
I think it'd be less effort to leave the converter as-is and remove the
need
for calling it, but that's a guess only too.
So, I took a look at this, and it was fairly easy to write an
(untested) patch that checks if RMLVO has been changed and bailing out
early from XkbInitKeyboardDeviceStruct if it hasn't. However, I think
this still leaves you with two xkbcomps during the startup in the
common configuration. XkbInitKeyboardDeviceStruct will be called first
via InitCoreDevices with the default ruleset (base, pc105, us,
NULL, NULL). Then the driver will call XkbInitKeyboardDeviceStruct
again with a ruleset that is probably different (most likely rules ==
evdev). So, you're going to get two keymap conversions in most
cases, anyway, but at least totally gratuitous conversions can be
removed. I'll send that patch after I test it a bit.
As the theory goes, at least under linux we should be able to change the
default ruleset of evdev, etc. anyway, which skips another xkbcomp
invocation.
Right now the defaults are hardcoded via XkbSetRulesDflts into dix/devices.c:
#ifdef XKB
if (!noXkbExtension) {
bzero(names, sizeof(names));
XkbSetRulesDflts(base, pc105, us, NULL, NULL);
XkbInitKeyboardDeviceStruct(pDev, names, keySyms, modMap,
CoreKeyboardBell, CoreKeyboardCtl);
}
else
#endif
The XkbSetRulesDflts isn't strictly necessary, though, since the xkb
code has it's own built-in defaults that it will fall back to if no
new defaults are set:
xkb/xkbInit.c:
#ifndef XKB_DFLT_RULES_FILE
#define XKB_DFLT_RULES_FILE rules
#endif
#ifndef XKB_DFLT_KB_LAYOUT
#define XKB_DFLT_KB_LAYOUT us
#endif
#ifndef XKB_DFLT_KB_MODEL
#define XKB_DFLT_KB_MODEL dflt
#endif
#ifndef XKB_DFLT_KB_VARIANT
#define XKB_DFLT_KB_VARIANT NULL
#endif
#ifndef XKB_DFLT_KB_OPTIONS
#define XKB_DFLT_KB_OPTIONS NULL
#endif
So, it would be fairly easy to use these macros and just set a linux
build time default:
if test $host_os = linux; then
dflt_xkb_rules=evdev
else
dflt_xkb_rules=base
fi
AC_DEFINE_UNQUOTED([XKB_DFLT_RULES_FILE], [$dflt_xkb_rules], [Default
XKB rules file])
And change the XKB_DFLT_KB_MODEL default to pc105. I don't know if
that screws with people trying to use the kbd driver on linux, though
(don't know if we care, either).
But, why not oh, same RMLVO, do nothing? Oh, that's because you have
to do it for each device. I guess then you probably want to keep the
xkm file in that case, and only unlink during CloseDownDevices or
something. Unfortunately, the xkm file is immediately deleted right
now. This really only helps if you have multiple keyboards or you're
hotplugging them, though.
well, since the xkms hardly ever change across server instances finding a way
to buffer them across multiple restarts would fix that problem too. Then you
really only have xkmRead() for each keyboard, no xkbcomp anymore.
Oh, btw:
[EMAIL PROTECTED]:~$ grep Configuring as keyboard /var/log/Xorg.0.log
(II) Power Button (FF): Configuring as keyboard
(II) AT Translated Set 2 keyboard: Configuring as keyboard
(II) Sleep Button (CM): Configuring as keyboard
(II) ThinkPad Extra Buttons: Configuring as keyboard
(II) Video Bus: Configuring as keyboard
(II) Video Bus: Configuring as keyboard
If I'm not completely wrong, xkbcomp will be called for all of them, so
chances are high that you have a multiple keyboard setup.
Wow, hadn't noticed that, and checking through the evdev source shows
that the keymap will be reconfigured each time. OK, I'll just work on
this first and see if I can get it down to a single xkbcomp for
default configurations.
--
Dan
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
Hi,
On Mon, Nov 17, 2008 at 11:25:25AM -0800, Dan Nicholson wrote:
I decided to take a crack at moving xkbcomp into the server so it's
not popen'd whenever a keymap is loaded. For the first crack, I'm
trying to just leave xkbcomp pretty much unchanged except for the
interface. What's causing me the most difficulty is converting to
server API. One snag I've hit is XStringToKeysym. Here's an example
usage in the xkbcomp parser:
int
LookupKeysym(char *str, KeySym * sym_rtrn)
{
[...]
}
Is there an equivalent API in the server to do this conversion?
Is this crazy/am I going about this the wrong way?
Any general suggestions for working on this?
No, yes, and break it out into a convenience library, respectively. I
toyed with creating a libxkbcommon ages ago for exactly this reason, but
never ended up finishing it.
Cheers,
Daniel
signature.asc
Description: Digital signature
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
Hi, On Tue, Nov 18, 2008 at 11:08:36AM -0800, Dan Nicholson wrote: On Tue, Nov 18, 2008 at 4:28 AM, Peter Hutterer [EMAIL PROTECTED] wrote: Right. So, ideally what would happen is: 1. Skip parsing completely if the rules haven't changed. 2. Go directly from RMLVO-internal structs. Or at least make the intra-conversion states not involve reading/writing/parsing files. right, you'd go from oh, same RMLVO to xkmRead() directly. In theory, you could just memcpy the structs from another device but that's not a task for the faint-hearted. But, why not oh, same RMLVO, do nothing? Oh, that's because you have to do it for each device. I guess then you probably want to keep the xkm file in that case, and only unlink during CloseDownDevices or something. Unfortunately, the xkm file is immediately deleted right now. This really only helps if you have multiple keyboards or you're hotplugging them, though. I'd rather ditch the XKM completely, and go from RMLVO to KcCGST (this part of the conversion isn't hideously painful, though I'm sure the rules parsing could be made faster and more efficient) to XkbDescRec. This basically just means shoving xkbcomp into the server and deleting all the code that deals with XKM: just delivering the XkbDescRec it builds anyway. I haven't profiled this, so I'm likely to be wrong, but my money's on having to create a new file, wait while our fork()ed process writes it out, waiting for it to bail, then reading back and deserialising the very same file into the exact same format, being a reasonable part of the performance problem. That and the actual lexer/etc is crap. Cheers, Daniel signature.asc Description: Digital signature ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Moving xkbcomp into the server
I decided to take a crack at moving xkbcomp into the server so it's
not popen'd whenever a keymap is loaded. For the first crack, I'm
trying to just leave xkbcomp pretty much unchanged except for the
interface. What's causing me the most difficulty is converting to
server API. One snag I've hit is XStringToKeysym. Here's an example
usage in the xkbcomp parser:
int
LookupKeysym(char *str, KeySym * sym_rtrn)
{
KeySym sym;
if ((!str) || (uStrCaseCmp(str, any) == 0)
|| (uStrCaseCmp(str, nosymbol) == 0))
{
*sym_rtrn = NoSymbol;
return 1;
}
else if ((uStrCaseCmp(str, none) == 0)
|| (uStrCaseCmp(str, voidsymbol) == 0))
{
*sym_rtrn = XK_VoidSymbol;
return 1;
}
sym = XStringToKeysym(str);
if (sym != NoSymbol)
{
*sym_rtrn = sym;
return 1;
}
return 0;
}
Is there an equivalent API in the server to do this conversion?
Is this crazy/am I going about this the wrong way?
Any general suggestions for working on this?
--
Dan
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Mon, Nov 17, 2008 at 11:25:25AM -0800, Dan Nicholson wrote: I decided to take a crack at moving xkbcomp into the server so it's not popen'd whenever a keymap is loaded. For the first crack, I'm trying to just leave xkbcomp pretty much unchanged except for the interface. What's causing me the most difficulty is converting to server API. One snag I've hit is XStringToKeysym. Here's an example usage in the xkbcomp parser: As much as I'd like to see it in the server - is the popen the painful bit? AFAIU, the current approach goes from RMLVO to Kkcstg to xkb to xkm, every time we call InitKeyboardDeviceStruct. Ideally, we'd like to cache and re-use as much as possible. Usually, all keyboards come up with the same map anyway and compiling it again is redundant. Just doing that might already save a significant chunk of time. This should also be much easier to achieve, and if it provides a relevant speedup it would be great as interim solution. So the path is XkbInitKeyboardDeviceStruct:xkb/xkbInit.c - XkbDDXNamesFromRules:xkb/ddxLoad.c this is where all the rules parsing happens, skipping that may save time. - XkbDDXLoadKeymapByNames:xkb/ddxLoad.c this is where xkbcomp is called with the Kcstg format. xkbcomp now parses that into an xkm format - XkmReadFile:xkb/xkmread.c here we read in the compiled keymap and basically copy it into the internal structs. Cheers, Peter ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
Dan Nicholson wrote: One snag I've hit is XStringToKeysym. Is there an equivalent API in the server to do this conversion? I haven't checked if there's one added now, but I know our Xsun pre-xkb keytable parser linked in a copy of the ks_tables.h file built in the libX11 build and included a static copy of the XStringToKeysym function to do the lookups in it. I wonder if going forward, moving XStringToKeysym into a separate library, or putting equivalent functionality in something like libxcb-keysyms wouldn't be a better way, to reduce duplication of this data/parsing code needed by users of both libX11 libxcb, and the Xserver itself. -- -Alan Coopersmith- [EMAIL PROTECTED] Sun Microsystems, Inc. - X Window System Engineering ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: Moving xkbcomp into the server
On Mon, Nov 17, 2008 at 11:25:25AM -0800, Dan Nicholson wrote: I decided to take a crack at moving xkbcomp into the server so it's not popen'd whenever a keymap is loaded. For the first crack, I'm trying to just leave xkbcomp pretty much unchanged except for the interface. What's causing me the most difficulty is converting to server API. One snag I've hit is XStringToKeysym. Here's an example usage in the xkbcomp parser: As much as I'd like to see it in the server - is the popen the painful bit? We are currently using the patch I posted sometime ago, to cache xkbcomp output on some oems. It should make significant difference on low profile computers. But I did not extend it to not not do things like parse the geometry, to avoid breaking the specs. But I think it is not quite reliably anymore... At least libxkbui was not working with xorgcfg for some time, and xorgcfg was probably the only client using it. But really, some major cleanup should be done. The geometry code I believe is more then half of the computing time, and not used. Also, the caching should be done on a more higher layer, instead of sha1'ing all the xkbmap, it should sha1 just the main descriptions, like the setxkbmap -print output. AFAIU, the current approach goes from RMLVO to Kkcstg to xkb to xkm, every time we call InitKeyboardDeviceStruct. Ideally, we'd like to cache and re-use as much as possible. Usually, all keyboards come up with the same map anyway and compiling it again is redundant. Just doing that might already save a significant chunk of time. This should also be much easier to achieve, and if it provides a relevant speedup it would be great as interim solution. So the path is XkbInitKeyboardDeviceStruct:xkb/xkbInit.c - XkbDDXNamesFromRules:xkb/ddxLoad.c this is where all the rules parsing happens, skipping that may save time. - XkbDDXLoadKeymapByNames:xkb/ddxLoad.c this is where xkbcomp is called with the Kcstg format. xkbcomp now parses that into an xkm format - XkmReadFile:xkb/xkmread.c here we read in the compiled keymap and basically copy it into the internal structs. Cheers, Peter Paulo ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
