Hi everyone,

I decided to play a bit with Virtualbox using a 64 bit guest on an x86_64 Bit host.

Host is running (K)ubuntu Karmic 64 bit and on a whim I used Debian Lenny 64 bit (net installer) for the guest.

Virtualbox was latest public svn (OSE) with no modifications.

After installing the official ISO guest additions (3.1.2) and starting gdm/kdm I noticed the mouse wasn't working correctly and seemed stuck in the bottom right corner.

I couldn't find any hints on what was causing this in the forums and/or bug tracker so I decided to hunt it down by building the vboxmouse-driver and stepping through the callbacks. I found out that the incorrect results came from "xf86ScaleAxis" which is called by "VBoxReadInput". This function maps a value in an input range to the according one in a given output range. But here it was always clamped to the maximum output value even though the input was valid. I then analysed the xorg-server sources to see how "xf86ScaleAxis" was implemented.

It turns out that the prototype of xf86ScaleAxis is "extern _X_EXPORT int xf86ScaleAxis(int Cx, int Sxhigh, int Sxlow, int Rxhigh, int Rxlow)" while Virtualbox uses "xf86ScaleAxis(cx, 0, screenInfo.screens[0]->width, 0, 65536)". Carefully checking the input parameter names shows that they are obviously used in reverse regarding minimum and maximum value.

In theory this should not work but obviously it did so checking the git commitlog of the xorg server showed that the "xf86ScaleAxis"-function (implemented in xorg-server/hw/xfree86/common/xf86Xinput.c) was broken for several years because it got the final clamp check backwards. Finally this was fixed in xorg-server commit a3a7c12fcf8e4ac1418f9ea53f76091f309a721b on 2008-06-08, preceding 1.6.0. Now, Lenny uses server 1.4.2 which predates this change so it shouldn't be affected. But the Debian folks backported this patch among others to 2:1.4.2-9 so that's the reason why it fails to work now even on a 1.4 server.

Going by the commit log of Virtualbox the newer driver (vboxmouse_15.c) was only extended to work for 1.3 and 1.4 by the end of November 2009, so it didn't affect people before (and testing the 3.1.0 GAs confirms this, they were still working fine). Also, "VBoxReadInput" uses xf86ScaleAxis in a conditional way (it's #ifdef'ed and the comment reads: "Bug in the 1.4 X server series - conversion_proc was no longer called, but the server didn't yet do the conversion itself") so it obviously won't get used on later versions of the xorg server.

There is another place in the Virtualbox driver where "xf86ScaleAxis" is used ("VBoxConvert") but from the commit I take it that this callback will only get used on 1.3 because it obviously is not called on 1.4 (see comment in last paragraph) and is no longer necessary for >=1.5.

So things would still be fine if not for Debian backporting the 1.6 changes to 1.4 which leads to vboxmouse_15.c using the parameters in a wrong order for "xf86ScaleAxis" on systems that use this patched version.

Avoiding any further headaches in trying to detect this situation my solution was to simply replace "xf86ScaleAxis" by a new function "VBoxScale" in vboxmouse_15.c.

In case anyone is interested I have attached this as a patch. It's licensed under MIT, in case it's needed.

It also fixes another bug due to missing brackets (the #ifdef lines for 1.4 were inserted without noticing that the preceding if-condition only covers one line due to missing brackets, this will only show up in case of an error and even then only on 1.4 servers since for all other cases there is still only one line after the if)

Note that I didn't test this on any other server than the Lenny one yet (1.4.2-10).


Carsten

diff --git a/src/VBox/Additions/x11/vboxmouse/vboxmouse_15.c b/src/VBox/Additions/x11/vboxmouse/vboxmouse_15.c
index 3c9b0e9..f2c4340 100644
--- a/src/VBox/Additions/x11/vboxmouse/vboxmouse_15.c
+++ b/src/VBox/Additions/x11/vboxmouse/vboxmouse_15.c
@@ -59,6 +59,43 @@
 #include <errno.h>
 #include <fcntl.h>
 
+/* replacement for Xorg's xf86ScaleAxis which used to be broken and got the clamp on the
+   result backwards so all users of this function had to reverse minimum and maximum values.
+   It was only fixed before Xorg Server 1.6 but was also backported to 1.4 at least by
+   Debian so there is no safe way to know if the fixed version is in use or not. */
+static int
+VBoxScale(InputInfoPtr pInfo, int scaleval, int oMin, int oMax, int iMin, int iMax)
+{
+    int iDiff = iMax - iMin;
+    int oDiff = oMax - oMin;
+    
+    /* try to clamp on input already to avoid
+       incorrect results due to overflow */
+    if(scaleval > iMax)
+        return oMax;
+    
+    if(scaleval < iMin)
+        return oMin;
+
+    
+    if(iDiff) {
+        scaleval = (((scaleval - iMin) * oDiff) / iDiff) + oMin;
+    } else {
+        if (pInfo) {
+            xf86Msg(X_ERROR, "%s:division by zero scaling %d: %d %d on %d %d\n", pInfo->name, scaleval, iMin, iMax, oMin, oMax);
+        }
+        scaleval = 0;
+    }
+    
+    if(scaleval > oMax)
+        scaleval = oMax;
+    
+    if(scaleval < oMin)
+        scaleval = oMin;
+    
+    return scaleval;
+}
+
 static void
 VBoxReadInput(InputInfoPtr pInfo)
 {
@@ -72,15 +109,16 @@ VBoxReadInput(InputInfoPtr pInfo)
            miPointerGetScreen(pInfo->dev) != NULL
 #endif
         &&  RT_SUCCESS(VbglR3GetMouseStatus(&fFeatures, &cx, &cy))
-        && (fFeatures & VMMDEV_MOUSE_HOST_CAN_ABSOLUTE))
+        && (fFeatures & VMMDEV_MOUSE_HOST_CAN_ABSOLUTE)) {
 #if ABI_XINPUT_VERSION == SET_ABI_VERSION(2, 0)
-        /* Bug in the 1.4 X server series - conversion_proc was no longer
-         * called, but the server didn't yet do the conversion itself. */
-        cx = xf86ScaleAxis(cx, 0, screenInfo.screens[0]->width, 0, 65536);
-        cy = xf86ScaleAxis(cy, 0, screenInfo.screens[0]->height, 0, 65536);
+            /* Bug in the 1.4 X server series - conversion_proc was no longer
+             * called, but the server didn't yet do the conversion itself. */
+            cx = VBoxScale(pInfo, cx, 0, screenInfo.screens[0]->width, 0, 65536);
+            cy = VBoxScale(pInfo, cy, 0, screenInfo.screens[0]->height, 0, 65536);
 #endif
-        /* send absolute movement */
-        xf86PostMotionEvent(pInfo->dev, 1, 0, 2, cx, cy);
+            /* send absolute movement */
+            xf86PostMotionEvent(pInfo->dev, 1, 0, 2, cx, cy);
+        }
 }
 
 static void
@@ -229,8 +267,8 @@ VBoxConvert(InputInfoPtr pInfo, int first, int num, int v0, int v1, int v2,
             int v3, int v4, int v5, int *x, int *y)
 {
     if (first == 0) {
-        *x = xf86ScaleAxis(v0, 0, screenInfo.screens[0]->width, 0, 65536);
-        *y = xf86ScaleAxis(v1, 0, screenInfo.screens[0]->height, 0, 65536);
+        *x = VBoxScale(pInfo, v0, 0, screenInfo.screens[0]->width, 0, 65536);
+        *y = VBoxScale(pInfo, v1, 0, screenInfo.screens[0]->height, 0, 65536);
         return TRUE;
     } else
         return FALSE;
_______________________________________________
vbox-dev mailing list
[email protected]
http://vbox.innotek.de/mailman/listinfo/vbox-dev

Reply via email to