It appears that on some hardware/diver combo such as nv30/nouveau, using GL_ALPHA as format for 8-bit depth will cause an incomplete attachment error (GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT) when trying to bind the texture.
As a result, the FBO is NULL and glamor segfaults when trying to access the FBO width/height in pixmap_priv_get_scale(). This happens with glamor-xv which uses a 8bit pixmap, meaning that on such hardware/driver, trying to play a video using Xv will lead to a crash of the Xserver. This affects Xwayland, Xephyr and modesetting driver with glamor accel: https://bugs.freedesktop.org/show_bug.cgi?id=100710 https://bugzilla.redhat.com/1412814 So this series of RFC patches consist of 3 things: 1. Check if using a 8bit pixmap causes a incomplete attachment error as soon as glamor_xv_core_init() and return a boolean value indicating the caller whether or not glamor-xv is usable. 2. Modify each callers, namely ephyr, modesetting and xwayland to take advantage of this boolean value and do not enable xv if not usable. One difficulty here is that to check FBO usability, one has to have the screen resources created, which means that the changes also include moving glamor_xv_core_init() to CreateScreenResources() instead of screen init. 3. Last patch is problably even more controversial, as I found that using GL_RGBA for those 8bit format actually works for this hardware, even though this is a clear waste of resources. Thins is, though, it makes glamor-xv perfectly usable, not only it doesn't crash but is's also reasonably fast even on this old hardware. Other thing to consider here is that modern hardware won't use than anyway and prefer GL_RED and texture swizzle, so for those, this patch makes no difference. Now, I am not sure who that flies with other "older" hardware, but given point 1. and 2. even if it doesn't work, it won't crash (at least)... Cheers, Olivier Olivier Fourdan (5): glamor-xv: Check that FBO is usable modesetting: Check that glamor-xv is usable Xephyr: Check that glamor-xv is usable Xwayland: Check that glamor-xv is usable glamor: Use GL_RGBA format for 8-bit depth glamor/glamor.c | 2 ++ glamor/glamor_priv.h | 3 ++- glamor/glamor_utils.h | 11 ++++++----- glamor/glamor_xv.c | 23 ++++++++++++++++++++++- hw/kdrive/ephyr/ephyr.c | 16 ++++++++++++++-- hw/kdrive/ephyr/ephyr.h | 5 +++-- hw/kdrive/ephyr/ephyr_glamor_xv.c | 7 ++++--- hw/xfree86/drivers/modesetting/driver.c | 26 +++++++++++++------------- hw/xfree86/glamor_egl/glamor_xf86_xv.c | 3 ++- hw/xwayland/xwayland-glamor-xv.c | 3 ++- hw/xwayland/xwayland-glamor.c | 11 ++++++----- 11 files changed, 76 insertions(+), 34 deletions(-) -- 2.9.3 _______________________________________________ [email protected]: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
