Endianity problems in XFree86-4 XAA on MipsEB

[Alexandr Andreev]

We have a MipsEB machine and a video card which has a 2D BitBLT engine.
It looks like we found a problem in XAA when we tried to use our hardware
8x8 Mono Pattern Fills. The problem appears when an application uses pixmaps.
Stipple and tile with the same pixmap are drawing in the different ways
(bytes in video memory are swapped). We looked through the XAA source tree
and found a dubious code in xaaPCache.c.

In two words... XAA tries to check that a pixmap (stipple/tile) can be reduced
to a 8x8 mono pattern, and if so, puts this stipple/tile to two dwords (patterns0,1)
and passes it to hw driver. And it looks like the stipple code works fine, but there
is an endianity problem in the "tile" case... or maybe vise versa, but in any case
patterns are maked up in different ways for BE.

Bool
XAACheckStippleReducibility(PixmapPtr pPixmap)
{
...
pPriv->pattern0 = bits[0] | SHIFT_L(bits[1],8) | SHIFT_L(bits[2],16) | SHIFT_L(bits[3],24);
pPriv->pattern1 = bits[4] | SHIFT_L(bits[5],8) | SHIFT_L(bits[6],16) | SHIFT_L(bits[7],24);
...
}
where SHIFT_L(value, shift) is defined as ((value) >> (shift)) for Big Endian.


Bool
XAACheckTileReducibility(PixmapPtr pPixmap, Bool checkMono)
{
...
pPriv->pattern0 = bits[0] | (bits[1]<<8) | (bits[2]<<16) | (bits[3]<<24);
pPriv->pattern1 = bits[4] | (bits[5]<<8) | (bits[6]<<16) | (bits[7]<<24);
...
}

In both cases the unsigned int bits[] array contains bytes! with the bitmask to be
passed to a driver via pPriv->pattern0, pPriv->pattern1.

When we tried to use the fbdev driver which is not using XAA, the problem
is gone.

Did anybody see something similar on Big Endian machines?

_______________________________________________
Devel mailing list
[EMAIL PROTECTED]
http://XFree86.Org/mailman/listinfo/devel