"Patterson, Ross" <[EMAIL PROTECTED]> writes:
>
> I know replying to one's own mail is gauche (and probably disgusting in some
> Wired JargonWatch list), but I've found what seems to be the major cause of
> trouble. IBM System/390 boxes are big-endian, and of course Intel x86'es are
> little-endian. There are at least three cases in the VNC server source where
> the attempt to be variously-endian fails.
Xvnc normally runs fine on big-endian architectures. I think the endian
problems you're seeing are in the XFree86 code, not in the VNC code.
Unfortunately I'm not sure how best to go about finding these. If characters
in your xterms come out funny, it sounds like they might be in the font
rendering code.
> In vnc_unixsrc/Xvnc/programs/Xserver/hw/vnc, rfb.h defines macros for
> mixed-endian byteswapping (Swap16IfLE() and Swap32IfLE()) but hard-codes the
> flag that determines the server's endianness to "1" (meaning little-endian).
> I think there should be some conditional definition, or at least it should be
> called out in the README etc.
Swap..IfLE actually does a run-time test to determine endianness - it sees
whether the first byte of an int initialised to one is zero or one.
> In the same directory, init.c defines the PF_EQ() macro assuming that
> byteswapping isn't necessary for 8-bit pixels, but in fact seems to be
> (although this doesn't make any sense to me). Running vncserver with "-depth
> 32" produces a non-garbled display (xterm contains readable characters,
> etc.). Either it's an endian-ness issue or there's some side effect of the
> translation functions in translate.c that this configuration requires and
> that rfbTranslateNone() (which is used with -depth 8) doesn't supply.
The PF_EQ macro is correct - there are no endian issues for 8-bit pixels.
> Lastly, in the same directory, translate.c defines the BGR233 pixel format
> with a constant 0 for rfbPixelFormat.bigEndian (meaning little-endian). If
> the evidence above is correct that there is actually some endian effect for
> 8-bit pixels, this needs to be conditional as well.
Similarly, the bigEndian flag for 8-bit pixels is irrelevant.
Cheers
Tristan
---------------------------------------------------------------------------
Tristan Richardson [EMAIL PROTECTED] www.uk.research.att.com/~tjr
---------------------------------------------------------------------------
AT&T Laboratories Cambridge, 24a Trumpington Street, Cambridge, CB2 1QA, UK
Tel: +44 1223 343000 Fax: +44 1223 313542 www.uk.research.att.com
---------------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, send a message with the line: unsubscribe vnc-list
to [EMAIL PROTECTED]
See also: http://www.uk.research.att.com/vnc/intouch.html
---------------------------------------------------------------------
