Yeah, those values should be causing cinfo.out_color_space to be set to
JCS_EXT_XRGB, which is what we intended. However, I think I now
understand why there is a mismatch (bear with the long-winded
explanation, as I am doing this partly so I can get it straight in my
own head):
When you decompress a JPEG image in libjpeg-turbo, you can specify how
the red, green, and blue pixels are ordered in the destination RGB
image, as well as the size of each RGB pixel. This is accomplished by
setting the output color space to RGB, BGR, RGBX, BGRX, XRGB, or XBGR.
This controls where the red, green, and blue components end up after the
YUV-to-RGB conversion. Let's take two neighboring RGB pixels {R0, G0,
B0} and {R1, G1, B1}. The various libjpeg-turbo colorspace options
would store the bytes as follows (in order of increasing memory addresses):
JCS_EXT_RGB: R0, G0, B0, R1, G1, B1
JCS_EXT_BGR: B0, G0, R0, B1, G1, R1
JCS_EXT_RGBX: R0, G0, B0, XX, R1, G1, B1, XX
JCS_EXT_BGRX: B0, G0, R0, XX, B1, G1, R1, XX
JCS_EXT_XRGB: XX, R0, G0, B0, XX, R1, G1, B1
JCS_EXT_XBGR: XX, B0, G0, R0, XX, B1, G1, R1
The opinion on whether or not this is correct for a big endian machine
seems to vary. One convention takes "RGBX" to mean: "R is at a lower
address than G, which is at a lower address than B, which is at a lower
address than X." That is the convention that libjpeg-turbo follows, and
there are other APIs that follow the same convention. Others say that
"RGBX" means: "R" is the least significant byte, and "X" is the most
significant. Given the latter convention, a 32-bit RGBX pixel {R0, G0,
B0} should be written (in order of increasing memory location) as R0,
G0, B0, XX on little endian machines and XX, B0, G0, R0 on big endian
machines. The latter convention seems to be what I was trying to follow
in tightDecode.h, so that's why there is a mismatch. TurboVNC seems to
also use the latter convention, so it would likely fail in the same way
when built against libjpeg-turbo on a big endian system.
At this point, I've managed to confuse myself as to what the "right"
approach is here. I think it's probably to modify libjpeg-turbo so that
its convention is for "A" to be the LSB and "D" to be the MSB in an
"ABCD" pixel, but I really need to log some time on a big endian machine
(any volunteers?) and play with it to convince myself of that.
If anyone else has any sage advice as to what the right convention is,
please post. I can't seem to find any official word about this on the web.
DRC
On 2/2/11 6:21 AM, cwjor...@cox.net wrote:
> DRC,
>
> Thanks for your quick reply. The Solaris machine is Intel.
>
> As you suggested I edited common/rfb/tightDecode.h to add the following at
> line 274:
>
> printf ("Debug: redShift = %d\n", redShift);
> printf ("Debug: greenShift = %d\n", greenShift);
> printf ("Debug: blueShift = %d\n", blueShift);
> printf ("Debug: pf.bigEndian = %d\n", pf.bigEndian);
>
> compiled it and ran that version of vncviewer.
>
> On the PPC mac I get the following output:
>
> Debug: redShift = 8
> Debug: greenShift = 16
> Debug: blueShift = 24
> Debug: pf.bigEndian = 1
>
> and the colors on the display are wrong.
>
> On the Intel mac I get:
>
> Debug: redShift = 16
> Debug: greenShift = 8
> Debug: blueShift = 0
> Debug: pf.bigEndian = 0
>
> and the colors on the display are right.
>
> Chris Jordan
>
> ---- DRC <dcomman...@users.sourceforge.net> wrote:
>> More than likely so. The color shifting would be consistent with trying
>> to draw an RGBX bitmap when the system expects XRGB, and I used to see
>> the same problems on Sparc TurboVNC clients whenever the pixel format
>> wasn't converted properly. The code that handles that in TigerVNC
>> (common/rfb/tightDecode.h) looks correct to me, but I don't know if
>> anyone has actually tested it on a big endian X server.
>>
>> Do you have any way to insert a print statement at line 273 in that file
>> (or run it in gdb) and determine what redShift, greenShift, blueShift,
>> and pf.bigEndian are being set to on your client machine? That will
>> tell me whether or not our guess as to what cinfo.out_color_space should
>> be set to on big endian machines is correct.
>>
>> Also, I am assuming this works correctly when you connect using a little
>> endian client? You didn't specify whether or not the Solaris machine
>> was Sparc or not, but I'm assuming not.
>>
>>
>
>
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> Tigervnc-users mailing list
> Tigervnc-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/tigervnc-users
------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Tigervnc-users mailing list
Tigervnc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tigervnc-users
