Feature Request Tracker item #3611531, was opened at 2013-04-21 23:28
Message generated for change (Comment added) made by dcommander
You can respond by visiting: 

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: UN*X version
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: liyusen (liyusen)
Assigned to: Adam Tkac (atkac)
Summary: X264 encoding support

Initial Comment:
Since TigerVNC is good for video and games. Why not consider to implement X264 
encoding in the protocol? I am not sure whether it is too difficult or the 
improvement is not so large?


>Comment By: DRC (dcommander)
Date: 2013-04-22 00:09

I can speak to this somewhat, since I designed the encoding methods used by
TigerVNC and TurboVNC.  Both solutions use a modified form of Tight
encoding, which was itself developed by Constantin Kaplinsky.  All variants
of VNC can send two types of framebuffer updates to the client:  full and
incremental.  Typically, full FBU's are sent only at the start of the
connection or if the client window becomes obscured or something like that.
 Incremental FBU's are sent whenever something in the remote framebuffer
changes and the server needs to send that change to the client.  An
incremental FBU can contain multiple rectangles, representing the areas of
the remote FB that have changed.  With Tight encoding, each rectangle is
further broken down into "subrectangles".  The encoder attempts to find any
significant areas of solid color within the rectangle first and encode
those as a bounding box and a fill color.  The remaining subrectangles in
the rectangle are encoded based on the number of colors in each.  If a
subrectangle has only 2 colors, it is encoded as a 2-deep palette and a
1-bit bitmap.  If it has a low number of colors > 2 and < N (N is typically
24 or 96 with TigerVNC/TurboVNC), it is encoded as an 8-bit indexed color
image.  Otherwise, it is encoded as either JPEG or raw RGB.  Raw RGB,
indexed color, and monochrome subrects are further compressed using Zlib.

In the case of TigerVNC and TurboVNC, the mix of subrectangles has been
carefully designed to provide the best balance between good throughput for
3D apps and good compression for 2D apps.  There are several knobs that can
be tweaked to fine-tune that.  Increasing the compression level from 1-2,
for instance, improves compression for 2D apps at the expense of 3D app

This is all a roundabout way of saying that H.264 isn't competing directly
with motion JPEG in our case.  It's competing with a hybrid protocol that
has been tailored somewhat to the needs of video and 3D apps and which uses
JPEG only for areas of the display that have a high number of unique
colors.  H.264 relies on interframe compression for a lot of its bandwidth
savings, and because we're dealing with a protocol that fundamentally
doesn't have a concept of "frames", we'd either have to make a simplifying
assumption -- such as assuming that H.264 will only be used with
full-screen video or 3D apps, and thus we can make all framebuffer updates
full and not incremental -- or we'd have to throw away the interframe
capabilities of H.264, which also throws away a lot of its advantages.

H.264 uses a more advanced entropy codec than JPEG, so it may still
compress a bit better than JPEG, even disregarding interframe compression,
but that is unknown and probably would depend a lot on the nature of the
image workload.  It's also unknown how the throughput compares, but I
seriously doubt that any H.264 codec is going to be as fast as
libjpeg-turbo, so that would further confine H.264 to being primarily
useful on WANs (well, slow WANs anyway.  I just found out that my town's
getting gigabit Google Fiber next year.)

I'm not saying H.264 isn't potentially useful-- I'm just saying it's a
nuanced issue, and H.264 would not be something that would make a good
"general-purpose" VNC solution.  It would be a specialty solution that
might only reduce bandwidth for specific types of apps in specific
scenarios, and even in those scenarios, it might only reduce bandwidth by
15% (which could be accomplished with the existing solution by lowering the
JPEG quality.)  I don't think it's going to be a quantum leap by any means.
 I think that in order to really see the advantages of it, we would have to
make the simplifying assumption above -- always send the full frame using
non-incremental FBU's, which would of course make the performance suck
pretty bad for anything besides the specific video app or game.  I would
also imagine that using the interframe capabilities of H.264 will add to
encoding latency.


You can respond by visiting: 

Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
Tigervnc-devel mailing list

Reply via email to