Feature Request Tracker item #3611531, was opened at 2013-04-21 23:28 Message generated for change (Comment added) made by dcommander You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126849&aid=3611531&group_id=254363
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: UN*X version Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: liyusen (liyusen) Assigned to: Adam Tkac (atkac) Summary: X264 encoding support Initial Comment: Since TigerVNC is good for video and games. Why not consider to implement X264 encoding in the protocol? I am not sure whether it is too difficult or the improvement is not so large? ---------------------------------------------------------------------- >Comment By: DRC (dcommander) Date: 2013-04-22 00:09 Message: I can speak to this somewhat, since I designed the encoding methods used by TigerVNC and TurboVNC. Both solutions use a modified form of Tight encoding, which was itself developed by Constantin Kaplinsky. All variants of VNC can send two types of framebuffer updates to the client: full and incremental. Typically, full FBU's are sent only at the start of the connection or if the client window becomes obscured or something like that. Incremental FBU's are sent whenever something in the remote framebuffer changes and the server needs to send that change to the client. An incremental FBU can contain multiple rectangles, representing the areas of the remote FB that have changed. With Tight encoding, each rectangle is further broken down into "subrectangles". The encoder attempts to find any significant areas of solid color within the rectangle first and encode those as a bounding box and a fill color. The remaining subrectangles in the rectangle are encoded based on the number of colors in each. If a subrectangle has only 2 colors, it is encoded as a 2-deep palette and a 1-bit bitmap. If it has a low number of colors > 2 and < N (N is typically 24 or 96 with TigerVNC/TurboVNC), it is encoded as an 8-bit indexed color image. Otherwise, it is encoded as either JPEG or raw RGB. Raw RGB, indexed color, and monochrome subrects are further compressed using Zlib. In the case of TigerVNC and TurboVNC, the mix of subrectangles has been carefully designed to provide the best balance between good throughput for 3D apps and good compression for 2D apps. There are several knobs that can be tweaked to fine-tune that. Increasing the compression level from 1-2, for instance, improves compression for 2D apps at the expense of 3D app performance. This is all a roundabout way of saying that H.264 isn't competing directly with motion JPEG in our case. It's competing with a hybrid protocol that has been tailored somewhat to the needs of video and 3D apps and which uses JPEG only for areas of the display that have a high number of unique colors. H.264 relies on interframe compression for a lot of its bandwidth savings, and because we're dealing with a protocol that fundamentally doesn't have a concept of "frames", we'd either have to make a simplifying assumption -- such as assuming that H.264 will only be used with full-screen video or 3D apps, and thus we can make all framebuffer updates full and not incremental -- or we'd have to throw away the interframe capabilities of H.264, which also throws away a lot of its advantages. H.264 uses a more advanced entropy codec than JPEG, so it may still compress a bit better than JPEG, even disregarding interframe compression, but that is unknown and probably would depend a lot on the nature of the image workload. It's also unknown how the throughput compares, but I seriously doubt that any H.264 codec is going to be as fast as libjpeg-turbo, so that would further confine H.264 to being primarily useful on WANs (well, slow WANs anyway. I just found out that my town's getting gigabit Google Fiber next year.) I'm not saying H.264 isn't potentially useful-- I'm just saying it's a nuanced issue, and H.264 would not be something that would make a good "general-purpose" VNC solution. It would be a specialty solution that might only reduce bandwidth for specific types of apps in specific scenarios, and even in those scenarios, it might only reduce bandwidth by 15% (which could be accomplished with the existing solution by lowering the JPEG quality.) I don't think it's going to be a quantum leap by any means. I think that in order to really see the advantages of it, we would have to make the simplifying assumption above -- always send the full frame using non-incremental FBU's, which would of course make the performance suck pretty bad for anything besides the specific video app or game. I would also imagine that using the interframe capabilities of H.264 will add to encoding latency. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126849&aid=3611531&group_id=254363 ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter _______________________________________________ Tigervnc-devel mailing list Tigervnc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tigervnc-devel
