Bug Tracker item #3305357, was opened at 2011-05-20 19:00 Message generated for change (Comment added) made by bphinz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126848&aid=3305357&group_id=254363
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: UN*X version Group: 1.1.X Status: Open Resolution: None Priority: 5 Private: No Submitted By: Brian Hinz (bphinz) Assigned to: Adam Tkac (atkac) Summary: Enabling custom compression level on client crashes server Initial Comment: Enabling custom compression causes the server to crash with the following log message: Fri May 20 18:39:07 2011 VNCSConnST: Client pixel format depth 24 (32bpp) little-endian rgb888 Fri May 20 18:40:11 2011 Connections: closed: 10.1.1.20::42053 (ZlibOutStream: deflate failed) SMsgWriter: framebuffer updates 148 SMsgWriter: copyRect rects 43, bytes 688 SMsgWriter: Tight rects 592, bytes 311570 SMsgWriter: raw bytes equivalent 18822164, compression ratio 60.410707 Segmentation fault Tried from both java and Windows exe. Tried DRC's latest nightly build as well as r4428 (1_1 branch) built on RHEL4. ---------------------------------------------------------------------- >Comment By: Brian Hinz (bphinz) Date: 2011-06-17 09:21 Message: Seems good. No problems at all on RHEL4 for several days now, limited testing with RHEL5, but so far so good. I say go ahead and close it. ---------------------------------------------------------------------- Comment By: D. R. Commander (dcommander) Date: 2011-06-14 23:25 Message: Try the latest pre-release build at: http://www.virtualgl.org/DeveloperInfo/TigerVNCPreReleases Seems to be fixed as far as I can tell. If it works for you, I'll go ahead and close the issue. ---------------------------------------------------------------------- Comment By: Brian Hinz (bphinz) Date: 2011-06-14 15:20 Message: Can you try applying the patch that I uploaded (rev2) and see if it fixes the issue? I've been chugging along on RHEL4 (x86_64) for about 4 hours now, periodically changing the compression level, and have not been able to reproduce the error. I was not previously linking against the static libraries, but this time I added "GNUTLS_FLAGS='/usr/lib64/libgnutls.a /usr/lib64/libgcrypt.a /usr/lib64/libgpg-error.a /usr/lib64/libgnutls-extra.a' --with-included-zlib" to 'build-xorg build' (the --with-included-zlib should be redundant because of '-static', but I left it there for good measure). I won't be able to test this on RHEL5 until later tonight, but it seems to me that the error is more reproducible on RHEL5 than RHEL4(?). FYI, the patch alone did not cure the issue for me, so if it does work, it seems to be due to some combination of the patch and the requirement to link against the static libraries... Thanks, -brian ---------------------------------------------------------------------- Comment By: D. R. Commander (dcommander) Date: 2011-06-14 14:16 Message: I don't think it will. I link against static everything, and I still get the error. ---------------------------------------------------------------------- Comment By: Brian Hinz (bphinz) Date: 2011-06-14 10:06 Message: I'm still struggling to figure this out, but I wonder if it's related to which version of zlib we're linking against in the legacy build. I'm using the statically linked binaries produced by the build-xorg script, which links Xvnc against the system version of gnutls, and the in-tree version of zlib. However, the system version of gnutls already depend on the system version of zlib. The in-tree version of zlib appears to be 1.2.5, while the system version of zlib is 1.2.1 and 1.2.3 on RHEL4 and RHEL5 respectively. I'm going to try rebuilding and linking everything against the static versions of gnutls, libgcrypt, and libgpg-error along with the in-tree zlib and see if that helps. ---------------------------------------------------------------------- Comment By: Brian Hinz (bphinz) Date: 2011-06-02 19:53 Message: Don't commit it yet, there's still something wrong... Setting the compression level to 1 still crashes the server. ---------------------------------------------------------------------- Comment By: D. R. Commander (dcommander) Date: 2011-06-02 14:00 Message: Seems OK to me. I'd like to hear from Adam before committing it. ---------------------------------------------------------------------- Comment By: Brian Hinz (bphinz) Date: 2011-05-28 10:15 Message: Sorry, SYNC_FLUSH does seem to work. FULL_FLUSH causes a segfault when the client chooses compression level 1. Attaching new patch. ---------------------------------------------------------------------- Comment By: Brian Hinz (bphinz) Date: 2011-05-22 15:47 Message: Can someone review the attached patch? It seems to resolve the issue, but to be honest I don't know much about compression. The libz spec says the following: <snip> Applications should ensure that the stream is flushed, e.g. by a call to deflate(stream, Z_SYNC_FLUSH) before calling deflateParams(), or ensure that there is sufficient space in next_out (as identified by avail_out) to ensure that all pending output and all uncompressed input can be flushed in a single call to deflate(). Rationale: Although the deflateParams() function should flush pending output and compress all pending input, the result is unspecified if there is insufficient space in the output buffer. Applications should only call deflateParams() when the stream is effectively empty (flushed). </snip> So it seems like the Z_FULL_FLUSH is not necessary, however a Z_SYNC_FLUSH didn't work. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126848&aid=3305357&group_id=254363 ------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ Tigervnc-devel mailing list Tigervnc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tigervnc-devel
