[This message was posted by Rolf Andersson of Pantor Engineering <[email protected]> to the "FAST Protocol" discussion forum at http://fixprotocol.org/discuss/46. You can reply to it on-line at http://fixprotocol.org/discuss/read/ac5c73d9 - PLEASE DO NOT REPLY BY MAIL.]
LiuAn, ZLIB is a generic text compression library based on the LZ77 (Lempel Ziv 1977) family of compression algorithms. It allows you to compress any message stream without reconfiguration or adaptation so in that sense ZLIB is less complex to deploy. I would argue that this difference is small given that the implementation of a feed is content specific anyway. The implementation (ie. the code) of FAST and ZLIB respectively are fairly complex but (having implemented both) I believe FAST is easier to implement than ZLIB. As there are FAST and ZLIB implementations available, you don't have to go to the trouble of actually doing your own implementation but can instead use an existing implementation. ZLIB and ZLIB-like implementations have been used by the CME, OMX (SAXESS and CLICK), Nasdaq (ITCH) and others. As far as I know, CME has discontinued its use of ZLIB and OMX has discontinued at least some of their use of a ZLIB-like implementation. Not sure if ITCH compression is still being offered. The big drawbacks with ZLIB is its relatively high processing overhead and its reliance on a long history of data to compress effectively. As a consequence ZLIB in not well-suited for high performance situations nor when UDP over multicast is used. It has been shown that the processing overhead of ZLIB is more than 30 times higher than FAST in some use cases. The compression ratios of ZLIB and FAST are comparable for TCP streams and FAST has much better compression efficiency for in the UDP case. In summary, if you plan to provide a low to medium thruput market data feed which is not latency sensitive and you plan to use TCP as your transport, then ZLIB could used to compress your data. For other scenarios I would recommend that you review the commercial alternatives available for deploying a FAST-based feed. Best, Rolf > Hi all, > > I'm considering to compress market data messages using zlib. > > I found some discussions in this forum regarding zlib v.s. FAST. From > performance point of view, FAST is a better choice. But FAST seems to be more > complex in implementation. > > I'd like to know if zlib is considered a valid implemetation for the market > data compression? Is it used by anybody in production? > > Thanks & Regards, > LiuAn [You can unsubscribe from this discussion group by sending a message to mailto:[email protected]] -- You received this message because you are subscribed to the Google Groups "Financial Information eXchange" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/fix-protocol?hl=en.
