[This message was posted by Daniel May of SpryWare, LLC <[EMAIL PROTECTED]> to the "FAST Protocol" discussion forum at http://fixprotocol.org/discuss/46. You can reply to it on-line at http://fixprotocol.org/discuss/read/04123780 - PLEASE DO NOT REPLY BY MAIL.]
I think the complexity could be diminished if an Exchange encoding FAST data provided a standard decoder in the various popular implementation languages. When SpryWare did the initial ARCA FAST implementation, we provided a 'C' decoder that was free to use by ARCA customers. It gave a baseline to compare decoded data to the original data so if you did decide to do your own decoder implementation, you could at least "diff" your decoder output with the official Exchange provided decoder. Just as not many people pick up and write their own zlib or jpeg decompressor, FAST needs to get to a point where there are good, reliable, high performance decoders available for free. Perhaps the business model is to sell encoders and give decoders away for free ? Adobe seems to be successful with this model, giving away the reader for free. In regards to production FAST based feeds, I agree that RLC was a much easier feed to handle, and the CME FAST implementation is very, very complicated. Here are my observations: 1. When ARCA and Opra went to FAST, it was a FAST only implementation. No FIX was involved. In the simplest sense you could take the supplied decoder and simply decode the FAST message back to the original, proprietary ARCA Book Binary message or Opra ASCII message. In some cases it was simply inserting a line of code to do the decoding, much like unzipping a zlib based data stream. Since both exchanges ran the old and the new feeds in parallel, you could compare the outputs and validate the data. 2. Once you had that up and running, most realized the conversion to the original ACSII or binary format was an unnecessary step, and moved to process the FAST message directly, but this was an option, not mandatory. 3. In both the Opra and ARCA implementations, the basic business logic involved in processing the feed did not change at all, it was purely for bandwidth (and CPU) reduction. 4. The CME decided to kill two birds with one stone, implement a replacement to RLC with a FIX based market data feed, and use FAST to encode/decode that feed. So the wire format changed, and the logical format, along with a new set of business rules to learn. Throw into that a mix of dozens of multicast groups, a separate "Instrument Definition" channel, a replay channel that needs to be synchronized with the live channel, and it takes a mound of resources to get your hands around it. An aggressive plan. 5. Along with the new CME FAST came the idea of templates and template management. The idea of a hard coded "state" machine became very complex because message structure was dynamic. 6. The only reference implementation was in Java, a C++ one surfaced then was marked unsupported. Our experience along with many others was the C++ implementation was buggy, over complicated (STL), and single threaded. It did not port well to 64-bit environments, and did not even compile under Win32. 7. So as most, we embarked on our own C/C++ decoder, found a reasonable xml parser to read the templates, and started producing "FIX like" messages from the FAST stream. But how do you know the "FIX like" message that you decoded and assembled is really correct ? There is no existing feed to compare it to as RLC does not contain the same FID's, and there is no checksum or CRC that says "When you build a FIX message from this FAST message, it will CRC to this value...." The best you can do is pass CME certification and assume you have it all correct, building in basic sanity checks that test the business rules against the data. 8. Then you run it against production data at a customer site (the certification is against canned data) and realize that the test data only covered a fraction of the production data set, and you start decoding multicast frames by hand to figure out what is going wrong. Not for the junior programmer or faint at heart. 9. So while all of the complaints against the CME feed are falling on FAST, some are not FAST or FIX related, but rather the implementation of the business logic of the feed. My long 2 cents. Daniel May SpryWare, LLC [You can unsubscribe from this discussion group by sending a message to mailto:[EMAIL PROTECTED] --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Financial Information eXchange" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/FIX-Protocol?hl=en -~----------~----~----~----~------~----~------~--~---
