[This message was posted by Mike Chan of NIL <[email protected]> to the "Product Discussion" discussion forum at http://fixprotocol.org/discuss/24. You can reply to it on-line at http://fixprotocol.org/discuss/read/cafc6cfb - PLEASE DO NOT REPLY BY MAIL.]
Hi All, The following is posted again in this forum as I've posted to General Q/A before. See if you have any suggestion:- *** Currently, we have 2 major problem areas in our FIX development: 1) whenever we have change in our FIX application that may affects our buy/sell side FIX parties (e.g. as minor as correcting passing/not passing a tag value), besides internal testing, we need to invite a whole lot of FIX parties for regression test. The tasks of the regression test is so time consuming. 2) Whether in production connection or certification testing on new connection, it is a headache to discover non-conformance area to FIX specification piece by piece, ending up impacting the production trade or timeline for go-live on new connection. A year ago, we try to find an automated testing tool which can alleviate our burden on FIX testing. However, the result is not good: > Requirement #1 Validation of Conformance to FIX Standard (e.g. version 4.2) a) Equipped with semantic based validation, e.g. correct values of orderStat, based on precedence b) Be flexible enough to overrided/bypass the rule according to customization of own / our FIX parties' application c) Have a rich set of scenario for validation (at least should have all cases from FIX spec 4.2 Appendix D) > Evaluation Result for Requirement #1 The tools we tried: - does not have built in test scenario for FIX 4.2 verification [e.g. semantic based validation, e.g. correct values of orderStat, based on precedence). We will need to create scenario as in FIX spec Appendix D to cover the whole scope to verify FIX conformance - there are only limited number of validation profile by default - unable to validate calculated tag in Execution Report e.g. AvgPx (6), CumQty (14), LeavesQty(151) > Requirement #2 Validation by Production Log a) Via import of production FIX log, the tool will help to do some automatic regression test: i.e. check if our buy-side / sell-side FIX application still behave correctly (i.e. as in current production) after the change. e.g. this may be done by replaying the FIX order from production log and comparing the execution report received against that in log. > Evaluation Result for Requirement #2 For the tool we tried, all the message (orders, execution report) should be in exactly the same sequence as in the log. That's imply we have to carefully subdivide / extract units of scenario from the production logs. Otherwise, we can reproduce the response as in the log via our FIX application to arrive with the same execution report as in the log for comparsion. To extract scenario, it would takes time to select the useful cases Recently, we are revisiting to see if there is any better automated FIX testing tool in the market which can somehow achieve part/most of our problems. So, I am making a post here just to see if there is any better tools that you guys/gals have tried and can be recommended. *** Thanks very much! Mike [You can unsubscribe from this discussion group by sending a message to mailto:[email protected]] -- You received this message because you are subscribed to the Google Groups "Financial Information eXchange" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/fix-protocol?hl=en.
