[This message was posted by Mikael Brannstrom of Nordic Growth Market <mikael.brannst...@ngm.se> to the "General Q/A" discussion forum at http://fixprotocol.org/discuss/22. You can reply to it on-line at http://fixprotocol.org/discuss/read/84aee162 - PLEASE DO NOT REPLY BY MAIL.]
Hi Dennis, The large retransmissions may occur on market data feeds. If a client gets disconnected for example one or few hours intraday and then reconnects, then that client have missed "GB of data" that need to be retransmitted. Instead a snapshot would be a better idea, since the client would get in sync much faster. Regards Mikael > Hi Mikael, > > "...with a new field, e.g. MsgHistoryLostIndicator=Y/N (default=N), that > tells the other side that it probably need to request snapshots since some > important messages could not be retransmitted." > > This is good. I've never encountered a system doing the "we only store X > messages" yet it seems to me that the MsgHistoryLostIndicator will be a vital > part of such functionality. > > I submit to you, though, that if systems are requesting such large > retransmissions there will be bigger hardware and software issues needing > resolution. (How were there so many messages 'in the pipe' between the two > systems unprocessed by the receiver? You suggest GB of messages. What > communications medium supports such buffer sizes? If not in the transport > layer, how can a receiving system with such insufficient performance get into > production?) > > Re: "bilateral agreement" - while this sounds good for making a proposal more > acceptable, in practice it isn't quite that simple. Vendors and clients can > become entrenched (as I am) as to their positions. I.e. - one party demands > the bilateral agreement while the other opposes it. > > In our "bad experiences" the only thing that resolved the impass was the fact > that we were not engaging in 24 hour connectivity. This proposal removes that > obstacle. > > Once again, best regards, > > Dennis > > > Hi Dennis, > > > > Thanks for your more detailed comments. > > > > Would the following be a better solution? > > > > 1. "the system only stores X messages" is responded with a GapFill with a > > new field, e.g. MsgHistoryLostIndicator=Y/N (default=N), that tells the > > other side that it probably need to request snapshots since some important > > messages could not be retransmitted. > > We will probably have to come up with a better name for the new field to > > capture other usages. For example, it might be a bad idea to resend several > > GB of data intraday - instead a snapshot would be a quicker way to recover. > > > > > > 2. "It seems that the more appropriate [...] include the 'next expected seq > > num' in the logout". Good point. This approach does not rule out the reset > > on logon feature. Both will work in parallel. > > > > 3. Reset on logon is an optional feature. Same as is "Logon Message > > NextExpectedMsgSeqNum Processing" and "24 hour logon reset" as it seems > > since it's a bilateral agreement when and who initiates the 24 hour logon > > reset. > > > > The "logon reset" is still usable for example if the client side did crash > > and quickly want to logon instead being forced to logon, receive a logout > > to see the NextExpectedSeqNum and then logon again. There are other use > > cases as well ... > > > > 1, 2 and 3 together meet the requirements of NGM. > > > > > > For future reference: > > > > "If memory serves, it is not allowed to seqeuence-reset-reset to a number > > lower than what is next expected." > > > > You're correct. This one was hard to find in the FIXT 1.1 spec. See test > > case with Ref ID 11c at page 46. Otherwise the spec is not that clear about > > this. > > > > > > > > Regards > > Mikael Brännström, NGM > > > > > > > > > Hi Mikael, > > > > > > I apologise for having to stand against this proposal. In response to > > > your queries... > > > > > > If memory serves, it is not allowed to seqeuence-reset-reset to a number > > > lower than what is next expected. You are only allowed to reset higher. > > > Any session approaching MAX_INT messages over one session likely more > > > urgently needs to split up the traffic over multiple sessions. > > > > > > As regards your previous question re: costs to redevelop and test > > > existing systems. Much of the session state and logic is predicated upon > > > existing data having been committed and applied to the receiving system's > > > state. Sequence numbers never go lower and the session rules guard > > > against that allowing deeper code to safely assume that predicate. > > > Further, the message logs document the contracted commitments of the two > > > parties on either side of the connection allowing for issue resolution. > > > > > > So existing systems would potentially need to be redeveloped, retested > > > and recertified if/when new clients claim that our FIX implementation > > > does not conform to the FIX standard. > > > > > > We have experienced clients & vendors demanding we support this behavior > > > after having seen the 24 hour reset seq num "feature" in the FIX > > > specification. This led to many headaches which too often were abused by > > > vendors and clients, creating potential liabilities for our company. The > > > vast majority of the time connecting parties simply wanted to avoid > > > fixing their bugs or avoid putting into place the required hardware. > > > > > > Ultimately, I would not have any concerns with this proposal if our > > > experiences would have been more positive. Instead, we had nothing but > > > bad experiences with remote connecting vendors and clients using the > > > reset seq num feature as an "out" which then required us to work around > > > their irresponsibility. Where does our liability and responsibility end? > > > > > > It seems that the more appropriate approach to the scenarios this > > > proposal covers is for the session initiator to accept that the bug or > > > hardware failure occured on *its* side and, rather than demanding that > > > the session acceptor reset sequence numbers lower, the session initiator > > > should increase its sequence numbers and notify operators that > > > investigation is required. The session acceptor can include the "next > > > expected seq num" in the logout which rejects the logon. In the case of > > > "the system only stores X messages" then the retrans request should be > > > gap filled up to the point at which the X messages can be resent. > > > Overall, I believe that this is what the FIX protocol presently > > > recommends and such a solution does not impact existing implementations. [You can unsubscribe from this discussion group by sending a message to mailto:unsubscribe+100932...@fixprotocol.org] -- You received this message because you are subscribed to the Google Groups "Financial Information eXchange" group. To post to this group, send email to fix-proto...@googlegroups.com. To unsubscribe from this group, send email to fix-protocol+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/fix-protocol?hl=en.
