[This message was posted by Fernando Jeronymo of Societe Generale <[email protected]> to the "FAST Protocol" discussion forum at http://fixprotocol.org/discuss/46. You can reply to it on-line at http://fixprotocol.org/discuss/read/11555ff4 - PLEASE DO NOT REPLY BY MAIL.]
Yes, I agree with you that continuing from where it failed is the best for this particular market... However we still can't differentiate between a fragmented message (incomplete but eventually will get there) and a real buggy one without adding counters and position checking and other overhead which nobody wants :) Thanks for the help/suggestions... the truth is, if they did message blocks then everything is much simpler / faster. Regards, --Fernando > Sorry Fernando as I have no experience with EBS. I'm curious now though, > you do "read/decode/error/repeat"... You're not starting all over > decoding are you? The reason I ask is because you mentioned earlier > about wanting the length of the FAST message. If you're waiting till the > entire FAST message is received to start decoding, that would be pretty > inefficient. In other words you should be decoding the message as it is > received and when you receive more data you should decode from where you > left off not starting at the beginning. It sounds like you already know > this since you know about nagling and fragmentation, but if not > hopefully this info is helpful for you. > > Brandon > > > We are aware of the Nagle algorithm and TCP fragmentation, the issue > > is that EBS Live FastFix is using FIX/FAST over TCP but they are not > > blocking it... yet they claim that they follow FAST 1x1 specs. > > > > I was hoping that someone else had worked with them before and had a > > better solution other than (read/decodeKO/error/repeat until decodeOK) > > or maybe we could put pressure on them to at least block their data as > > Rolf suggested. > > > > Regards, --Fernando > > > > > Hi Fernando, > > > > > > TCP packets have the ability of being fragmented or even merged > > > together especially when using the Nagle algorithm (which most > > > sockets use). I guess what I'm trying to say is you should never > > > expect to receive just one complete packet when dealing with TCP. So > > > I think what you are doing is perfect (decode as much as you can, > > > wait for more data then continue decoding). > > > > > > Brandon > > > > > > > Hi Rolf, > > > > > > > > As per FAST 1x1 Section 10, we would have "BLOCK SIZE (1000 bytes) > > > > | MSG > > > > | ... | MSG | X (byte 1001)" > > > > > > > > Now the question is, regarding EBS, I am not sure they are doing > > > > that... What we seem to receive (and as per their documentation) > > > > we have: > > > > > > > > "MSG | MSG | MSG | ... | MSG" without the block. > > > > > > > > As per EBS spec (EBS Live FastFIX version 6.3.pdf): > > > > > > > >3. 4 FAST/FIX Considerations The FAST/FIX message will consist of > > > > a > > > > sequence of individual messages, not blocks of messages. > > > > > > > > Can you or someone else confirm that EBS does not send blocks? I > > > > just am unsure on how to decode FIX over FAST over TCP if they are > > > > not within a block... right now we try to decode, error, wait for > > > > more data, try to decode, error, until I can successfully decode a > > > > message. > > > > > > > > Regards, --Fernando > > > > > > > > > Hi Fernando, > > > > > > > > > > please have a look at section 10 of the FAST specification. You > > > > > can use blocks to enable reading a whole message before starting > > > > > to decode the message. You can also put more than one (complete) > > > > > message into a block. > > > > > > > > > > Let me know if you have any further questions. > > > > > > > > > > /Rolf > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > What is the proposed way of correctly parsing a TCP FAST > > > > > > encoded message? (FIX over FAST)? > > > > > > > > > > > > If we are in FIX world, I have: "8=FIX.M.N^A9=...^A10=123^A" > > > > > > > > > > > > So I can easily search for "8=FIX...^A" and "^A10=NNN^A", and > > > > > > here I have my SOM (Start of Message) and EOM (End of Message) > > > > > > > > > > > > The issue is, this gets encoded over FAST and sent on TCP... > > > > > > then we retrieve a FAST encoded message but we don't have any > > > > > > way to know when we should start FAST decoding. At this point > > > > > > we try to decode, if the message is incomplete I generate an > > > > > > error and wait for more data and then when I receive more data > > > > > > I try to decode again. > > > > > > > > > > > > Ideally I would have some kind of framing (not FAST encoded) > > > > > > that tells me how many bytes I should expect, or gives me the > > > > > > Begin/End of the message, so I can read everything from the > > > > > > socket first to then decode later. > > > > > > > > > > > > Is this something that has been discussed already? If there > > > > > > are some guidelines please share them with me, so I can then > > > > > > forward them to the market in question :) > > > > > > > > > > > > Best Regards, --Fernando [You can unsubscribe from this discussion group by sending a message to mailto:[email protected]] -- You received this message because you are subscribed to the Google Groups "Financial Information eXchange" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/fix-protocol?hl=en.
