About section 4.6.4 of XEP-0301: http://xmpp.org/extensions/xep-0301.html#message_retransmission
> 1. When the recipient's presence changes. (i.e. offline to online) > So let's say you are typing a message to me and my presence changes from > normal availability to do not disturb or whatever. Why does your client > need to retransmit what you have typed so far? Are there specific > presence scenarios that you have in mind here? > Problem: Without retransmit - Sender: Starts typing message while recipient is offline - (Recipient Signs on) - Sender: Continues typing - (Recipient does not read real time text because XEP-0301 stays out of sync due to section 4.6.2) - (Recipient needs to wait until Sender starts typing the next message, due to section 4.6.3) Solution: Retransmit when offline becomes online - Sender: Starts typing message while recipient is offline - (Recipient Signs on) (Can be new program launch, or new chat window, existing chat window in logoff/login cycle) - (Sender software resets the message buffer, recipient software sees it right away) - Recipient: Instantly see what the sender is typing - Sender: Continues typing, XEP-0301 stays in sync. > 2. When the recipient sends a <message/> from a different full JID than > before. (i.e. Simultaneous Logins) > You might want to explain how this recommendation is consistent with > XEP-0296. > You're right, let me do some thinking on how to incorporate this. It benefits regardless whether or not XEP-0296 is used, but I should describe the two different behaviours in Implementation Notes, and make a link to that from here. (Section 6.4.3.2) > 3. At regular intervals, to allow recovery from unexpected situations > such as lost <message/> stanzas. > [snip] > What are you designing for here? Are you defining ways to work around > dropped stanzas? If so, isn't the solution to use XEP-0198? It's not > clear to me why we're solving a lower-layer problem in the RTT spec. You are right, 0198 is one of the many solutions, but it is more complicated for many implementations, and not always pratical with all libraries (i.e. existin library, etc) Retransmission is useful for all of the below cases: - (Good excuse) Next Generation 9-1-1 reliability. Additional useful redundancy above-and-beyond that is effective in real world testing. - (Good excuse) Repopulating real-time messages in MUC for new sign-ins (because they ignore <rtt> until an <rtt> containing an event attribute) - (Good excuse) Repopulating real-time messages when clients come back (i.e. relogin, or after a crash/termination, or mobile reception loss, etc) - (Poor excuse) There are often bugs at various points in different layers (i.e. HTTP level disconnections on BOSH) that cause lost stanzas. Not all companies have time to fix buggy libraries. Yes, poor excuse. - (Reasonable excuse) Lost stanzas caused by server - (Good excuse) Intermittent mobile phone reception loss. Even if the whole chain has proper flow control & verification of all buffers/delivery, we STILL need to refresh the real-time buffer immediately upon end-to-end communications, so the recipient can instantly "catch up" - (Reasonable excuse) Eliminates latency caused by message acknowledge - (Good ECONOMIC excuse) We don't want to exclude any XMPP library from being able implement XEP-0301 because of user frustrations with real-time text usuability issues that are solved by retransmissions. Libraries can be improved separately and concurrently - (Good END-USER excuse) It is a very serious usability issue if real-time text gets lagged or frozen. People don't want to use real-time text if it is not ultra-reliable in getting real-time text there. - (Reasonable excuse) XEP-0198 benefits XEP-0301, but we believe XEP-0301 shouldn't require XEP-0198 in order to function reliably I agree that in an *ideal* world, we only need one layer of good error recovery. But in the real world, we have lots of safeguards, so that several things have to fail in order for things to stop being usable. We found that it's been necessary to allow XEP-0301 to retrasmit. Although it's not necessary for just technical interop -- perhaps it should be an Implementation Note for section 4.6.4 (while keeping 4.6.1 / 4.6.2 / 4.6.3 an interop issue, as they are) It is our feeling that 4.6.4 should stay in Section 4 (Protocol) rather than moved to Section 6 (Implementation Notes), due to the criticalness of reliability of real time text always being current. Redundancy doesn't hurt, except a few percent of bandwidth, usually less than a 10% premium. RFC 4103 and T.140 have a mode of redundancy too, which was also found necessary to be included in those respective specifications (though that's more because it needs to be UDP friendly) > I will clarify this in the spec; > > Section 7.2 would occur if the text was written fast enough to fit in > > one transmission inteval (i.e. 700ms, the default recommendation) > > Section 7.3 would occur if the text was written slow enough to cover > > four transmission interval cycles (i.e. 700ms x 4 = 2.8 seconds) > > This is unrelated to congestion control; can you give me context as to > > what made you think 7.2 vs 7.3 was a congestion control issue? > > Because if I send 20 message stanzas when I could send one message > stanza instead, I'm potentially causing additional overhead on my > connection (and yours). > Yes, but that's a necessary design of the spec. *Real-time text MUST stay _real-time_.* The typing should be lagged by less than 1 second. When the recipient types, the sender should see the recipient's typing virtually instantly, in real-time. Therefore, if you're continuously typing non-stop for 20 seconds, it's necessary to send at LEAST 20 stanzas -- there's no way around that -- or you're not compliant with ITU-T Rec. F.700, or the Total Conversation requirements -- it is already in use by the deaf public in Europe's 1-1-2 emergency service trials at http://www.reach112.eu .... This is one of XEP-0301 requirements: Low latency real time text. I'm surprised you asked that question, because it suggests that the purpose of XEP-0301 may be partially misunderstood -- Do you mind if you can clarify the part of XEP-0301 that made you say that you could just send "one message stanza"? I need to understand so I can improve the spec to clarify why it's necessary to continuously send <message/> stanzas every 300ms to 1000ms. NOTE: We found that Google Talk can handle approximately 50 stanzas per second before their flooding protection kicks in, and jabber.org servers (while slower) handle it fine in our testing. Most instant messages are usually less than 40 characters, so people are already sending stanzas rapidly nowadays -- so this isn't much of an increased load. > I have never seen a message size limit that small. Some services might > have a limit of 64k. > Ok, same problem applies -- 64k copy and paste (From a webpage), resulting in 64k <rtt/> + 64k <body/> + overhead. However, you're right, it might be overly reactive to complicate the spec. I'll try to simplify this section. My main concern was about Unicode normalization forms -- e.g., NFC would > leave Roman numeral IV (U+2163) as one code point, whereas NFKC would > normalize that character to I (U+0049) V (U+0056). The XMPP RFCs say > nothing about Unicode normalization of XML character data, and I think > it would be best to leave it that way. Normalization is not a problem as long as it's done before the RTT encode on the sender end, and only after the RTT decode on the recipient end. As you can see, we solved the normalization problem -- but we removed the wording about normalization from the spec. Should we re-add the wording about normalization? Please open the PDF file at http://www.realjabber.org/flowchart_of_xmpp_rtt_path.pdf -- see where it says "Normalization" at the sender end and at the recipient end. Christian/Norm (the author of real time text in American Online's AIM, but wants to adopt XEP-0301 from now on) totally agreed that normalization is not a problem as long as it occurs /before/ the RTT encode, or /after/ the RTT decode, according to the flowchart. It's more natural for actual XEP-0301 implementors to work with Code Points than to work with UTF-8. Christian Vogler/Norm Williams -- the author of the real-time feature now built into AOL Instant Messenger -- agrees that we must do Code Points instead of UTF-8. Over time as we worked with XEP-0301, we have gradually come to actual near-unamious agreement between actual XEP-0301 implementors that it's more natural to do Code Points. Since you're still recommending UTF-8, we need to be absolutely sure that Code Point is the right way to go; while the actual people working with real time text, are insisting on Code Points rather than UTF-8. > So in the RTT spec we could > assume that no normalization is applied, and I think that is the safest > approach for several reasons (in fact we probably want to explicitly > state that applications MUST NOT apply Unicode normalization to the XML > character data of RTT messages). > I agree with you that talk about Normalization should now be added back to the spec. Do you agree with where normalization is allowed/disallowed as indicated in http://www.realjabber.org/flowchart_of_xmpp_rtt_path.pdf ? Is there a reason you don't want to calculate p and n values based on > characters instead of code points? The reason I ask is that a code point > is a rather abstract notion -- if I scribble αΎ§ on a piece of paper or my > whiteboard at the office, that's still a code point (U+1FA7). In > computer systems, we typically talk about code points that have been > encoded using a character encoding scheme such as UTF-8. However, the > term "character" is used in the XML specification and is also defined in > the Unicode standard as "The basic unit of encoding for the Unicode > character encoding." It seems more natural to talk in terms of > characters than in terms of code points. > On cover, I agree... But for internal implementation, it's problematic. Especially on platforms that don't store internally in UTF-8. Actual real-time implementors within our taskgroup have gradually shifted to a preference for Code Points for many reasons (not just the ones I indicated). Your comments are welcome and useful! Thank you very much! We'd love a response to the remaining comments too - Thanks Mark Rejhon
