I have updated my XEP to be message based (the XML is attached to this
email) and would appreciate any feedback.
As previously mentioned, the only issue now is being able to specify that
chat markers should be archived as they are in a message without a body.
Regards
Spencer
On Wed, Jun 5, 2013 at 10:18 AM, Spencer MacDonald <
[email protected]> wrote:
> Should I migrate the Chat Marker XEP to a Message based solution and
> assume the issues regarding the storage of the messages with no body will
> be resolved in some way?
>
> Regards
>
> Spencer
>
>
> On Fri, May 31, 2013 at 4:05 PM, David Laban <[email protected]> wrote:
>
>> On Thu, 2013-05-30 at 18:16 +0100, Matthew Wild wrote:
>> > A more general issue is whether this XEP (or rather the specific
>> > protocol it defines) s necessary at all. I'm not saying it definitely
>> > isn't, but need a little more persuasion. For example it seems that
>> > the primary issue it is working around is that XEP-0136 and XEP-0313
>> > might not save messages with no body. Might it not be easier to solve
>> > this problem instead?
>>
>> Sorry I'm late to the party. I have actually been discussing this with
>> Spencer in the office over the last couple of weeks, so maybe I can give
>> some more motivation for why we feel that an XEP is required. I'm not
>> too attached
>>
>> There are actually a bunch of things that Chat Markers is trying to
>> solve:
>>
>> 1) Atomic "Read Receipt" messages (Or "Seen Receipts" or whatever[1]).
>> 1.1.1) Currently, our pre-alpha implementation of "Seen by $user"
>> markers requires each client to keep track of the state machine
>> (xep-0085) for *each* remote resource. Any incoming <active/>
>> notification, or any incoming <received> notification from a resource
>> that is in the 'active' state currently updates the "Seen by $user"
>> marker.
>>
>> 1.2.1) xep-0022 basically solves this problem, but it is marked as
>> obsolete. I didn't feel like digging it out of the grave, but maybe we
>> should re-consider it?
>>
>> 1.3.1) I suggested that we could simply include our state (if active) as
>> a sister element to <received/>, but Spencer pointed out that xep-0184
>> section 7 states: "When the recipient sends an ack message, it SHOULD
>> ensure that the message stanza contains only one child element". What
>> would break if we did this?
>>
>>
>> 2) State recovery for disconnected clients that come online.
>> 2.1.1) Currently, this is impossible, so our "Seen by $user" marker
>> stays where it is, and messages appear as "Not delivered" when they are
>> retrieved from MAM until a reply is received from the remote party (at
>> which point, we assume that their client has done state recovery from
>> MAM, and mark all messages as received.
>>
>> 2.1.2) xep-0184 section 5.5 Archived Messages states "An entity MUST NOT
>> send an ack message when a user views messages that have been archived
>> or stored on the client or the server (e.g., via Message Archiving [8]),
>> only when first receiving the message."
>>
>> This is annoying, but quite understandable (e.g. what should a client do
>> if it gets <received id=1234/> when it doesn't have any knowledge of
>> <message id=1234/> or when it might have been sent?)
>>
>> 2.3.1) We could allow MAM to store *all messages*, but then then a query
>> for "how many unread messages to I have since $time" returns a hugely
>> inflated answer. The only way to get an accurate count would then be to
>> retrieve *all messages* and classify them.
>>
>> 2.3.2) We could create a clone of the MAM XEP (let's call it Message
>> State Recovery: MSR) that stores everything without a body, and let the
>> clients query that in order to do state recovery.
>>
>> A little benchmarking of our client's <message/> datastore and a simple
>> thought experiment suggests that this will be many times as large as the
>> MAM database in a naive implementation (when Kate sends a message to
>> Pete, her client will send <active/>; <composing/>; (<paused/>;
>> <composing/>)* <body/>... and each of Pete's clients will send
>> <received/> and 0 or more will send <active/>.
>>
>> Note that we would need to bend the rules for xep-0184 (see 2.1.2) for
>> this to be useful. Specifically (after retrieving all messages from MSR
>> and MAM) for each incoming message in MAM that doesn't have a
>> corresponding outgoing entry in MSR, send a receipt anyway.
>>
>> 2.3.3) We could get the server to store markers for "delivered" and
>> "seen" etc. This is what Chat States attempts to do.
>>
>>
>>
>> 3) Efficiency
>> 3.1.1) 2.3.2 and 1.1.1 cover a couple of the obvious problems with what
>> we have now.
>>
>> 3.3.1) If we create a MSR XEP, what is the minimum amount of information
>> that we can store? If we have solved problem 1) then we can make a lot
>> of optimizations. For example, if we used my 1.3.1 proposal, then could
>> simply store the last message of each type?
>>
>> Concretely, could we have a database with the following uniqueness
>> constraint:
>> (sender_barejid, receiver_barejid, sibling_ns, sibling_name)
>>
>> where sibling_ns is the namespace of the element after <received/> in
>> the <message/> stanza, and sibling_name is its name (e.g. 'active' or
>> NULL)?
>>
>>
>> And in reply to Matthew Wild's specific comments:
>> >
>> > For XEP-0136, it appears to be configurable already in the archiving
>> > preferences (surprise surprise!). For XEP-0313, I'm open to discussion
>> > about what it recommends.
>> >
>> We have gone for a Message Archive Management + Message Carbons approach
>> so far, which means that clients only need to know how to unpack
>> <forwarded/>. I don't fancy forcing 3 teams to implement XEP-0136 if I
>> can avoid it.
>>
>> > XEP-0313 intentionally remains silent on most policy decisions like
>> > that. However it seemed sensible at the time that nobody would want to
>> > archive messages without a body, which on the network today are
>> > primarily chat states and notifications of various sorts. The XEP is
>> > still experimental, perhaps we can come up with better rules? I don't
>> > know, that's a discussion for another thread.
>> >
>> see 2.3.1 for why I think that MAM's rules are probably correct, and if
>> anything, we should have a parallel store for messages without bodies.
>>
>> > Forgetting archiving completely for the moment, offline messages might
>> > do enough already, no? XEP-0160 doesn't actually have any
>> > recommendations about what to store or what not to store. It seems
>> > that servers are expected to identify things like chat states already
>> > (XEP-0085 says that servers "SHOULD NOT store them offline"). This
>> > doesn't seem like a good model, but it's what we currently have.
>> >
>> XEP-0160 breaks in any use-case that involves multiple mobile devices
>> per account. I am actually thinking of disabling support for it on our
>> server completely, since all of our supported clients understand Message
>> Archive Management.
>>
>>
>> David.
>>
>>
>> [1] The jdev thread was repeatedly derailed by people querying the
>> semantics of "read", so if I say "read" and it annoys you, translate it
>> to "seen" in your head. There are also use-cases for states like
>> "notified about" and "sent/delivered out-of-band" (e.g. via Apple Push
>> Notifications or SMS) and "acknowledged". I would prefer to avoid going
>> down that particular rabbit-hole yet, but any protocol should be
>> extensible in that direction (the benchmark for extensibility here is
>> the set of SIP status codes 100 Trying (= reached first server), 180
>> Ringing (= notified about), 200 OK (= acknowledged/accepted)).
>>
>> --
>>
>> Section numbers are of the form x.y.z) where x = topic, y = status: (1=
>> where are we now, 2= where did we come from, 3= where could we be), z =
>> incrementing integer. Sometimes I have nothing to say about x.2.z.
>>
>>
>
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE xep SYSTEM 'xep.dtd' [
<!ENTITY % ents SYSTEM 'xep.ent'>
%ents;
]>
<?xml-stylesheet type='text/xsl' href='xep.xsl'?>
<xep>
<header>
<title>Chat Markers</title>
<abstract>This specification describes a solution of marking the last received, read and acknowledged message in a chat.</abstract>
<legal>
<copyright>This XMPP Extension Protocol is copyright (c) 1999 - 2013 by the XMPP Standards Foundation (XSF).</copyright>
<permissions>Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation.</permissions>
<warranty>## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. In no event shall the XMPP Standards Foundation or the authors of this Specification be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification. ##</warranty>
<liability>In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising out of the use or inability to use the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages.</liability>
<conformance>This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which may be found at <<link url='http://www.xmpp.org/extensions/ipr-policy.shtml'>http://www.xmpp.org/extensions/ipr-policy.shtml</link>> or obtained by writing to XSF, P.O. Box 1641, Denver, CO 80201 USA).</conformance>
</legal>
<number>xxxx</number>
<status>ProtoXEP</status>
<type>Standards Track</type>
<sig>Standards</sig>
<approver>Council</approver>
<dependencies>
<spec>XMPP Core</spec>
<spec>XEP-0001</spec>
<spec>Etc.</spec>
</dependencies>
<supersedes/>
<supersededby/>
<shortname>NOT_YET_ASSIGNED</shortname>
<author>
<firstname>Spencer</firstname>
<surname>MacDonald</surname>
<email>[email protected]</email>
<jid>[email protected]</jid>
</author>
<revision>
<version>0.0.2</version>
<date>2013-06-11</date>
<initials>sdm</initials>
<remark><p>Change to a message based protocol.</p></remark>
</revision>
<revision>
<version>0.0.1</version>
<date>2013-05-24</date>
<initials>sdm</initials>
<remark><p>First draft.</p></remark>
</revision>
</header>
<section1 topic='Introduction' anchor='intro'>
<p>The concept of delivery and read receipts has been popularised by other messaging services such as iMessage, Google Hangouts and Blackberry Messenger.
These services provide a visual indication of when a message has been delivered to any of the recipients resources and (optionally) when it has been read.
These visual indications (referred to herein as "Chat Markers") are synced between all of the sender's
and recipient's resources automatically so the state of a chat is always consistent and
if one of the resources is not connected it can fetch Chat Marker from the Message Archive.</p>
<p>&xep0184; currently provides delivery receipts on a per message basis,
but it does not provide any mechanism for the user to indicate that they have read or acknowledged the message.
As delivery receipts are sent on a per message basis it would require multiple messages to "sync" up delivery receipts between resources.</p>
<p>Moreover by using &xep0085; you could infer that a message has been read if the recipient becomes active at any point after the message has been delivered, but again it would require multiple messages to "sync" up chat states between resources.</p>
<p>This XEP outlines an efficient message based protocol to provide this functionally using Chat Markers.</p>
</section1>
<section1 topic='Terminology' anchor='terms'>
<p>The acronym "MUC" refers to Multi User Chat as defined in &xep0045;.</p>
<p>The term "active chat" refers to a chat that a user is currently active in.</p>
</section1>
<section1 topic='Requirements' anchor='reqs'>
<p>This document addresses the following requirements:</p>
<ol>
<li>Enable a recipient to mark up to an individual message in a chat as received.</li>
<li>Enable a recipient to mark up to an individual message in a chat as read.</li>
<li>Enable a recipient to mark up to an individual message in a chat as acknowledged.</li>
<li>Enable a entity to update and query Chat Markers regardless of the other entities presence.</li>
</ol>
</section1>
<section1 topic='Chat Markers' anchor='chat-markers'>
<p>Chat Markers use a dedicated protocol extension qualified by the 'urn:xmpp:chat-markers:tmp' namespace.</p>
<p>There are three allowable elements in this namespace (in order of significance):</p>
<ul>
<li>received -- the message has been delivered.</li>
<li>read -- the message the user has viewed it in a active chat.</li>
<li>acknowledged -- the message has been acknowledged by some user interaction e.g. pressing a button.</li>
</ul>
<p>The Chat Marker MAY have a 'stamp' which is the timestamp of the Chat Marker.</p>
<p>The Chat Marker MUST have an 'id' which is the 'id' of the message being marked.</p>
<p>A Chat Marker Indicates that all messages up to and including that message 'id' have been marked.</p>
<example caption='Example Chat Marker'>
<![CDATA[
<message
from='[email protected]/throne'
id='bi29sg183b4v'
to='[email protected]/westminster'>
<received xmlns='urn:xmpp:chat-markers:tmp'
id='richard2-4.1.247'
stamp='2002-10-13T23:58:43Z'/>
</message>
]]>
</example>
</section1>
<section1 topic='When to send Chat Markers' anchor='when'>
<section2 topic='Bare JID' anchor='when-bare'>
<p>If the sender knows only the recipient's bare JID, it cannot cannot determine (via &xep0030; or &xep0115;) whether the intended recipient supports the Chat Markers protocol. In this case, the sender MAY send a chat marker.</p>
</section2>
<section2 topic='Full JID' anchor='when-full'>
<p>If the sender knows a full JID for the recipient (e.g., via presence), it SHOULD attempt to determine (via service disco or entity capabilities) whether the client at that full JID supports the Chat Markers protocol before attempting to send a Chat Marker.</p>
<p>If the sender determines that the recipient's client does not support the Chat Markers protocol then it SHOULD NOT send Chat Markers.</p>
<p>If the sender determines that the recipient's client supports the Chat Markers protocol then it MAY send a Chat Marker to that full JID.</p>
</section2>
<section2 topic='Ack Messages' anchor='when-ack'>
<p>To prevent looping, an entity MUST NOT send a Chat Maker to mark up to a Chat Marker.</p>
</section2>
</section1>
<section1 topic='Requirements' anchor='requirements'>
<p>Clients SHOULD use &xep0280; to support multiple online resources.</p>
<p>Clients SHOULD use &xep0136; or &xep0313; to support offline updating of Chat Markers. Chat Markers SHOULD be archived, so they can be updated regardless of the recipients current presence.</p>
<p>All messages MUST have an 'id' to use Chat Markers.</p>
</section1>
<section1 topic='Business Rules' anchor='rules'>
<section2 topic='Optimizations' anchor='chatstates'>
<p>Less Significant Chat Markers should only be sent if they are later than the more significant Chat Marker i.e. if a Message has been marked as read,
a delivered Chat Marker should only be sent if it has a later timestamp than the read Chat Marker.</p>
</section2>
<section2 topic='Interaction with Delivery Receipts' anchor='receipts'>
<p>If a delivery receipt is sent to a resource that advertises support,
then a delivery receipt MUST be sent if it is satisfies &xep0184;.</p>
<p>If a resource advertises both Chat Maker and Delivery Receipt support it
SHOULD NOT request a receipt.</p>
</section2>
<section2 topic='Interaction with Chat States' anchor='chatstates'>
<p>Chat Markers can be used alongside Chat States.</p>
</section2>
<section2 topic='Auto Mark' anchor='auto-responses'>
<p>Clients MUST NEVER mark a message as acknowledged without any user interaction.</p>
</section2>
<section2 topic='Mobile Considerations' anchor='mobile'>
<p>Since mobile devices often must pay for network traffic based on usage, you may wish not to enable Chat Marker Subscription.</p>
</section2>
</section1>
<section1 topic='Determining support' anchor='disco'>
<p>If an entity supports the Chat Markers protocol, it MUST report that by including a &xep0030;
feature of "urn:xmpp:chat-markers:tmp" in response to disco#info requests:</p>
<example caption='Client queries for server features'><![CDATA[
<iq type='get' id='disco1' to='capulet.lit' from='[email protected]/balcony'>
<query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>
]]></example>
<example caption='Server responds with features'><![CDATA[
<iq type='result' id='disco1' from='capulet.lit' to='[email protected]/balcony'>
<query xmlns='http://jabber.org/protocol/disco#info'>
...
<feature var='urn:xmpp:chat-markers:tmp'/>
...
</query>
</iq>
]]></example>
<p>Support can also be determined via &xep0115;, a.k.a. "caps".</p>
</section1>
<section1 topic='Security Considerations' anchor='security'>
<p>A user may not wish to disclose that they have received, read or acknowledge a message.</p>
<p>It is possible for a recipient to leak its presence when updating Chat Markers;
therefore, a recipient SHOULD NOT return message delivery receipts to senders who are not otherwise authorized to view its presence.</p>
</section1>
<section1 topic='IANA Considerations' anchor='iana'>
<p>This document requires no interaction with the Internet Assigned Numbers Authority (IANA).</p>
</section1>
<section1 topic='XMPP Registrar Considerations' anchor='registrar'>
<section2 topic='Protocol Namespaces' anchor='registrar-ns'>
<p>This specification defines the following XML namespace:</p>
<ul>
<li>'urn:xmpp:chat-markers:tmp'</li>
</ul>
</section2>
</section1>
<section1 topic='XML Schema' anchor='schema'>
<code><![CDATA[
<?xml version='1.0' encoding='UTF-8'?>
<xs:schema attributeFormDefault="unqualified"
elementFormDefault="qualified"
targetNamespace="urn:xmpp:chat-markers:tmp"
xmlns:xs="http://www.w3.org/2001/XMLSchema";>
<xs:annotation>
<xs:documentation>
The protocol documented by this schema is defined in
XEP-XXXX: http://xmpp.org/extensions/xep-XXXX.html
</xs:documentation>
</xs:annotation>
<xs:element name="received">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute type="xs:string" name="id"/>
<xs:attribute type="xs:dateTime" name="stamp"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
<xs:element name="read">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute type="xs:string" name="id"/>
<xs:attribute type="xs:dateTime" name="stamp"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
<xs:element name="acknowledged">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute type="xs:string" name="id"/>
<xs:attribute type="xs:dateTime" name="stamp"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
</xs:schema>
]]></code>
</section1>
</xep>
