On unread markers, having re-read the sent message, I think I wasn't clear enough. What was meant is that the last read message ID will likely point to a message that we don't yet have on device, and thus will be unable to calculate, how many messages we missed.
пн, 3 июн. 2019 г. в 19:26, Ненахов Андрей <[email protected]>: > Ok, it took me some time to get to this. I think some preamble is > necessary, with explanation of our motives. *Those who're just interested > in stanza format can skip to big TL;DR header below. * > > The whole idea, and even utter necessity of such mechanics came to us when > we were preparing to make an XMPP client that is capable to work on iOS. As > you all know, basic XMPP works well enough when there is a persistent > connection with server. This persistent connection can be more or less > maintained on most modern OSs, even on Android (though it is more and more > difficult, later versions of Android really fight against any persistent > processes). But with iOS, there is no way other than to work fully offline, > relying on push notifications to wake up. > > Not only that, on iOS, an app has less than 30 seconds before it is shut > down, or brought into foreground by user. Thus, an iOS app has to quickly > connect, fetch all the necessary data, present, if necessary, all > notifications to a user, and prepare to go offline. > > The most direct approach to catch up what was missed is with offline > messages. It works more or less OK (more on that later) if a user is active > on a single device, but starts to fail utterly if user is trying to use > several devices simultaneously. Second approach is to fetch all messages > from an archive, between now and the latest message received from the > server. This *can* work more or less OK, if periods of disconnection are > relatively brief. (btw this is the method employed in current builds for > Xabber for iOS). However, there are scenarios when such approach will > clearly fail. Most of them revolve around device being offline for a > significant amount of time and a user being active on another device, > sending and receiving multiple messages in one conversation, 'burying' > incoming unread message from another conversation deeper than query to > message archive. > > To effectively tackle this problem we came up with an approach, > essentially orthogonal to what was tried with XMPP before: *instead of an > attempt to re-establish a stream, catching up on what was missed, we're > trying to receive a slice of a current account state*. > > This, naturally, leads to ideas about 'inbox' that are very alike to > Dave's proposal. True, with regular messages it is enough for us to build a > list of recent conversations and some delivered/read marker IDs. However, > we do not agree that unread message counter is not needed: if a client is > doing a cold sync, it'll only have the most recent message in a > conversation, and won't know how many are there until it loads the whole > history up to that point. As telegram channels show, there are often many > thousands of unread messages in some channels, and it is not a good idea to > make clients load all those messages only to present user a 4-digit number. > > There are two more cases that we think should be accounted for: VoIP calls > and Message Editing/Deleting/Retracting. > > First, in VoIP. We've stumbled in this just recently, when it became clear > that if a user is called via XEP-0353, and a message with session > initiation ends up in an archive, it can be quickly followed by subsequent > messages, even from a same contact, which will inevitably 'bury' call in > some unfetched unread messages and recipient device would never know that > there was a call (or that there is still a call in progress!) > > Second, Message Editing (as we call it for now cause we didn't yet come up > with a proper name, thought we have a pretty nicely working > implementation). If you have 7 unread messages, 3 or them can very well be > edits to your previously received messages, and you'll never know that. > > One more thing to consider: we don't think this will work well as a roster > extension. Users generally want to be able to access chat history even with > those who are already deleted from their rosters, and maybe even want to > have separate chat threads with single contact. Great example for this and > most obvious use is separating chats with e2e and without them into > separate chats, like Telegram or WhatsApp do. So, instead of roster we came > up with entity we call 'conversation', and last message, unread counter, > message edits and VoIP calls are property of that entity. > > To make this work, a server intercepts all passing stanzas, and creates a > slice of current user account's state, based on messages, chat markers, > voip session messages, retract/edit messages. > Reconnecting client tells server the timestamp of the last message > received from the server (in our implementation all stanzas received from > the server are timestamped by the server, and clients rely on server time > to be correct for the needs of synchronization and message ordering), and > server responds with a list of conversations that was updated since that > time. > > TL;DR > Client initiates synchronization with sync request: > > > <iq type='get' id=’id2'> > <query xmlns='http://xabber.com/protocol/synchronization' > stamp=’1556111424456379’/> > </iq> > > Server responds with list of conversations updated since > '1556111424456379': > > <iq type=’result’ to=’[email protected]/Xabber-web’ from=’ > redsolution.com’ id=’id1’> > <synchronization xmlns=’http://xabber.com/protocol/synchronization’ > stamp=’1556111424456980’> > <conversation jid=’[email protected]’ > thread=’1asd123sd’ stamp=’1556111424456980’> > <retract version='3'> > <unread count=’4’ after=’andrew_id_211’ /> > <displayed id=’andrey_id_210’ /> > <delivered id=’andrey_id_215’ /> > <call> > <propose xmlns='urn:xmpp:jingle-message:0' > id='a73sjjvkla37jfea'> > <description xmlns='urn:xmpp:jingle:apps:rtp:1' > media='audio'/> > </propose> > </call> > <last-message> > <message from=’[email protected]/Xabber-web’ > to=’[email protected]/Xabber-web’ id=’andrew_id_214’> > <body>Hi!</body> > <stanza-id id=’id342’ by=’ > [email protected]’ /> > </message> > </last-message> > </conversation> > <set xmlns='http://jabber.org/protocol/rsm'> > <first index='0'>1556111424456980</first> > <last>1556111424456980</last> > <count>1</count> > </set> > </synchronization> > </iq> > > thread - name of a thread, or, 'conversation'. Same conversation should > have same thread, and default is nil, like, general converstion with a > person > displayed - AKA 'read' by remote chat partner > deliverred - received by remote partner's client > unread ... after - refers to an ID of a message of a last message read by > user > call - if there is an active call in progress, we pass all the propose > stanza, so client can immediately pick up the phone. > retract - versions of edits in this convesation > > > > In case of a cold start (like, first connection with new client), client > just asks for a synchoronization without timestamp, and receives full list > of conversations in return. > > > > > > > ср, 29 мая 2019 г. в 19:28, Dave Cridland <[email protected]>: > >> >> >> On Wed, 29 May 2019 at 12:27, Ненахов Андрей < >> [email protected]> wrote: >> >>> We have this (not exactly this, but for the very same purpose) mostly >>> implemented and already working, would be happy to share the results with >>> everyone. Currently, it's implemented on a server, client support (in Web >>> version of Xabber) will arrive in a week or two. We also plan to release an >>> open-source server (ejabberd fork) that will support this. >>> >>> Problem is, we definitely lack skills putting this as a 'XEP' formatted >>> thing with proper description, and, what's worse, current documentation is >>> mostly in Russian, but XMLs are in english and if someone would volunteer >>> help us putting it in a XEP-like way, I'll muster myself to translate >>> crucial bits into English. >>> >>> >> Right - I knew you had this but had forgotten. It'd be great to collate >> all these ideas and pick the best bits from them all. >> >> Just a high-level technical sketch would be really useful. >> >> >>> ср, 29 мая 2019 г. в 16:12, Dave Cridland <[email protected]>: >>> >>>> Having spent a while playing with - gasp! - non-XMPP based chat >>>> systems, I'm quite taken with the notion that some kind of Inbox might be >>>> rather useful to us. >>>> >>>> Currently, there is Erlang Solutions (ESL)'s Inbox, which is >>>> essentially a duplication of MAM with some chat state tracking. It's more >>>> than just that, of course, but the essential concept I see is that it's a >>>> different, but largely equivalent interface to MAM, with the concept of an >>>> unread counter added. >>>> >>>> Instagram, on the other hand, has no roster, as such, and its Inbox >>>> simply lists (recent) conversations, in much the same way as a client might >>>> display them. Each record contains the conversation's participants, and the >>>> most recent message. Things like presence subscription, in the Instagram >>>> model, are simply the open conversations. >>>> >>>> We do have a roster, of course, and we're putting more things into it - >>>> MIX channels, MUC Light in ESL's case, and so on. >>>> >>>> This makes me wonder if the right way to design an Inbox is actually to >>>> enhance our Roster with MAM and state awareness, and make it the >>>> conversation information hub for IM. >>>> >>>> Let's suppose that the nature of what is "unread" is equivalent across >>>> all clients of a particular user, to begin with. >>>> >>>> If a client requests the inbox "since" a particular point, it would >>>> then receive a series of records: >>>> >>>> First, a set of N records similar to a roster item, containing a jid, >>>> subscription state, the last archive-id received in the conversation, and >>>> the number of unacknowledged (by some definition) messages. Our roster also >>>> includes groups and a name; we could also include the type (MUC, MIX, or a >>>> user), the full message, etc - these are all optimisations. >>>> >>>> We do not, actually, need the numbers of unread messages - a client >>>> seeing that the last archive-id isn't in its cache knows the conversation >>>> has messages that are unread to it, at least. But if we can track message >>>> read state, that's useful for multi-device. >>>> >>>> Finally, an update message to indicate the current point, where things >>>> change. I would use an archive-id again here - even if the thing causing >>>> the update isn't an archived message at all. This allows a client to ask >>>> for the archive either since a particular message, or since an event. >>>> >>>> You'll note I'm not building this directly on PubSub in the XEP-0060 >>>> (or XEP-0163) sense - instead I'm proposing building this on the existing >>>> Roster and MAM. >>>> >>>> So: >>>> >>>> <iq type='get' id='some-id'> >>>> <query xmlns='jabber:iq:roster' ver='last-archive-id'> >>>> <inbox xmlns='urn:xmpp:inbox'><!-- Add enhanced inbox info --> >>>> <messages/><!-- Include the entire last message? --> >>>> <unread/><!-- Include unread count --> >>>> </inbox> >>>> </query> >>>> </iq> >>>> >>>> I'd dearly love to return updates using <message/>, MAM-style, >>>> actually. But let's say we stick with the roster design, the pushes look >>>> like: >>>> >>>> <iq from='[email protected]' id='b2gs90j5' to='[email protected]/home' >>>> type='set'> >>>> <query xmlns='jabber:iq:roster' ver='ver42'> >>>> <item jid='[email protected]' subscription='both'> >>>> <unread count='4'/> >>>> <stanza-id id='ver42' xmlns='...'/> >>>> <message ...> >>>> <body>Yeah, sure - whenever you like.</body> >>>> </message> >>>> </query> >>>> </iq> >>>> >>>> Any message "counts" for updating the roster version, by dint of >>>> unifying the namespace of stanza-id (for archive) and roster version. So >>>> any new message arriving updates the current point of the roster as well, >>>> and any new change in the roster changes the archive pointer similarly. >>>> >>>> Updating the shared unread count could be by Carbonizing the >>>> read-receipts (or similar), and using the stanza-id from that, or by an >>>> explicit roster push. Either's good - I think the roster pushes are more >>>> explicit, which is helpful. >>>> >>>> I'm fully expecting some push-back here. Comments are, of course, >>>> welcome. >>>> >>>> Dave. >>>> _______________________________________________ >>>> Standards mailing list >>>> Info: https://mail.jabber.org/mailman/listinfo/standards >>>> Unsubscribe: [email protected] >>>> _______________________________________________ >>>> >>> >>> >>> -- >>> Andrew Nenakhov >>> CEO, Redsolution, Inc. >>> https://redsolution.com <http://www.redsolution.com> >>> _______________________________________________ >>> Standards mailing list >>> Info: https://mail.jabber.org/mailman/listinfo/standards >>> Unsubscribe: [email protected] >>> _______________________________________________ >>> >> _______________________________________________ >> Standards mailing list >> Info: https://mail.jabber.org/mailman/listinfo/standards >> Unsubscribe: [email protected] >> _______________________________________________ >> > > > -- > Andrew Nenakhov > CEO, Redsolution, Inc. > https://redsolution.com <http://www.redsolution.com> > -- Andrew Nenakhov CEO, Redsolution, Inc. https://redsolution.com <http://www.redsolution.com>
_______________________________________________ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: [email protected] _______________________________________________
