Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Dave Cridland]

On Sat, 31 Dec 2022 at 13:15, Florian Schmaus  wrote:

> On 31.12.22 13:19, Dave Cridland wrote:
> >
> >
> > On Fri, 30 Dec 2022 at 19:40, Florian Schmaus  > > wrote:
> >
> > On 30.12.22 11:05, Dave Cridland wrote:
> >  > On Thu, 22 Dec 2022 at 09:23, Matthew Wild  > 
> >  > >> wrote:
> >  > On Wed, 21 Dec 2022, 15:05 Florian Schmaus,  > 
> >  > >> wrote:
> >  > Zash's proposal is, as far as I understand it, just an
> >  > optimization
> >  > allowing a sending entity to determine if a stanza will
> > hit the
> >  > limit or
> >  > not, without trying to actually send it.
> >   >
> >  > It is and it isn't. Right now, if a stanza is over the limit
> then
> >  > most implementations will close the connection with a stream
> > error.
> >  > Not closing the connection is non-trivial for implementations
> if
> >  > they want to avoid DoS and use a standard parser.
> >  >
> >  > I think this is a good summary of why the  is needed
> > - it
> >  > allows a sending implementation to alter its behaviour for the
> > better.
> >  >
> >  > I don't think there's the same justification for ,
> >  > though, is there? What would a sending implementation do?
> >
> > Uh, there is much value in the information found in .
> > Simply put, it allows the receiving side to delay liveness checks for
> > the connection until it has been idle longer than idle-seconds.
> >
> >
> > Really? So as a client, if a server says that connections can be idle
> > for up to 1800s, you'd not bother checking them for a half hour?
>
> I guess this is up to the implementation to decide.
>
>
Yes, though I wonder if this falls into the category of a little knowledge.


> > I can see why the *server* might adapt its liveness checks to match the
> > client's, but this XEP doesn't cover that.
>
> I assume the XEP has also s2s connections in mind? Whereas I am mostly
> concerned about c2s connections, especially mobile ones.


I suppose so, but the only case where this makes sense (to me, mind, and
what would I know about S2S on flakey networks?) is where you have a bidi
S2S session and both sides can adapt the liveness checks to be the minimum
of either party, only that's what they'd be anyway in effect. But explicit
> implicit.

Also, if you have '198 in effect, do we care about idle anymore?

Dave.
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Florian Schmaus]

On 31.12.22 13:19, Dave Cridland wrote:



On Fri, 30 Dec 2022 at 19:40, Florian Schmaus > wrote:


On 30.12.22 11:05, Dave Cridland wrote:
 > On Thu, 22 Dec 2022 at 09:23, Matthew Wild mailto:mwi...@gmail.com>
 > >> wrote:
 >     On Wed, 21 Dec 2022, 15:05 Florian Schmaus, mailto:f...@geekplace.eu>
 >     >> wrote:
 >         Zash's proposal is, as far as I understand it, just an
 >         optimization
 >         allowing a sending entity to determine if a stanza will
hit the
 >         limit or
 >         not, without trying to actually send it.
  >
 >     It is and it isn't. Right now, if a stanza is over the limit then
 >     most implementations will close the connection with a stream
error.
 >     Not closing the connection is non-trivial for implementations if
 >     they want to avoid DoS and use a standard parser.
 >
 > I think this is a good summary of why the  is needed
- it
 > allows a sending implementation to alter its behaviour for the
better.
 >
 > I don't think there's the same justification for ,
 > though, is there? What would a sending implementation do?

Uh, there is much value in the information found in .
Simply put, it allows the receiving side to delay liveness checks for
the connection until it has been idle longer than idle-seconds.


Really? So as a client, if a server says that connections can be idle 
for up to 1800s, you'd not bother checking them for a half hour?


I guess this is up to the implementation to decide.

I can see why the *server* might adapt its liveness checks to match the 
client's, but this XEP doesn't cover that.


I assume the XEP has also s2s connections in mind? Whereas I am mostly 
concerned about c2s connections, especially mobile ones.


- Flow


OpenPGP_0x8CAC2A9678548E35.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Dave Cridland]

On Fri, 30 Dec 2022 at 19:40, Florian Schmaus  wrote:

> On 30.12.22 11:05, Dave Cridland wrote:
> > On Thu, 22 Dec 2022 at 09:23, Matthew Wild  > > wrote:
> > On Wed, 21 Dec 2022, 15:05 Florian Schmaus,  > > wrote: > Zash's proposal is, as
> far as I understand it, just an
> > optimization
> > allowing a sending entity to determine if a stanza will hit the
> > limit or
> > not, without trying to actually send it.
>  >
> > It is and it isn't. Right now, if a stanza is over the limit then
> > most implementations will close the connection with a stream error.
> > Not closing the connection is non-trivial for implementations if
> > they want to avoid DoS and use a standard parser.
> >
> > I think this is a good summary of why the  is needed - it
> > allows a sending implementation to alter its behaviour for the better.
> >
> > I don't think there's the same justification for ,
> > though, is there? What would a sending implementation do?
>
> Uh, there is much value in the information found in .
> Simply put, it allows the receiving side to delay liveness checks for
> the connection until it has been idle longer than idle-seconds.
>
>
Really? So as a client, if a server says that connections can be idle for
up to 1800s, you'd not bother checking them for a half hour? If a server
sees the connection drop (TCP RST from NAT or something) then the client
won't know about it, so surely both need to perform liveness checks as they
see fit.

I can see why the *server* might adapt its liveness checks to match the
client's, but this XEP doesn't cover that.


> I actually have some very unfinished ProtoXEP from 2017 with similar
> goals (and a bit more [1]):
>
> https://geekplace.eu/xeps/xep-dpmipy/xep-dpmipy.html
>
>
> I think many initial implementations of (bidirectional) XMPP connections
> start with some sort of periodic ping to check if the connection is
> still alive. Then, at some point later some nifty developer recognizes
> that such checks only need to be performed after the connection has been
> idle for a while. Now the developer changes the implementation to check
> every N minutes if the connection has been idle for M minutes, and only
> performs the liveness check if this is the case. This avoid unnecessary
> pings, which is especially important for mobile devices so that they are
> not unnecessary woken up by pings.
>
> Now the question is how to determine M? If the mobile client tells the
> server that it will perform liveness check every 60 minutes, because
> that is when other scheduled actions will take place on the mobile
> device and hence the device won't be in deep sleep anyway, then the
> server could simply set M to 60 minutes [2].
>
> That, at least, was my motivation for said ProtoXEP. But it appears,
> from the description of , that it also shares the
> motivation (but maybe there is more to it?).
>
>
This all makes (a lot of) sense, but isn't possible with the proposal.


> - Flow
>
>
> 1: The idea of my ProtoXEP is that the client tells to the server that
> the server should not ping the client, as the client will ping the
> server after the connection has been idle an client-announced period of
> time. With the goal to reduce unnecessary wakeups of the mobile device
> because of server-to-client pings.
>
> 2: The situation is actually a tiny bit more complex because scheduled
> actions on mobile devices are typically coalesced. But that only means
> that clients have to add the maximum coalescing period to the scheduled
> interval when calculating M.
>

This feels like some guidance would be warranted, akin to (or part of)
XEP-0286.


> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___
>
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Florian Schmaus]

On 30.12.22 11:05, Dave Cridland wrote:
On Thu, 22 Dec 2022 at 09:23, Matthew Wild > wrote:

On Wed, 21 Dec 2022, 15:05 Florian Schmaus, mailto:f...@geekplace.eu>> wrote: > Zash's proposal is, as far as 
I understand it, just an
optimization
allowing a sending entity to determine if a stanza will hit the
limit or
not, without trying to actually send it.

>

It is and it isn't. Right now, if a stanza is over the limit then
most implementations will close the connection with a stream error.
Not closing the connection is non-trivial for implementations if
they want to avoid DoS and use a standard parser.

I think this is a good summary of why the  is needed - it 
allows a sending implementation to alter its behaviour for the better.


I don't think there's the same justification for , 
though, is there? What would a sending implementation do?


Uh, there is much value in the information found in . 
Simply put, it allows the receiving side to delay liveness checks for 
the connection until it has been idle longer than idle-seconds.


I actually have some very unfinished ProtoXEP from 2017 with similar 
goals (and a bit more [1]):


https://geekplace.eu/xeps/xep-dpmipy/xep-dpmipy.html


I think many initial implementations of (bidirectional) XMPP connections 
start with some sort of periodic ping to check if the connection is 
still alive. Then, at some point later some nifty developer recognizes 
that such checks only need to be performed after the connection has been 
idle for a while. Now the developer changes the implementation to check 
every N minutes if the connection has been idle for M minutes, and only 
performs the liveness check if this is the case. This avoid unnecessary 
pings, which is especially important for mobile devices so that they are 
not unnecessary woken up by pings.


Now the question is how to determine M? If the mobile client tells the 
server that it will perform liveness check every 60 minutes, because 
that is when other scheduled actions will take place on the mobile 
device and hence the device won't be in deep sleep anyway, then the 
server could simply set M to 60 minutes [2].


That, at least, was my motivation for said ProtoXEP. But it appears, 
from the description of , that it also shares the 
motivation (but maybe there is more to it?).


- Flow


1: The idea of my ProtoXEP is that the client tells to the server that 
the server should not ping the client, as the client will ping the 
server after the connection has been idle an client-announced period of 
time. With the goal to reduce unnecessary wakeups of the mobile device 
because of server-to-client pings.


2: The situation is actually a tiny bit more complex because scheduled 
actions on mobile devices are typically coalesced. But that only means 
that clients have to add the maximum coalescing period to the scheduled 
interval when calculating M.

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Dave Cridland]

On Thu, 22 Dec 2022 at 09:23, Matthew Wild  wrote:

> On Wed, 21 Dec 2022, 15:05 Florian Schmaus,  wrote:
>
>>
>>  Zash's proposal is, as far as I understand it, just an optimization
>> allowing a sending entity to determine if a stanza will hit the limit or
>> not, without trying to actually send it.
>
>
> It is and it isn't. Right now, if a stanza is over the limit then most
> implementations will close the connection with a stream error. Not closing
> the connection is non-trivial for implementations if they want to avoid DoS
> and use a standard parser.
>

I think this is a good summary of why the  is needed - it
allows a sending implementation to alter its behaviour for the better.

I don't think there's the same justification for , though,
is there? What would a sending implementation do? Defeat the idle check? To
what end? I *thought* we recommended a proper liveness check, so in
principle the idle timeout is a mere implementation detail, and shouldn't
alter behaviour on the other side.

Also, is it worth including a full stream features inside (or after?) bidi?
(I've lost track of how well bidi is implemented, and therefore whether
it's worth doing a version-bump, but this is the first case I'm aware of
where bidi would include a child element, so maybe a version bump here is
easiest?)
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Matthew Wild]

On Wed, 21 Dec 2022, 15:05 Florian Schmaus,  wrote:

>
>  Zash's proposal is, as far as I understand it, just an optimization
> allowing a sending entity to determine if a stanza will hit the limit or
> not, without trying to actually send it.


It is and it isn't. Right now, if a stanza is over the limit then most
implementations will close the connection with a stream error. Not closing
the connection is non-trivial for implementations if they want to avoid DoS
and use a standard parser.

For this reason, we already have in the wild some s2s links being closed
and re-established frequently due to a handful of people with oversized
avatars.

With this XEP, the a server can return a stanza error to the sender and not
break the s2s stream.

Path "MTU" discovery has been discussed in the past. However it has limited
utility in practice for most use cases. The MTU of a MUC could vary every
time someone joins from a new server, for example. I'm not against
something like this (though I don't think caps would be a good place for
it).

You're right that, in general, the safest thing to do is abide by the
minimum limits whenever possible, and handle failure gracefully for
exceptional things.

Regards,
Matthew
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Florian Schmaus]

On 21/12/2022 14.49, Peter Waher wrote:

Hello


This specification defines a way for an XMPP entity to announce the

limits it will enforce for data received on a stream.


URL: https://xmpp.org/extensions/inbox/xep-sla.html 



This is a sorely needed extension. However, the proposal does not solve 
the more general problem of knowing the limitations when you communicate 
with another, only the size and time limits between immediately 
connected entities. A sender (client1) who needs to send something to 
another client (client2) passes one or two brokers (broker[1, broker2]), 
each one in this path might have different size limitations. A 
notification in the stream element does not resolve this more general 
problem. Furthermore, being constantly advertised in every request, 
creates a lot of extra bytes being communicated, with not added value, 
as the information is only needed once per limitation (best case).


Would it not be better with a statement in a service discovery response? 
There, each server/component/client can declare their limitations. Any 
change in limitation would change the hash in entities capabilities, 
which is already sent. So a change would be detected by all 
participants, without the added bytes each time an entity connects. 
Also, a service discovery mechanism, would allow an entity to query all 
participants in a route, to figure out what the limitations are along 
that particular route.


I can see the appeal of entities announcing the stanza size limit as 
part of their disco#info.


However, I wonder if many implementations would really put that 
information to a good use and implement a maximum stanza size discovery 
for a given path and adjust their behavior accordingly.


Really, the only robust way to deal with the stanza size limit is to 
take the lower bound for the maximum stanza size of 10 000 bytes, 
provided by the RFC, into account. Give a "little" room for stanza 
modifications on the path, allowing for additional elements to be added 
on the way, and send only stanzas of say, 4 KiB in size and you should 
be fine.


This, of course, requires that XMPP and its extensions is able to split 
request and result sets, and messages/notifications into smaller chunks. 
RSM certainly is helpful here, but I believe not everything we defined 
allows for a RSM-ish behavior.


For example, what if you discovered that the maximum stanza size of a 
path is M, but the user just typed a message with M+1 bytes? Split the 
user message into two message stanzas? What if the user later wants to 
revert or correct the message? What about reactions on that message?


I think the only reason we did not yet discuss those issues is that the 
maximum stanza size limit of most implementations is rather generous (I 
hear 128 KiB). And we can just live with the inadequateness of XMPP in 
this regard.


Zash's proposal is, as far as I understand it, just an optimization 
allowing a sending entity to determine if a stanza will hit the limit or 
not, without trying to actually send it.


And this goal is achieved with the proposed specification.

- Flow
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Proposed XMPP Extension: Stream Limits Advertisement

[Peter Waher]

Hello

> This specification defines a way for an XMPP entity to announce the
limits it will enforce for data received on a stream.
>
> URL: https://xmpp.org/extensions/inbox/xep-sla.html

This is a sorely needed extension. However, the proposal does not solve the 
more general problem of knowing the limitations when you communicate with 
another, only the size and time limits between immediately connected entities. 
A sender (client1) who needs to send something to another client (client2) 
passes one or two brokers (broker[1, broker2]), each one in this path might 
have different size limitations. A notification in the stream element does not 
resolve this more general problem. Furthermore, being constantly advertised in 
every request, creates a lot of extra bytes being communicated, with not added 
value, as the information is only needed once per limitation (best case).

Would it not be better with a statement in a service discovery response? There, 
each server/component/client can declare their limitations. Any change in 
limitation would change the hash in entities capabilities, which is already 
sent. So a change would be detected by all participants, without the added 
bytes each time an entity connects. Also, a service discovery mechanism, would 
allow an entity to query all participants in a route, to figure out what the 
limitations are along that particular route.

Best regards,
Peter Waher
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

[Standards] Proposed XMPP Extension: Stream Limits Advertisement

[XSF Editor]

The XMPP Extensions Editor has received a proposal for a new XEP.

Title: Stream Limits Advertisement
Abstract:
This specification defines a way for an XMPP entity to announce the
limits it will enforce for data received on a stream.

URL: https://xmpp.org/extensions/inbox/xep-sla.html

The Council will decide in the next two weeks whether to accept this
proposal as an official XEP.
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___