Re: [Standards] SIMS issues

[Daniel Gultsch]

Am Sa., 7. Sept. 2019 um 12:54 Uhr schrieb Sergey Ilinykh :

> Regarding references. Andrew Nenakhov gave me a good idea what to reference 
> to when want to just an audio message.
> It's a text for clients which do not support SIMS. For example it can be an 
> HTTP link to the audio record or something
> like "Go and install another client to listen this" in case there is only s5b 
> stream available.
> So when SIMS-capable client renders this, it removes the referenced text. As 
> far as I remember Andrew even added
> a special attribute the references where they have to be removed or not. So I 
> believe his ideas could be applied here.


Without getting into the pros and cons of fallback messages
(especially the ones that tell you to download another client) the
SIMS XEP should not depend on them.

Even if  is a direct child of  you can still
include a body.

cheers
Daniel
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Sergey Ilinykh]

Thanks for the feedback Jonas.

Honestly I already thought about this after implementation and I agree with
you.
Removing "coding" in favor of always base64(u8) would make things way
easier and still sufficient.
And I'm fine with 

---

Regarding references. Andrew Nenakhov gave me a good idea what to reference
to when want to just an audio message.
It's a text for clients which do not support SIMS. For example it can be an
HTTP link to the audio record or something
like "Go and install another client to listen this" in case there is only
s5b stream available.
So when SIMS-capable client renders this, it removes the referenced text.
As far as I remember Andrew even added
a special attribute the references where they have to be removed or not. So
I believe his ideas could be applied here.

Best Regards,
Sergey


сб, 7 сент. 2019 г. в 14:29, Jonas Schäfer :

> On Mittwoch, 19. Juni 2019 19:46:02 CEST Sergey Ilinykh wrote:
> > Hi
> >
> > I mostly implemented SIMS in Psi and see next problems
> >
> > 1) The requirement for top-level reference element looks strange.
> >In Most of the case when I want to share something, I don't want to
> > refer to anything.
> >If I really want to have a reference I would add it inside of
> > .
> >
> > 2) Top-level reference doesn't state what has to be in "uri" which is
> > required
> >
> > 3) Seems like only one  element is allowed while it's
> quite
> > common to share multiple files at once with just one description. Sending
> > each shared file via separate stanza doesn't look to be a good option
> since
> > servers often limit sending rate and separate stanzas somehow corrupt
> > logical scope.
> >
> > 4) I want to use SIMS for voice messages but there is no any metadata for
> > audio.
> >I need at least duration and amplitude diagram there. Something like
> > following would be really
> >nice to have:
> >
> >   tune data
> here
> >coding="u8">0,3,135,237,210,195,243,137,...,4,4,1
> > 
>
> Nitpick: If you are going to implement this and intend on standardising
> it,
> may I suggest to use base64-encoded u8 bytes instead of the proposed
> string
> syntax? It will be smaller by approximately 63% for uniformly distributed
> amplitudes and by approximately 33% for all-zeros (worst-case). (I am also
> fairly certain that for the preview, u8 is fully sufficient.) (base64
> starts
> to be consistently smaller (in the all-zeros worst-case) once you have
> more
> than 5 samples.)
>
> Another Nitpick: From your description, it is more like an 
> preview instead of a  (which implies something like a Fourier
> transform or another way to obtain per-frequency values).
>
> kind regards,
> Jonas___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___
>
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Jonas Schäfer]

On Mittwoch, 19. Juni 2019 19:46:02 CEST Sergey Ilinykh wrote:
> Hi
> 
> I mostly implemented SIMS in Psi and see next problems
> 
> 1) The requirement for top-level reference element looks strange.
>In Most of the case when I want to share something, I don't want to
> refer to anything.
>If I really want to have a reference I would add it inside of
> .
> 
> 2) Top-level reference doesn't state what has to be in "uri" which is
> required
> 
> 3) Seems like only one  element is allowed while it's quite
> common to share multiple files at once with just one description. Sending
> each shared file via separate stanza doesn't look to be a good option since
> servers often limit sending rate and separate stanzas somehow corrupt
> logical scope.
> 
> 4) I want to use SIMS for voice messages but there is no any metadata for
> audio.
>I need at least duration and amplitude diagram there. Something like
> following would be really
>nice to have:
>
>   tune data here
>   0,3,135,237,210,195,243,137,...,4,4,1
> 

Nitpick: If you are going to implement this and intend on standardising it, 
may I suggest to use base64-encoded u8 bytes instead of the proposed string 
syntax? It will be smaller by approximately 63% for uniformly distributed 
amplitudes and by approximately 33% for all-zeros (worst-case). (I am also 
fairly certain that for the preview, u8 is fully sufficient.) (base64 starts 
to be consistently smaller (in the all-zeros worst-case) once you have more 
than 5 samples.)

Another Nitpick: From your description, it is more like an  
preview instead of a  (which implies something like a Fourier 
transform or another way to obtain per-frequency values).

kind regards,
Jonas

signature.asc
Description: This is a digitally signed message part.
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Jonas Schäfer]

On Samstag, 7. September 2019 13:08:39 CEST Daniel Gultsch wrote:
> Am Sa., 7. Sept. 2019 um 10:51 Uhr schrieb Marvin W :
> > On 07.09.19 11:55, Daniel Gultsch wrote:
> > > To cover the use case Sergey is talking about here (stand alone voice
> > > message, stand alone image, stand alone file) without any description
> > > wouldn’t it make more sense to be able to put a sims as a direct child
> > > of a message (and essentially the only child ( + origin-id, 184
> > > request)?
> > > 
> > > I mean it is nice and all that it can _also_ be used within a
> > > reference but i don’t understand the hard dependency (that the XEP in
> > > it's current state implies) on references.
> > 
> > I actually think the current way of SIMS is even semantically wrong,
> > because it puts  into . However the message
> > usually does not reference a media sharing, it *is* the media sharing
> > that maybe has a comment attached (like in the examples in XEP-0385
> > itself).
> > 
> > Therefor even the case of referencing might even be better realized
> > using two messages where one is the media sharing and one is a message
> > with a reference to that media sharing:
> > 
> > 
> > 
> >   [...]
> > 
> > 
> > 
> > and
> > 
> > 
> > 
> >   [...]
> >   
> > 
> > .
> > 
> > This has the positive side-effect that SIMS can have a proper fallback
> > body:
> > 
> > 
> > 
> > https://download.montague.lit/4a771ac1-f0b2-4a4a-9700-f2a26fa2bb67/s
> > ummit.jpg (summit.jpg, 3032449 byte)
> > 
> >   [...]
> > 
> > 
> > 
> > > As I see it a lot of developers are currently in the market for a
> > > replacement for the current usage of the x-oob tag and are not
> > > necessarily looking for anything more complicated like references
> > > (which involve having to implement an URI parser among other things)
> > > Therefor I see a danger in essentially forcing those people to
> > > implement half-assed references that are only an empty, mostly unused
> > > reference wrapper around the sims element. This has the potential of
> > > getting us into compatibility trouble if later we will actually use
> > > references for something more than this.
> > 
> > The clients would still need to implement XEP-0372 (or parts thereof)
> > for the s inside the 's . I
> > actually like using the concept of references here, because it seems to
> > be exactly what XEP-0372 is for: referencing resources, both external
> > (http url) and internal (jingle).
> > 
> > As we now do have XEP-0420 (even though not realized yet) and XEP-0373,
> > we don't need to rely on transporting the download link and encryption
> > information in the only encrypted part of messages, the body (like in
> > OMEMO Media sharing protoxep), so I feel SIMS might become more
> > widespread soonish.
> > 
> > In the context of encryption, I'd also like to express my hope that we
> > also make up some concept for sharing encrypted files with SIMS, so that
> > we can get rid of `aesgcm://` links longterm and don't need to use them
> > inside ...
> 
> Well even ignoring encryption and the less than ideal aesgcm URI
> scheme for a moment here I'm afraid that people will very soon
> implement SIMS because they are looking for a modern alternative to
> out of band data.
> 
> However if I just want to send you a voice message I simply want to
> send you this:
> 
> 
> …
> 
> 
> Without any body or anything else.
> I don’t understand (and consider it harmful) to needlessly wrap this
> into a reference that is not referencing anything. Yes we established
> that start and end can be empty but how do you suppose the message
> stanza is going to look like?
> 
> 
> 
> …
> 
> 
> 
> 
> Then what is this reference referring to? I'm afraid that if we don’t
> fix this know people will implement the later even though it makes no
> sense at all.

+1.

This also neatly solves the issue of packing multiple media sharing elements 
in a single message.

If we want to use References at some point to establish a relationship between 
 elements and parts of the text, that could be solved by a 
proper URI scheme or child element in the References tag which then 
references(!) the respective  element.

kind regards,
Jonas

signature.asc
Description: This is a digitally signed message part.
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Daniel Gultsch]

Am Sa., 7. Sept. 2019 um 10:51 Uhr schrieb Marvin W :
>
> On 07.09.19 11:55, Daniel Gultsch wrote:
> > To cover the use case Sergey is talking about here (stand alone voice
> > message, stand alone image, stand alone file) without any description
> > wouldn’t it make more sense to be able to put a sims as a direct child
> > of a message (and essentially the only child ( + origin-id, 184
> > request)?
> >
> > I mean it is nice and all that it can _also_ be used within a
> > reference but i don’t understand the hard dependency (that the XEP in
> > it's current state implies) on references.
>
> I actually think the current way of SIMS is even semantically wrong,
> because it puts  into . However the message
> usually does not reference a media sharing, it *is* the media sharing
> that maybe has a comment attached (like in the examples in XEP-0385 itself).
>
> Therefor even the case of referencing might even be better realized
> using two messages where one is the media sharing and one is a message
> with a reference to that media sharing:
>
> 
>   [...]
> 
>
> and
>
> 
>   [...]
>   
> .
>
> This has the positive side-effect that SIMS can have a proper fallback body:
>
> 
>
> https://download.montague.lit/4a771ac1-f0b2-4a4a-9700-f2a26fa2bb67/summit.jpg
> (summit.jpg, 3032449 byte)
>   [...]
> 
>
>
> > As I see it a lot of developers are currently in the market for a
> > replacement for the current usage of the x-oob tag and are not
> > necessarily looking for anything more complicated like references
> > (which involve having to implement an URI parser among other things)
> > Therefor I see a danger in essentially forcing those people to
> > implement half-assed references that are only an empty, mostly unused
> > reference wrapper around the sims element. This has the potential of
> > getting us into compatibility trouble if later we will actually use
> > references for something more than this.
> The clients would still need to implement XEP-0372 (or parts thereof)
> for the s inside the 's . I
> actually like using the concept of references here, because it seems to
> be exactly what XEP-0372 is for: referencing resources, both external
> (http url) and internal (jingle).
>
> As we now do have XEP-0420 (even though not realized yet) and XEP-0373,
> we don't need to rely on transporting the download link and encryption
> information in the only encrypted part of messages, the body (like in
> OMEMO Media sharing protoxep), so I feel SIMS might become more
> widespread soonish.
>
> In the context of encryption, I'd also like to express my hope that we
> also make up some concept for sharing encrypted files with SIMS, so that
> we can get rid of `aesgcm://` links longterm and don't need to use them
> inside ...


Well even ignoring encryption and the less than ideal aesgcm URI
scheme for a moment here I'm afraid that people will very soon
implement SIMS because they are looking for a modern alternative to
out of band data.

However if I just want to send you a voice message I simply want to
send you this:


…


Without any body or anything else.
I don’t understand (and consider it harmful) to needlessly wrap this
into a reference that is not referencing anything. Yes we established
that start and end can be empty but how do you suppose the message
stanza is going to look like?



…




Then what is this reference referring to? I'm afraid that if we don’t
fix this know people will implement the later even though it makes no
sense at all.

cheers
Daniel
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Marvin W]

On 07.09.19 11:55, Daniel Gultsch wrote:
> To cover the use case Sergey is talking about here (stand alone voice
> message, stand alone image, stand alone file) without any description
> wouldn’t it make more sense to be able to put a sims as a direct child
> of a message (and essentially the only child ( + origin-id, 184
> request)?
> 
> I mean it is nice and all that it can _also_ be used within a
> reference but i don’t understand the hard dependency (that the XEP in
> it's current state implies) on references.

I actually think the current way of SIMS is even semantically wrong,
because it puts  into . However the message
usually does not reference a media sharing, it *is* the media sharing
that maybe has a comment attached (like in the examples in XEP-0385 itself).

Therefor even the case of referencing might even be better realized
using two messages where one is the media sharing and one is a message
with a reference to that media sharing:


  [...]


and


  [...]
  
.

This has the positive side-effect that SIMS can have a proper fallback body:



https://download.montague.lit/4a771ac1-f0b2-4a4a-9700-f2a26fa2bb67/summit.jpg
(summit.jpg, 3032449 byte)
  [...]



> As I see it a lot of developers are currently in the market for a
> replacement for the current usage of the x-oob tag and are not
> necessarily looking for anything more complicated like references
> (which involve having to implement an URI parser among other things)
> Therefor I see a danger in essentially forcing those people to
> implement half-assed references that are only an empty, mostly unused
> reference wrapper around the sims element. This has the potential of
> getting us into compatibility trouble if later we will actually use
> references for something more than this.
The clients would still need to implement XEP-0372 (or parts thereof)
for the s inside the 's . I
actually like using the concept of references here, because it seems to
be exactly what XEP-0372 is for: referencing resources, both external
(http url) and internal (jingle).

As we now do have XEP-0420 (even though not realized yet) and XEP-0373,
we don't need to rely on transporting the download link and encryption
information in the only encrypted part of messages, the body (like in
OMEMO Media sharing protoxep), so I feel SIMS might become more
widespread soonish.

In the context of encryption, I'd also like to express my hope that we
also make up some concept for sharing encrypted files with SIMS, so that
we can get rid of `aesgcm://` links longterm and don't need to use them
inside ...

Marvin
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Daniel Gultsch]

Am Mi., 19. Juni 2019 um 19:07 Uhr schrieb Sergey Ilinykh :

> Then yes the xep has to be updated with use-cases. Since for now only
> internal  elements looks usable and it's hard to imagine any
> good UX for the top-level one.

To cover the use case Sergey is talking about here (stand alone voice
message, stand alone image, stand alone file) without any description
wouldn’t it make more sense to be able to put a sims as a direct child
of a message (and essentially the only child ( + origin-id, 184
request)?

I mean it is nice and all that it can _also_ be used within a
reference but i don’t understand the hard dependency (that the XEP in
it's current state implies) on references.

As I see it a lot of developers are currently in the market for a
replacement for the current usage of the x-oob tag and are not
necessarily looking for anything more complicated like references
(which involve having to implement an URI parser among other things)
Therefor I see a danger in essentially forcing those people to
implement half-assed references that are only an empty, mostly unused
reference wrapper around the sims element. This has the potential of
getting us into compatibility trouble if later we will actually use
references for something more than this.
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Sergey Ilinykh]

ср, 19 июн. 2019 г. в 21:33, Ralph Meijer :

> On 19/06/2019 19.46, Sergey Ilinykh wrote:
> > Hi
> >
> > I mostly implemented SIMS in Psi and see next problems
> >
> > 1) The requirement for top-level reference element looks strange.
> > In Most of the case when I want to share something, I don't want to
> > refer to anything.
> > If I really want to have a reference I would add it inside of
> > .
>
> The idea here is very similar to Twitter Entities [1,2]. The primary
> advantage of using References is that you can provide a fallback to
> refer to a location where the media can be found, in case the client
> doesn't support certain media descriptors. I don't think the current
> examples show this very well, but instead of the "view" being referenced
> in the example, you could have a URL to access the picture in a browser.
>

> [1]
> <
> https://developer.twitter.com/en/docs/tweets/data-dictionary/overview/entities-object
> >
> [2]
> 
> 
> Best Regards,
> Sergey
> ta-dictionary/overview/extended-entities-object
> 
> >
>
> Also note that you don't have to use `start` and `end`, if you don't
> want to point to the plain text.
>
>
Then yes the xep has to be updated with use-cases. Since for now only
internal  elements looks usable and it's hard to imagine any
good UX for the top-level one.


>
> > 2) Top-level reference doesn't state what has to be in "uri" which is
> > required
>
> I agree that References still needs a lot of work. However, beyond the
> value needing to be a valid URI, I think everything works. How I used
> it, was providing a URI to retrieve the resource over HTTP. Do you have
> any particular concerns here?
>
>
I rather see it like "most-available-uri". So an application should
prioritize
the sources and select one with highest availibility for top-level
reference.


>
> > 3) Seems like only one  element is allowed while it's
> > quite common to share multiple files at once with just one description.
> > Sending each shared file via separate stanza doesn't look to be a good
> > option since servers often limit sending rate and separate stanzas
> > somehow corrupt logical scope.
>
> But you *can* use multiple References. Would that work for you?
>
>
Can I? It has to be documented.


>
> > 4) I want to use SIMS for voice messages but there is no any metadata
> > for audio.
> > I need at least duration and amplitude diagram there. Something like
> > following would be really
> > nice to have:
> > 
> > tune data here
> > > coding="u8">0,3,135,237,210,195,243,137,...,4,4,1
> > 
>
> That sounds very interesting. I don't remember anyone suggesting a
> metadata format for this before and I didn't get to specify this in my
> previous project which would have eventually needed a format for voice
> messages. So what you could do is first define your own namespace and
> experiment with what works for your use case, and then eventually maybe
> submit it as a new XEP.
>
> Also, I support this element would go inside the  wrapper?
>
>
Correct. I need it inside the . Probably I'll implement it like in
my example.

Thanks,
Sergey
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] SIMS issues

[Ralph Meijer]

On 19/06/2019 19.46, Sergey Ilinykh wrote:

Hi

I mostly implemented SIMS in Psi and see next problems

1) The requirement for top-level reference element looks strange.
    In Most of the case when I want to share something, I don't want to 
refer to anything.
    If I really want to have a reference I would add it inside of 
.


The idea here is very similar to Twitter Entities [1,2]. The primary 
advantage of using References is that you can provide a fallback to 
refer to a location where the media can be found, in case the client 
doesn't support certain media descriptors. I don't think the current 
examples show this very well, but instead of the "view" being referenced 
in the example, you could have a URL to access the picture in a browser.


[1] 

[2] 



Also note that you don't have to use `start` and `end`, if you don't 
want to point to the plain text.



2) Top-level reference doesn't state what has to be in "uri" which is 
required


I agree that References still needs a lot of work. However, beyond the 
value needing to be a valid URI, I think everything works. How I used 
it, was providing a URI to retrieve the resource over HTTP. Do you have 
any particular concerns here?



3) Seems like only one  element is allowed while it's 
quite common to share multiple files at once with just one description. 
Sending each shared file via separate stanza doesn't look to be a good 
option since servers often limit sending rate and separate stanzas 
somehow corrupt logical scope.


But you *can* use multiple References. Would that work for you?


4) I want to use SIMS for voice messages but there is no any metadata 
for audio.
    I need at least duration and amplitude diagram there. Something like 
following would be really

    nice to have:
    
tune data here
   coding="u8">0,3,135,237,210,195,243,137,...,4,4,1

    


That sounds very interesting. I don't remember anyone suggesting a 
metadata format for this before and I didn't get to specify this in my 
previous project which would have eventually needed a format for voice 
messages. So what you could do is first define your own namespace and 
experiment with what works for your use case, and then eventually maybe 
submit it as a new XEP.


Also, I support this element would go inside the  wrapper?

--
ralphm
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

[Standards] SIMS issues

[Sergey Ilinykh]

Hi

I mostly implemented SIMS in Psi and see next problems

1) The requirement for top-level reference element looks strange.
   In Most of the case when I want to share something, I don't want to
refer to anything.
   If I really want to have a reference I would add it inside of
.

2) Top-level reference doesn't state what has to be in "uri" which is
required

3) Seems like only one  element is allowed while it's quite
common to share multiple files at once with just one description. Sending
each shared file via separate stanza doesn't look to be a good option since
servers often limit sending rate and separate stanzas somehow corrupt
logical scope.

4) I want to use SIMS for voice messages but there is no any metadata for
audio.
   I need at least duration and amplitude diagram there. Something like
following would be really
   nice to have:
   
  tune data here
  0,3,135,237,210,195,243,137,...,4,4,1
   


Best Regards,
Sergey
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___