Re: [Standards] Thoughts on XEP-0385 Stateless Inline Media Sharing
On 03.07.21 17:44, Linus Jahn wrote: Hello, I just wanted to note that there's also XEP-0447: Stateless file sharing [1] which has some improvements over SIMS (like multiple files in one message and a little simpler API (no need to use character counting / the References XEP) and later attaching of additional sources). Not sure if I've missed something, but I thought there're no reasons to use SIMS anymore, right? Anyway, I guess making checksums OPTIONAL/RECOMMENDED is probably also a good idea for SFS. Thanks, I wasn't aware of this XEP. I had a look and it's not fit for what I have in mind. Not using references is a drawback for me, since Converse already supports and uses references. Not supporting mixed content is a deal breaker, since I'm interested in the case where users write messages containing URLs pointing to media, not just sharing files. ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Thoughts on XEP-0385 Stateless Inline Media Sharing
Hi Marvin On 03.07.21 17:01, Marvin W wrote: Hi, Remember that URLs are not guaranteed to be globally reachable (examples: restricted to company network or requiring authentication). For most users it's not easily understandable if a URL can be reached by the recipient client or not. Thus sending an URL of funnypics.com instead of downloading that file and uploading it can lead to unexpected behavior. If you're behind a corporate firewall, then you're probably already aware that certain sites are blocked, so the behavior isn't necessarily that unexpected. I'm happy to leave the decision to the user whether they want to share a URL or a file. Not all servers support HTTP file upload and not all clients support Jingle, so sharing via 3rd party URL is the only option some people have. Given that users already share URLs, I would like to have better metadata so that I can provide a better UX. Fetching files from untrusted third-party servers is a privacy nightmare. We already have this issue with oob file sharing and I remember a proposal to restrict automatic fetching to known-good servers or if the domain of the URL matches the domainpart of the JID (or is a subdomain thereof). This wouldn't work when we want to accept sending URLs from funnypics.com. Sounds like configuration, and not something that would ever be universal. When doing inline sharing, the sending client would want to display the media inline as well. For this it also needs to fetch the media anyway and thus calculating the hash would be easy. I don't see this as simply "inline sharing" and I don't share the assumption that you'll always show the image to yourself when you're sharing the URL. Also, at least in my web client, the images are shown asynchronously after the message has been sent, and they're shown via tags, so the browser is responsible for fetching the image and caching it, not my client. Also, especially when sharing image files, it would be great to have a thumbnail so that you don't have to fetch the full (maybe high quality) image when not explicitly needed, i.e. on mobile. To generate a thumbnail, the sending client has to first download the image. Nice to have, but not essential. Providing the hash of the file is a good way to allow caching. Thus if you receive the same meme several times, but from different URLs, the hash will allow you to only download it once. This is not possible via ETag, due to ETags not following a common generation scheme. Indeed, I'm however dealing with the case where it's not deemed desirable to first download the entire file in order to generate a hash. When receiving a file via SIMS, the URL to the file is typically not going to be exposed to the end user (instead the inline media or thumbnail and maybe filename are displayed). Thus when forwarding the file to another contact, users will typically want to do this directly instead of through copying the URL. In that case, all the details from SIMS can be just forwarded as is. I'd also like to mention that it probably is a rare case (although it may happen from time to time) where you forward a file without downloading it first. I read a lot of assumptions in this last paragraph that don't match up to the use-case that I have and seem to come from the perspective of a desktop or perhaps mobile client developer. When sharing video URLs on the web, the browser is not going to download the entire video before starting to show it to you in a tag. I don't know any instant messaging client that would have the equivalent of sharing a URL using SIMS. Some clients do allow sending a URL as a message and then have the server attach some SIMS-like metadata to that message. Yes, and that's another use-case where it might be preferable to simply do a HEAD request and use the headers rather than downloading the whole file. There's also client side generated link previews, but these are typically not the same as I'd expect SIMS to be used, because links don't necessarily qualify as media (although I agree media is a very broad term and thus probably most things that URLs point to are actually somewhat media). When they link directly to media they do qualify as media in my opinion. Currently, when I share an image from my browser on Android, I can just use the browsers "share image" feature (long press on the image) and select Conversations. The browser will transfer the image to Conversations without the need to download it again (might also actually do another download, but that's another story) and Conversations uploads it using HTTP file upload as usual. This works great for me and I wonder what would be wrong with this approach? Nothing, but I'm working on a web client, not a mobile client. - JC On 02.07.21 12:15, JC Brand wrote: Hi everyone I've been reading XEP-0385 SIMS with an eye towards potentially implementing it, when I noticed that a XEP-0300 hash is required, in order to allow
Re: [Standards] Thoughts on XEP-0385 Stateless Inline Media Sharing
Hi, XEP-0447 currently lacks a content-disposition information to clarify if a (media) file should be displayed inline or not. This is on my TODO. XEP-0447 does not allow for mixed content which XEP-0385 supports. Including the hash is already "only" RECOMMENDED for XEP-0447. However it is required for making use of source attaching (3.3), as otherwise the integrity of attached sources cannot be verified. Marvin On 03.07.21 17:44, Linus Jahn wrote: > Hello, > > I just wanted to note that there's also XEP-0447: Stateless file sharing [1] > which has some > improvements over SIMS (like multiple files in one message and a little > simpler API (no need to > use character counting / the References XEP) and later attaching of > additional sources). > > Not sure if I've missed something, but I thought there're no reasons to use > SIMS anymore, right? > > Anyway, I guess making checksums OPTIONAL/RECOMMENDED is probably also a good > idea for SFS. > > Cheers > > > [1]: https://xmpp.org/extensions/xep-0447.html > ___ > Standards mailing list > Info: https://mail.jabber.org/mailman/listinfo/standards > Unsubscribe: standards-unsubscr...@xmpp.org > ___ > ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Thoughts on XEP-0385 Stateless Inline Media Sharing
Hello, I just wanted to note that there's also XEP-0447: Stateless file sharing [1] which has some improvements over SIMS (like multiple files in one message and a little simpler API (no need to use character counting / the References XEP) and later attaching of additional sources). Not sure if I've missed something, but I thought there're no reasons to use SIMS anymore, right? Anyway, I guess making checksums OPTIONAL/RECOMMENDED is probably also a good idea for SFS. Cheers [1]: https://xmpp.org/extensions/xep-0447.html ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Thoughts on XEP-0385 Stateless Inline Media Sharing
Hi, Remember that URLs are not guaranteed to be globally reachable (examples: restricted to company network or requiring authentication). For most users it's not easily understandable if a URL can be reached by the recipient client or not. Thus sending an URL of funnypics.com instead of downloading that file and uploading it can lead to unexpected behavior. Fetching files from untrusted third-party servers is a privacy nightmare. We already have this issue with oob file sharing and I remember a proposal to restrict automatic fetching to known-good servers or if the domain of the URL matches the domainpart of the JID (or is a subdomain thereof). This wouldn't work when we want to accept sending URLs from funnypics.com. When doing inline sharing, the sending client would want to display the media inline as well. For this it also needs to fetch the media anyway and thus calculating the hash would be easy. Also, especially when sharing image files, it would be great to have a thumbnail so that you don't have to fetch the full (maybe high quality) image when not explicitly needed, i.e. on mobile. To generate a thumbnail, the sending client has to first download the image. Providing the hash of the file is a good way to allow caching. Thus if you receive the same meme several times, but from different URLs, the hash will allow you to only download it once. This is not possible via ETag, due to ETags not following a common generation scheme. When receiving a file via SIMS, the URL to the file is typically not going to be exposed to the end user (instead the inline media or thumbnail and maybe filename are displayed). Thus when forwarding the file to another contact, users will typically want to do this directly instead of through copying the URL. In that case, all the details from SIMS can be just forwarded as is. I'd also like to mention that it probably is a rare case (although it may happen from time to time) where you forward a file without downloading it first. I don't know any instant messaging client that would have the equivalent of sharing a URL using SIMS. Some clients do allow sending a URL as a message and then have the server attach some SIMS-like metadata to that message. There's also client side generated link previews, but these are typically not the same as I'd expect SIMS to be used, because links don't necessarily qualify as media (although I agree media is a very broad term and thus probably most things that URLs point to are actually somewhat media). Currently, when I share an image from my browser on Android, I can just use the browsers "share image" feature (long press on the image) and select Conversations. The browser will transfer the image to Conversations without the need to download it again (might also actually do another download, but that's another story) and Conversations uploads it using HTTP file upload as usual. This works great for me and I wonder what would be wrong with this approach? Marvin On 02.07.21 12:15, JC Brand wrote: > Hi everyone > > > I've been reading XEP-0385 SIMS with an eye towards potentially > implementing it, when I noticed that a XEP-0300 hash is required, in > order to allow integrity checking and caching. > > In the introduction of SIMS, the following is written: > "This proposal aims to provide a protocol that will enable XMPP clients > to implement a great user experience (UX) around the process of sharing > media in conversations." > > I think making inclusion of the file hash mandatory is at odds with that > goal, the reason being that a lot of the media being shared in chats is > shared as URLs to 3rd parties where the media is hosted. > In other words, the sharer of the media doesn't necessarily know the > hash of the media because it was never on their own machine to begin with. > > Here's the use-case I have in mind: > > Romeo sees a funny picture on funnypics.com. He right-clicks on the > picture, copies the URL and sends it to Juliet. Before the message is > sent, Romeo's client does an HTTP HEAD request, to get information on > the URL, and learns that its an image. > > Here's the response: > > HTTP/2 200 > last-modified: Fri, 02 Jul 2021 02:46:55 GMT > etag: "a1d5ea7e796f5da6980bed86a8396664" > content-type: image/jpeg > cache-control: public, max-age=31536000 > accept-ranges: bytes > date: Fri, 02 Jul 2021 06:44:39 GMT > access-control-allow-methods: GET, OPTIONS > access-control-allow-origin: * > content-length: 110220 > > The "content-type" header can be used for the SIMS tag and > the "content-length" header for the . > > The server also responds with an "etag" header, which can be used for > caching, but unfortunately not for integrity checking, since the method > by which it's generated is not specified. > If the etag is passed along via SIMS (instead of the hash), the > receiving client could also check whether it matches what's returned by > the webserver, although I'm not sure what conclusions can be drawn if >
