Re: Clipboard API: remove dangerous formats from mandatory data types
Hallvord, IIUC the issue is that while transcoding complex formats via formats that can be easily sanity-checked by the browser takes care of letting content set complex formats like JPEG, GIF while protecting local content, but it loses the ability for content to pass that content to other local applications losslessly. e.g. transcoding to BMP would remove JPEG location, time/date, etc annotations. The bug you refer to looks like it's talking about intra-Browser, i.e. content->content, copy/paste operations for arbitrary MIME-types? Cheers, Wez @ Google On 6 February 2016 at 05:01, Hallvord Reiar Michaelsen Steen < hst...@mozilla.com> wrote: > BTW, we have a slightly related and interesting discussion regarding > custom data types going on here: > https://bugzilla.mozilla.org/show_bug.cgi?id=860857 > > I should try to sum up some of the arguments and proposals and present > them on this list but if anyone wants to chime in, I'd appreciate more > ideas in that bug too. I'll do my best to present them objectively and > fairly on the list :) > -Hallvord R > >
Re: Clipboard API: remove dangerous formats from mandatory data types
My 2c inline below: On 27 July 2015 at 12:03, Hallvord Reiar Michaelsen Steen hst...@mozilla.com wrote: On Tue, Jun 9, 2015 at 8:39 PM, Daniel Cheng dch...@google.com wrote: Currently, the Clipboard API [1] mandates support for a number of formats. Unfortunately, we do not believe it is possible to safely support writing a number of formats to the clipboard: - image/png - image/jpg, image/jpeg - image/gif Hi Daniel, I've been pondering this a bit and I think a first step is to split the list of mandatory data types into two: one list for types you must support reading from the clipboard, and one (smaller) for types you must support writing to the clipboard. So PNG, JPG et al go in the support reading from clipboard list, and the support writing starts out with text/plain, text/html and text/uri-list - although it would be nice if CSV was also considered safe enough. Do we want to make the distinction between the formats the UA must allow content to pass-through directly to/from the local clipboard, versus the ones that it must support setting/getting, but may transcode from/to some preferred format for? e.g. a UA might support content setting BMP, JPEG, GIF, etc but only actually place an HBITMAP on the Windows clipboard, and rely on the OS and/or peer applications to do any necessary conversions from that? It would also be good if we could come up with an API for safely writing images to the clipboard. Just playing: event.clipboardData.addImageFromCanvas(canvasElm, 'image/png') This seems like a good idea to me, though we then get into the fun game of what the mandatory formats for this API are, too. :D And what about a getImageToCanvas() API? Hot or not? -Hallvord R.
Re: Clipboard API: remove dangerous formats from mandatory data types
Florian, as I pointed out earlier, this proposal is to remove the requirement that user agents allow content to set the local system clipboard directly with certain pre-existing clipboard formats, because doing so safely is not possible in general. Removing the requirement from the spec will simply mean that the spec more accurately reflects what is actually implemented in existing user agents. Defining how user agents can/should support setting of arbitrary formats is a separate discussion; user agents don't generally support that - including not supporting application/octet-stream, which the spec doesn't actually define the behaviour of! - so it would in effect be a new feature of the API the behaviour of which would need to be properly specified. Please feel free to fork this thread if that's something you'd like to propose ideas for. :) Thanks! On Thu, 25 Jun 2015 at 19:13 Florian Bösch pya...@gmail.com wrote: I think you underestimate the integrative need that web-apps will acquire and the lengths they will go to faced with a business need to make it work once clipboard API becomes common developer knowledge.
Re: Clipboard API: remove dangerous formats from mandatory data types
Sorry Florian, but I don't see what that has to do with whether or not the Clipboard Events spec mandates that web content can generate their own JPEG or PNG and place it directly on the local system clipboard. What is it that you're actually proposing? On Thu, 25 Jun 2015 at 13:31 Florian Bösch pya...@gmail.com wrote: No idea. Also doesn't matter jack. There could be some now or in the future. There's a variety of programs that support HDRi (photoshop, lightroom, hdri-studio, etc.). It's fairly logical that at some point some or another variant of HDR format will make its way into clipboards. The same applies to pretty much any other data format be that a file or something else.
Re: Clipboard API: remove dangerous formats from mandatory data types
And, again, I don't see what that has to do with whether the spec mandates that user agents let apps place JPEG, PNG or GIF directly on the local system clipboard. The spec doesn't currently mandate OpenEXR be supported, so it's currently up to individual user agents to decide whether they can support that format safely. On Thu, 25 Jun 2015 at 14:16 Florian Bösch pya...@gmail.com wrote: On Thu, Jun 25, 2015 at 3:13 PM, Wez w...@google.com wrote: I think there's obvious value in support for arbitrary content-specific formats, but IMO the spec should at least give guidance on how to present the capability in a safe way. Which is exactly the core of my question. If you intend to make it say, safe to put OpenEXR into the clipboard (as opposed to letting an app just put any bytes there), the UA has to understand OpenEXR. Since I don't see how the UA can understand every conceivable format in existence both future and past, I don't see how that should work.
Re: Clipboard API: remove dangerous formats from mandatory data types
Which user agents currently allow content to post OpenEXR to the local clipboard? On Wed, 24 Jun 2015 at 19:58 Florian Bösch pya...@gmail.com wrote: No, but the specification doesn't require you to exclude it. So how're applications going to swap OpenEXR if you only let em stick in jpegs, pngs and gifs? On Wed, Jun 24, 2015 at 8:46 PM, Wez w...@google.com wrote: I don't think OpenEXR is one of the formats required by the Clipboard Events spec, is it..? On Wed, Jun 24, 2015, 18:49 Florian Bösch pya...@gmail.com wrote: And how exactly do you intend to support for instance OpenEXR? On Wed, Jun 24, 2015 at 5:44 PM, Wez w...@google.com wrote: Hallvord, Yes, content would be limited to providing text, image etc data to the user agent to place on the clipboard, and letting the user agent synthesize whatever formats (JPEG, PNG etc) other apps require. That has the advantage of preventing malicious content using esoteric flags or features to compromise recipients, but conversely means that legitimate content cannot use format-specific features, e.g. content would not be able to write a JPEG containing a comment block, geo tags or timestamp information. Wez On Sat, 13 Jun 2015 at 11:57 Hallvord Reiar Michaelsen Steen hst...@mozilla.com wrote: On Thu, Jun 11, 2015 at 7:51 PM, Wez w...@google.com wrote: Hallvord, The proposal isn't to remove support for copying/pasting images, but to restrict web content from placing compressed image data in one of these formats on the clipboard directly - there's no issue with content pasting raw pixels from a canvas, for example, since scope for abusing that to compromise the recipient is extremely limited by comparison to JPEG, PNG or GIF. Well, but as far as I can tell we don't currently *have* a way JS can place pixels from a canvas on the clipboard (except by putting a piece of data labelled as image/png or similar there). So if you're pushing back against the idea that JS can place random data on the clipboard and label it image/png, how exactly would you propose JS-triggered copy of image data to work? Say, from a CANVAS-based image editor? -Hallvord
Re: Clipboard API: remove dangerous formats from mandatory data types
I don't believe I've said any such thing re jpeg, png and gif... quite the opposite, in fact. The point of this thread is that the spec currently *requires* user agents allow content to supply JPEG, PNG or GIF data directly *to the local clipboard*, which is risky. We're therefore proposing to remove that requirement - content can still supply images to the clipboard and the user agent can still synthesize whatever formats it chooses to. Whether or not user agents support web content setting other arbitrary content types (such as OpenEXR) to the local system clipboard is a separate question - there's nothing in the spec mandating that user agents support it, nor mandating that they don't - at present each user agent can choose whether or not to support arbitrary formats. I think there's obvious value in support for arbitrary content-specific formats, but IMO the spec should at least give guidance on how to present the capability in a safe way. HTH, Wez On Thu, 25 Jun 2015 at 13:56 Florian Bösch pya...@gmail.com wrote: Well let's say some webapp generates an OpenEXR and wants to put it into the clipboard as image/x-exr which would make sense cause any eventual program that'd support OpenEXR would probably look for that mime type. You've said you're going to restrict image types to jpeg, png and gif, and so my question is, how exactly do you intend to support OpenEXR? On Thu, Jun 25, 2015 at 2:51 PM, Wez w...@google.com wrote: Sorry Florian, but I don't see what that has to do with whether or not the Clipboard Events spec mandates that web content can generate their own JPEG or PNG and place it directly on the local system clipboard. What is it that you're actually proposing? On Thu, 25 Jun 2015 at 13:31 Florian Bösch pya...@gmail.com wrote: No idea. Also doesn't matter jack. There could be some now or in the future. There's a variety of programs that support HDRi (photoshop, lightroom, hdri-studio, etc.). It's fairly logical that at some point some or another variant of HDR format will make its way into clipboards. The same applies to pretty much any other data format be that a file or something else.
Re: Clipboard API: remove dangerous formats from mandatory data types
You've mentioned resorting to application/octet-stream several times in the context of this discussion, where AFAICT the spec actually only describes using it as a fall-back for cases of file references on the clipboard for which the user agent is unable to determine the file type. So IIUC you're suggesting that user agents should implement application/octet-stream (as is also mandated by the spec, albeit without a clear indication of what it means in this context) by putting the content on the clipboard as an un-typed file? Again, I'm unclear as to what the alternative is that you're proposing? On Thu, 25 Jun 2015 at 15:27 Florian Bösch pya...@gmail.com wrote: Surely you realize that if the specification where to state to only safely expose data to the clipboard, this can only be interpreted to deny any formats but those a UA can interprete and deem well-formed. If such a thing where to be done, that would leave any user of the clipboard no recourse but to resort to application/octett-stream and ignore any other metadata as the merry magic header guessing game gets underway. For all you'd have achieved was to muddle any meaning of the mime-type and forced applications to work around an unenforceable restriction. On Thu, Jun 25, 2015 at 3:21 PM, Wez w...@google.com wrote: And, again, I don't see what that has to do with whether the spec mandates that user agents let apps place JPEG, PNG or GIF directly on the local system clipboard. The spec doesn't currently mandate OpenEXR be supported, so it's currently up to individual user agents to decide whether they can support that format safely. On Thu, 25 Jun 2015 at 14:16 Florian Bösch pya...@gmail.com wrote: On Thu, Jun 25, 2015 at 3:13 PM, Wez w...@google.com wrote: I think there's obvious value in support for arbitrary content-specific formats, but IMO the spec should at least give guidance on how to present the capability in a safe way. Which is exactly the core of my question. If you intend to make it say, safe to put OpenEXR into the clipboard (as opposed to letting an app just put any bytes there), the UA has to understand OpenEXR. Since I don't see how the UA can understand every conceivable format in existence both future and past, I don't see how that should work.
Re: Clipboard API: remove dangerous formats from mandatory data types
Florian, you keep referring to using application/octet-stream - that's not a format that all user agents support (although the spec says they should ;), nor is there any mention in the spec of what it means to place content on the clipboard in that format (given that platform native clipboards each have their own content-type annotations). So it sounds like you're saying we should also remove application/octet-stream as a mandatory format? On Thu, 25 Jun 2015 at 15:55 Florian Bösch pya...@gmail.com wrote: It's very simple. Applications need to know what's in the clipboard to know what to do with it. There is also a vast variety of things that could find itself in the clipboard in terms of formats, both formal and informal. Mime types are one of these things that applications would use to do that. If a UA where to restict what mime type you can put into the clipboard, that forces the clipboard user to use application/octet-stream. And in consequence, that forces any such-willing application to forgoe the mime-type information from the OS'es clipboard API and figure out what's in it from the content. In turn this would give rise to another way to markup mime-types in-line with the content. And once you've forced such ad-hoc solutions to emerge for meddling with what people can put in the clipboard, you'll have no standing to put that geenie back in the bottle, again, relevant XKCD quote omitted. On Thu, Jun 25, 2015 at 4:48 PM, Wez w...@google.com wrote: You've mentioned resorting to application/octet-stream several times in the context of this discussion, where AFAICT the spec actually only describes using it as a fall-back for cases of file references on the clipboard for which the user agent is unable to determine the file type. So IIUC you're suggesting that user agents should implement application/octet-stream (as is also mandated by the spec, albeit without a clear indication of what it means in this context) by putting the content on the clipboard as an un-typed file? Again, I'm unclear as to what the alternative is that you're proposing? On Thu, 25 Jun 2015 at 15:27 Florian Bösch pya...@gmail.com wrote: Surely you realize that if the specification where to state to only safely expose data to the clipboard, this can only be interpreted to deny any formats but those a UA can interprete and deem well-formed. If such a thing where to be done, that would leave any user of the clipboard no recourse but to resort to application/octett-stream and ignore any other metadata as the merry magic header guessing game gets underway. For all you'd have achieved was to muddle any meaning of the mime-type and forced applications to work around an unenforceable restriction. On Thu, Jun 25, 2015 at 3:21 PM, Wez w...@google.com wrote: And, again, I don't see what that has to do with whether the spec mandates that user agents let apps place JPEG, PNG or GIF directly on the local system clipboard. The spec doesn't currently mandate OpenEXR be supported, so it's currently up to individual user agents to decide whether they can support that format safely. On Thu, 25 Jun 2015 at 14:16 Florian Bösch pya...@gmail.com wrote: On Thu, Jun 25, 2015 at 3:13 PM, Wez w...@google.com wrote: I think there's obvious value in support for arbitrary content-specific formats, but IMO the spec should at least give guidance on how to present the capability in a safe way. Which is exactly the core of my question. If you intend to make it say, safe to put OpenEXR into the clipboard (as opposed to letting an app just put any bytes there), the UA has to understand OpenEXR. Since I don't see how the UA can understand every conceivable format in existence both future and past, I don't see how that should work.
Re: Clipboard API: remove dangerous formats from mandatory data types
It still sounds like you're advocating removing the requirement that UAs support application/octet-stream (which makes sense, since its behaviour doesn't seem to be specified anyway). If you're suggesting something else, please elaborate on what that something else is. :) On Thu, 25 Jun 2015 at 16:23 Florian Bösch pya...@gmail.com wrote: No, what I'm saying is that if you restrict mime types (or don't explicitly prohibit such restriction), but require application/octet-stream, that application/octet-stream becomes the undesirable mime-type dumping ground. And that would be bad because that makes it much harder for applications to deal with content. But if that's the only way UAs are going to act, then applications will work around that by using elaborate guessing code based on magic bytes, and perhaps some application developers will use their own mime-type annotation pretended to the octet-stream. If you inconvenience people, but don't make it impossible to work around the inconvenience, then people will work around the inconvenience. It can't be the intention to encourage them work around it. So you've got to either not inconvenience them, or make working around impossible. On Thu, Jun 25, 2015 at 5:07 PM, Wez w...@google.com wrote: Florian, you keep referring to using application/octet-stream - that's not a format that all user agents support (although the spec says they should ;), nor is there any mention in the spec of what it means to place content on the clipboard in that format (given that platform native clipboards each have their own content-type annotations). So it sounds like you're saying we should also remove application/octet-stream as a mandatory format? On Thu, 25 Jun 2015 at 15:55 Florian Bösch pya...@gmail.com wrote: It's very simple. Applications need to know what's in the clipboard to know what to do with it. There is also a vast variety of things that could find itself in the clipboard in terms of formats, both formal and informal. Mime types are one of these things that applications would use to do that. If a UA where to restict what mime type you can put into the clipboard, that forces the clipboard user to use application/octet-stream. And in consequence, that forces any such-willing application to forgoe the mime-type information from the OS'es clipboard API and figure out what's in it from the content. In turn this would give rise to another way to markup mime-types in-line with the content. And once you've forced such ad-hoc solutions to emerge for meddling with what people can put in the clipboard, you'll have no standing to put that geenie back in the bottle, again, relevant XKCD quote omitted. On Thu, Jun 25, 2015 at 4:48 PM, Wez w...@google.com wrote: You've mentioned resorting to application/octet-stream several times in the context of this discussion, where AFAICT the spec actually only describes using it as a fall-back for cases of file references on the clipboard for which the user agent is unable to determine the file type. So IIUC you're suggesting that user agents should implement application/octet-stream (as is also mandated by the spec, albeit without a clear indication of what it means in this context) by putting the content on the clipboard as an un-typed file? Again, I'm unclear as to what the alternative is that you're proposing? On Thu, 25 Jun 2015 at 15:27 Florian Bösch pya...@gmail.com wrote: Surely you realize that if the specification where to state to only safely expose data to the clipboard, this can only be interpreted to deny any formats but those a UA can interprete and deem well-formed. If such a thing where to be done, that would leave any user of the clipboard no recourse but to resort to application/octett-stream and ignore any other metadata as the merry magic header guessing game gets underway. For all you'd have achieved was to muddle any meaning of the mime-type and forced applications to work around an unenforceable restriction. On Thu, Jun 25, 2015 at 3:21 PM, Wez w...@google.com wrote: And, again, I don't see what that has to do with whether the spec mandates that user agents let apps place JPEG, PNG or GIF directly on the local system clipboard. The spec doesn't currently mandate OpenEXR be supported, so it's currently up to individual user agents to decide whether they can support that format safely. On Thu, 25 Jun 2015 at 14:16 Florian Bösch pya...@gmail.com wrote: On Thu, Jun 25, 2015 at 3:13 PM, Wez w...@google.com wrote: I think there's obvious value in support for arbitrary content-specific formats, but IMO the spec should at least give guidance on how to present the capability in a safe way. Which is exactly the core of my question. If you intend to make it say, safe to put OpenEXR into the clipboard (as opposed to letting an app just put any bytes there), the UA has to understand OpenEXR. Since I don't see how the UA can understand
Re: Clipboard API: remove dangerous formats from mandatory data types
Hallvord, Yes, content would be limited to providing text, image etc data to the user agent to place on the clipboard, and letting the user agent synthesize whatever formats (JPEG, PNG etc) other apps require. That has the advantage of preventing malicious content using esoteric flags or features to compromise recipients, but conversely means that legitimate content cannot use format-specific features, e.g. content would not be able to write a JPEG containing a comment block, geo tags or timestamp information. Wez On Sat, 13 Jun 2015 at 11:57 Hallvord Reiar Michaelsen Steen hst...@mozilla.com wrote: On Thu, Jun 11, 2015 at 7:51 PM, Wez w...@google.com wrote: Hallvord, The proposal isn't to remove support for copying/pasting images, but to restrict web content from placing compressed image data in one of these formats on the clipboard directly - there's no issue with content pasting raw pixels from a canvas, for example, since scope for abusing that to compromise the recipient is extremely limited by comparison to JPEG, PNG or GIF. Well, but as far as I can tell we don't currently *have* a way JS can place pixels from a canvas on the clipboard (except by putting a piece of data labelled as image/png or similar there). So if you're pushing back against the idea that JS can place random data on the clipboard and label it image/png, how exactly would you propose JS-triggered copy of image data to work? Say, from a CANVAS-based image editor? -Hallvord
Re: Clipboard API: remove dangerous formats from mandatory data types
I don't think OpenEXR is one of the formats required by the Clipboard Events spec, is it..? On Wed, Jun 24, 2015, 18:49 Florian Bösch pya...@gmail.com wrote: And how exactly do you intend to support for instance OpenEXR? On Wed, Jun 24, 2015 at 5:44 PM, Wez w...@google.com wrote: Hallvord, Yes, content would be limited to providing text, image etc data to the user agent to place on the clipboard, and letting the user agent synthesize whatever formats (JPEG, PNG etc) other apps require. That has the advantage of preventing malicious content using esoteric flags or features to compromise recipients, but conversely means that legitimate content cannot use format-specific features, e.g. content would not be able to write a JPEG containing a comment block, geo tags or timestamp information. Wez On Sat, 13 Jun 2015 at 11:57 Hallvord Reiar Michaelsen Steen hst...@mozilla.com wrote: On Thu, Jun 11, 2015 at 7:51 PM, Wez w...@google.com wrote: Hallvord, The proposal isn't to remove support for copying/pasting images, but to restrict web content from placing compressed image data in one of these formats on the clipboard directly - there's no issue with content pasting raw pixels from a canvas, for example, since scope for abusing that to compromise the recipient is extremely limited by comparison to JPEG, PNG or GIF. Well, but as far as I can tell we don't currently *have* a way JS can place pixels from a canvas on the clipboard (except by putting a piece of data labelled as image/png or similar there). So if you're pushing back against the idea that JS can place random data on the clipboard and label it image/png, how exactly would you propose JS-triggered copy of image data to work? Say, from a CANVAS-based image editor? -Hallvord
Re: Clipboard API: remove dangerous formats from mandatory data types
Hallvord, The proposal isn't to remove support for copying/pasting images, but to restrict web content from placing compressed image data in one of these formats on the clipboard directly - there's no issue with content pasting raw pixels from a canvas, for example, since scope for abusing that to compromise the recipient is extremely limited by comparison to JPEG, PNG or GIF. The UA is still at liberty to synthesize these formats itself, based on the raw imagery provided by the content, to populate the clipboard with formats that other applications want. HTH, Wez On 10 June 2015 at 23:31, Hallvord Reiar Michaelsen Steen hst...@mozilla.com wrote: On Tue, Jun 9, 2015 at 8:39 PM, Daniel Cheng dch...@google.com wrote: Currently, the Clipboard API [1] mandates support for a number of formats. Unfortunately, we do not believe it is possible to safely support writing a number of formats to the clipboard: - image/png - image/jpg, image/jpeg - image/gif Hi Daniel, copying images to the clipboard is an important use case. Do you have any suggestions for how we could meet this use case in a safer way? For example, would it be safe and easy to add a little bit of magic to make clipboardData.items.add(canvasElement) put a PNG on the clipboard? Perhaps copying a rendered imgElement should work too? -Hallvord
Re: Clipboard API: remove dangerous formats from mandatory data types
IIUC that approach won't help, because the problem here is not necessarily invalid/malformed data, but even valid data that some decoders fail to handle gracefully. On 9 June 2015 at 14:13, Paul Libbrecht p...@hoplahup.net wrote: On 9/06/15 23:08, Daniel Cheng wrote: So the solution is to require that browsers that make known media-types in the clipboard actually parse it for its value? That sounds doable (and probably even useful: e.g. put other picture flavours in case of a pictures). I don't think I understand what this means. Since the browser is what would act on behalf of JS when putting a given data into the clipboard, it could check that this data is well formed and maybe matches the patterns of known exploits. paul
Re: Proposal for a DOM L3 Events Telecon
Hi guys, mind if I tag along with Gary on the call? On 30 April 2013 13:46, Gary Kačmarčík (Кошмарчик) gary...@google.comwrote: On Mon, Apr 29, 2013 at 12:59 PM, Travis Leithead travis.leith...@microsoft.com wrote: I’d like to propose we start a call to begin to work toward resolving the final bugs in the spec and for other business related to getting DOM L3 Events to CR. On the call we can workout what subsequent meetings we should also arrange. ** ** Does next Tuesday (May 7th), at 11 am PST work your you? Note that 11am PDT = 3am JST. If Masayuki is interested in joining, we should pick a late afternoon PDT time: 4pm PDT (Tue) = 8am JST (Wed) 5pm PDT (Tue) = 9am JST (Wed)
