Re: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
In addition, from a security perspective, what stops a malicious website from embedding something like img src=file:///etc/passwd style=display:none/img in the markup? We disallow this on copy by stripping such references. Hi Ben, picking up this old thread.. So we need to add a sanitize local references step/algorithm somewhere when JS writes data to clipboard? It would be great if you could have a look at https://w3c.github.io/clipboard-apis/#dfn-writing-contents-to-the-clipboard and suggest some text - maybe even in the form of a GitHub pull request? :) (I assume you strip *all* local references, not just specific blacklisted stuff like /etc/passwd - this probably needs testing with various types of slashes etc..) Do you have any other safety measures when data is written to the clipboard? -Hallvord
Re: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
So, the E-mail to Ben Peters bounced - he's no longer at Microsoft? Is there anyone on the IE team present on the list who is able to comment on this? -Hallvord R On Mon, Apr 20, 2015 at 10:38 PM, Hallvord Reiar Michaelsen Steen hst...@mozilla.com wrote: In addition, from a security perspective, what stops a malicious website from embedding something like img src=file:///etc/passwd style=display:none/img in the markup? We disallow this on copy by stripping such references. Hi Ben, picking up this old thread.. So we need to add a sanitize local references step/algorithm somewhere when JS writes data to clipboard? It would be great if you could have a look at https://w3c.github.io/clipboard-apis/#dfn-writing-contents-to-the-clipboard and suggest some text - maybe even in the form of a GitHub pull request? :) (I assume you strip *all* local references, not just specific blacklisted stuff like /etc/passwd - this probably needs testing with various types of slashes etc..) Do you have any other safety measures when data is written to the clipboard? -Hallvord
RE: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
Hi Daniel, I'm trying to make sure I correctly understand how the IE11 version of this works. From the sample (http://msdn.microsoft.com/en-us/library/ie/dn254935(v=vs.85).aspx), it looks like if a user pastes in some HTML that references local images, IE11 automatically captures the referenced files into the clipboard. Then the page uses msConvertUrl() rewrites references to the src attributes in the text/html DataTransferItem to reference the blob URLs, right? Given that the drag data store is in read only mode at this point, it seems weird to allow mutations at this point. My understanding was that read only was intended to keep sites from changing the system clipboard outside of cut/copy events. We don’t change the system clipboard, only the pasted html. Sites could easily change it right after the paste happens, so we're saving them a step. In addition, from a security perspective, what stops a malicious website from embedding something like img src=file:///etc/passwd style=display:none/img in the markup? We disallow this on copy by stripping such references.
Re: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
On Wed, Feb 12, 2014 at 1:25 AM, Hallvord R. M. Steen hst...@mozilla.comwrote:
Hi Hallvord!
Hi Ben! Thanks for responding to my request for feedback - especially
since the IE team has done some interesting work in this area and is
arguably ahead of the rest! :-)
The IE11 API you mentioned is msConvertURL [1] (also on the IE blog
[2]), and it was designed as a
simple way for sites to choose DataURI or Blob for otherwise
inaccessible images.
X
As you mention, it is not possible to tell which file/image corresponds
to which img because it’s
really designed as a simple approach for cases where a site wants to
always use blob or dataURI for
images that they couldn’t otherwise access.
That's more or less what I thought, based on the blog post, so thanks for
describing it in detail.
We are considering doing the CID approach as well in the future. It is
nice to have the additional
control of seeing which img src you are changing, and it will likely
work better for copy, not just
paste like convertURL.
Actually, I haven't truly considered the copy case here yet. I've sort of
assumed that given that you can put multiple bits of data on a clipboard,
the various clipboard implementations should already have a way one piece
of data can reference one specific other piece of data - I haven't really
found the technical details here.
I don't actually know of any native clipboard formats that allow data
associated with one format to reference other data in the same operation,
so browsers would have to roll their own implementation/figure out how to
get it to work well with native apps.
Of course it would be nice to support a script that wants to generate
random HTML with embedded files to place on the clipboard (although I think
most of those use cases can already be met by using URLs and assuming that
any software reading HTML from the clipboard can understand URLs..).
However, one can imagine a use case for example with a CANVAS app where the
script wants to copy the state of the CANVAS as an image inside HTML it
places on the clipboard - having the script create src=cid:n type markup,
append files, and make the UA translate that to the platform's native
clipboard implementation's way of referencing one part on the clipboard
from another part..
We believe that convertURL does not block using the CIDs you have in the
current spec.
I suppose not, but perhaps the more relevant question is: should we
standardise convertURL? Would it still have a use case if we take the cid:
route? (And I guess a related question is: given that we've done data: URLs
for a while, how much content will we break if, say, Firefox moves from
data: to cid:? Do we need to make cid: opt-in somehow, like you're doing
with convertURL?)
I'm trying to make sure I correctly understand how the IE11 version of this
works. From the sample (
http://msdn.microsoft.com/en-us/library/ie/dn254935(v=vs.85).aspx), it
looks like if a user pastes in some HTML that references local images, IE11
automatically captures the referenced files into the clipboard. Then the
page uses msConvertUrl() rewrites references to the src attributes in the
text/html DataTransferItem to reference the blob URLs, right?
Given that the drag data store is in read only mode at this point, it
seems weird to allow mutations at this point.
In addition, from a security perspective, what stops a malicious website
from embedding something like img src=file:///etc/passwd
style=display:none/img in the markup?
To better understand your approach and allow us to help move it forward,
can you give us sample
javascript that a site would use to set the DataTransferItems for HTML
and the related images during copy?
As I said, I have not really considered this use case - so the spec
doesn't actually cover this. If we want to make this work, I suppose the JS
would look somewhat like this?
document.addEventListener('copy', function(e){
// So, you want to copy the current status of your game? No
problem.
// Let's say this game generates a number of PNG graphics from
CANVAS tags
// and keeps them in memory in ArrayBuffers or something
var html = 'divbplayer/b\'s medals: img src=cid:1img
src=cid:2/div';
e.clipboardData.items.add(html, 'text/html');
e.clipboardData.items.add(new File(medals[0], 'medal.png',
{type:'image/png'}));
e.clipboardData.items.add(new File(medals[1], 'medal.png',
{type:'image/png'}));
e.preventDefault();
}, false);
Second, can you provide the javascript for how a site would put them
into the pasted markup during paste?
The way I thought this would work, would be that the site starts XHR
uploads from the paste processing, and shows some intermediate 'loading'
animation or something before it gets the final URLs back from the server.
A bit like this (although some things could be more elegant, like the
insertion of
RE: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
Second, can you provide the javascript for how a site would put them into the pasted markup during paste? The way I thought this would work, would be that the site starts XHR uploads from the paste processing, and shows some intermediate 'loading' animation or something before it gets the final URLs back from the server. This generally makes sense. If sites prefer to use local dataURI or blob, they can use the blob URL, and then upload the file later (like in an email scenario). This means they don't have to wait for them to be on the server before displaying them. If they want to upload them first and use the server's new URL for them, they would need to do what you're saying. Sounds good, but this requires standardising something similar to msConvertURL(), right? I don't believe so. Couldn't the site just get the HTML DataTransferItem (which contains the CIDs), replace the CIDs with Blobs or DataURIs, and then insert the HTML where the paste was going to happen? Personally I think msConvertURL is a convenient way to do this. But it's not the only way. -Ben
RE: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
Of course it would be nice to support a script that wants to generate random
HTML with embedded files to place on the clipboard (although I think most of
those use cases can already be met by using URLs and assuming that any
software reading HTML from the clipboard can understand URLs..). However, one
can imagine a use case for example with a CANVAS app where the script wants
to copy the state of the CANVAS as an image inside HTML it places on the
clipboard - having the script create src=cid:n type markup, append files,
and make the UA translate that to the platform's native clipboard
implementation's way of referencing one part on the clipboard from another
part..
I was thinking about images that aren't available cross-domain, like Facebook
pictures. If you copy them, Facebook could use CID to place them on the
clipboard since a Facebook URL wouldn’t be accessible to an email client that
receives the pasted content.
document.addEventListener('copy', function(e){
// So, you want to copy the current status of your game? No problem.
// Let's say this game generates a number of PNG graphics from CANVAS
tags
// and keeps them in memory in ArrayBuffers or something
var html = 'divbplayer/b\'s medals: img src=cid:1img
src=cid:2/div';
e.clipboardData.items.add(html, 'text/html');
e.clipboardData.items.add(new File(medals[0], 'medal.png',
{type:'image/png'}));
e.clipboardData.items.add(new File(medals[1], 'medal.png',
{type:'image/png'}));
e.preventDefault();
}, false);
This seems like we're depending on add() working in a precise order, which
seems vulnerable to code changes or other issues. Maybe add() should return the
location in the list where the item was added, and then it can be used with CID
more safely? I know this is part of HTML5 not ClipboardAPI though. Thoughts?
Second, can you provide the javascript for how a site would put them into the
pasted markup during paste?
The way I thought this would work, would be that the site starts XHR uploads
from the paste processing, and shows some intermediate 'loading' animation or
something before it gets the final URLs back from the server. A bit like this
(although some things could be more elegant, like the insertion of the data
which needs to take cursors and selections into account):
http://jsfiddle.net/2Qqrp/
Thinking about it, it may be considered somewhat wasteful (or exceedingly
slow - if for example the embedded data is a video) to do it this way - but
it quickly gets complex and/or confusing if we have a some show this local
file until it's uploaded, then reference the online file instead magic..?
This generally makes sense. If sites prefer to use local dataURI or blob, they
can use the blob URL, and then upload the file later (like in an email
scenario). This means they don't have to wait for them to be on the server
before displaying them. If they want to upload them first and use the server's
new URL for them, they would need to do what you're saying.
-Ben
Re: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
Of course it would be nice to support a script that wants to generate random
HTML with embedded files
X
use case for example with a CANVAS app where the script wants to copy the
state of the CANVAS
I was thinking about images that aren't available cross-domain
Indeed, that's another use case which is pretty important.
document.addEventListener('copy', function(e){
// So, you want to copy the current status of your game? No problem.
// Let's say this game generates a number of PNG graphics from CANVAS
tags
// and keeps them in memory in ArrayBuffers or something
var html = 'divbplayer/b\'s medals: img src=cid:1img
src=cid:2/div';
e.clipboardData.items.add(html, 'text/html');
e.clipboardData.items.add(new File(medals[0], 'medal.png',
{type:'image/png'}));
e.clipboardData.items.add(new File(medals[1], 'medal.png',
{type:'image/png'}));
e.preventDefault();
}, false);
This seems like we're depending on add() working in a precise order, which
seems vulnerable
to code changes or other issues. Maybe add() should return the location in
the list where
the item was added, and then it can be used with CID more safely?
That is a good observation and a very good suggestion! We should suggest a
change to the HTML spec.
Second, can you provide the javascript for how a site would put them into
the pasted markup during paste?
The way I thought this would work, would be that the site starts XHR uploads
from the paste processing,
and shows some intermediate 'loading' animation or something before it gets
the final URLs back from the server.
This generally makes sense. If sites prefer to use local dataURI or blob,
they can use the blob URL, and then
upload the file later (like in an email scenario). This means they don't have
to wait for them to be on the
server before displaying them. If they want to upload them first and use the
server's new URL for them,
they would need to do what you're saying.
Sounds good, but this requires standardising something similar to
msConvertURL(), right?
-Hallvord
Re: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
Hi Hallvord!
Hi Ben! Thanks for responding to my request for feedback - especially since the
IE team has done some interesting work in this area and is arguably ahead of
the rest! :-)
The IE11 API you mentioned is msConvertURL [1] (also on the IE blog [2]), and
it was designed as a
simple way for sites to choose DataURI or Blob for otherwise inaccessible
images.
X
As you mention, it is not possible to tell which file/image corresponds to
which img because it’s
really designed as a simple approach for cases where a site wants to always
use blob or dataURI for
images that they couldn’t otherwise access.
That's more or less what I thought, based on the blog post, so thanks for
describing it in detail.
We are considering doing the CID approach as well in the future. It is nice
to have the additional
control of seeing which img src you are changing, and it will likely work
better for copy, not just
paste like convertURL.
Actually, I haven't truly considered the copy case here yet. I've sort of
assumed that given that you can put multiple bits of data on a clipboard, the
various clipboard implementations should already have a way one piece of data
can reference one specific other piece of data - I haven't really found the
technical details here.
Of course it would be nice to support a script that wants to generate random
HTML with embedded files to place on the clipboard (although I think most of
those use cases can already be met by using URLs and assuming that any software
reading HTML from the clipboard can understand URLs..). However, one can
imagine a use case for example with a CANVAS app where the script wants to copy
the state of the CANVAS as an image inside HTML it places on the clipboard -
having the script create src=cid:n type markup, append files, and make the UA
translate that to the platform's native clipboard implementation's way of
referencing one part on the clipboard from another part..
We believe that convertURL does not block using the CIDs you have in the
current spec.
I suppose not, but perhaps the more relevant question is: should we standardise
convertURL? Would it still have a use case if we take the cid: route? (And I
guess a related question is: given that we've done data: URLs for a while, how
much content will we break if, say, Firefox moves from data: to cid:? Do we
need to make cid: opt-in somehow, like you're doing with convertURL?)
To better understand your approach and allow us to help move it forward, can
you give us sample
javascript that a site would use to set the DataTransferItems for HTML and
the related images during copy?
As I said, I have not really considered this use case - so the spec doesn't
actually cover this. If we want to make this work, I suppose the JS would look
somewhat like this?
document.addEventListener('copy', function(e){
// So, you want to copy the current status of your game? No problem.
// Let's say this game generates a number of PNG graphics from CANVAS
tags
// and keeps them in memory in ArrayBuffers or something
var html = 'divbplayer/b\'s medals: img src=cid:1img
src=cid:2/div';
e.clipboardData.items.add(html, 'text/html');
e.clipboardData.items.add(new File(medals[0], 'medal.png',
{type:'image/png'}));
e.clipboardData.items.add(new File(medals[1], 'medal.png',
{type:'image/png'}));
e.preventDefault();
}, false);
Second, can you provide the javascript for how a site would put them into the
pasted markup during paste?
The way I thought this would work, would be that the site starts XHR uploads
from the paste processing, and shows some intermediate 'loading' animation or
something before it gets the final URLs back from the server. A bit like this
(although some things could be more elegant, like the insertion of the data
which needs to take cursors and selections into account):
http://jsfiddle.net/2Qqrp/
Thinking about it, it may be considered somewhat wasteful (or exceedingly slow
- if for example the embedded data is a video) to do it this way - but it
quickly gets complex and/or confusing if we have a some show this local file
until it's uploaded, then reference the online file instead magic..?
-Hallvord
RE: [clipboard events] seeking implementor feedback on using CID: URI scheme for pasting embedded binary data
Hi Hallvord!
The IE11 API you mentioned is msConvertURL [1] (also on the IE blog [2]), and
it was designed as a simple way for sites to choose DataURI or Blob for
otherwise inaccessible images. We default to DataURI to be interoperable with
Firefox’s current design of always doing DataURI for local images on the
clipboard. With a little bit of code, blobs can be used instead. Our
clipboardData.files implementation contains only images at the moment because
we have only partial support of the current version of the Clipboard API spec.
This means that with a quick ‘for’ loop over those images, javascript can
choose to create blobs, manage the blob’s memory, and upload the blobs to a
server. As you mention, it is not possible to tell which file/image corresponds
to which img because it’s really designed as a simple approach for cases
where a site wants to always use blob or dataURI for images that they couldn’t
otherwise access.
We are considering doing the CID approach as well in the future. It is nice to
have the additional control of seeing which img src you are changing, and it
will likely work better for copy, not just paste like convertURL. We believe
that convertURL does not block using the CIDs you have in the current spec. To
better understand your approach and allow us to help move it forward, can you
give us sample javascript that a site would use to set the DataTransferItems
for HTML and the related images during copy? Second, can you provide the
javascript for how a site would put them into the pasted markup during paste?
Regarding a couple other questions you ask:
* As far as I’ve seen, WebKit/Blink do not yet support images with local-system
sources. They do support binary images on the clipboard using
ClipboardData.items like in the example I found online [3].
* Some sites prefer DataURI to Blob because it’s all inline and doesn’t require
sending separate objects or managing memory, so I don’t think DataURI is
something we should discount.
Looking forward to seeing your sample code!
Ben Peters
[1] http://msdn.microsoft.com/en-us/library/ie/dn254951(v=vs.85).aspx
[2]
http://blogs.msdn.com/b/ie/archive/2013/10/24/enhanced-rich-editing-experiences-in-ie11.aspx
[3] http://strd6.com/2011/09/html5-javascript-pasting-image-data-in-chrome/
-Original Message-
From: Hallvord R. M. Steen [mailto:hst...@mozilla.com]
Sent: Thursday, January 23, 2014 5:30 PM
To: public-webapps
Subject: [clipboard events] seeking implementor feedback on using CID: URI
scheme for pasting embedded binary data
Hi,
pasting HTML that contains embeds (images, video) into a rich text editor is
a use case we should cover. It's currently handled in different ways -
* IE11 supports pasting images as either data: URLs or blobs [1] (and has a
non-standard method to fill in a gap in the blob approach). I don't
understand from this blog post how/if it supports referencing the binary
parts from the HTML. If for example you paste a snippet from a Word page that
contains two images, the DataTransferItemList is presumably populated with
two image files, which can be processed/uploaded using the blob method - but
how is the script processing the data supposed to know what IMG tag in the
pasted HTML each image file belongs to?
* Pasting stuff as data: URLs seems like a hack, wasting memory and requiring
quite some extra processing if there is a lot of data.
* Firefox apparently happily passes on file:/// URLs with local paths and all
[2], this is of course a bug.
* Right now I'm not sure what WebKit/Blink - based implementations do. Test
results welcome!
As the editor of the Clipboard Events spec, I'm proposing a somewhat
different take on this: cid:-URIs for embeds. See
http://dev.w3.org/2006/webapi/clipops/clipops.html (search for cid:).
The idea is that rather than embedding potentially very huge data: URLs or
reference local files in the embedded markup, we add a reference to the
DataTransferItemList, and use the index of this reference to construct a cid:
URI in the markup that clipboardEvent.getData('text/html') will see. The
script processing this data can then pull out the cid: URIs, do drag-and-drop
style file uploads for referenced clipboard parts, and update the data to
refer to the locations on the server eventually (maybe first using an
intermediate placeholder image or something like that.)
AFAIK, outside of the used in HTML intended for E-mail, this would be the
first usage of CID: URIs in web platform specs. I'm looking for feedback
regarding whether this is implementable and a good solution. I haven't had
much (if any) feedback from implementors on this issue yet, so thank you all
in advance for your ideas and input.
-Hallvord
[1]
http://blogs.msdn.com/b/ie/archive/2013/10/24/enhanced-rich-editing-experiences-in-ie11.aspx
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=665341
