Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Siterer Paul Libbrecht p...@hoplahup.net: Hm.. you mean a page should be able to say Hello web browser, I just lve processing application/pdf data from the clipboard ? I don't see the use case for that. If I'm misunderstanding you please clarify :-) It means that if pdf is in the clipboard the paste command can stop being greyed out! Aha. I keep forgetting the UI part of this, thanks for the reminder and clarification ;-) I think that the best way to address this is to spec the before(cut|copy|paste) events, let them be real ClipboardEvents (no idea what interface they use in IE and Chrome at the moment but I think they are so rarely used we can rework them a little) and say that clipboardData.types should be populated according to real supported types on clipboard in the beforepaste event. Hence, a page that can handle PDF data being pasted can listen for beforepaste and check if event.clipboardData.types contains application/pdf. (Just my preliminary thoughts, nothing about before* events is spec'ed yet and actually I don't think they are an elegant solution. Perhaps a clipboardchange event would be more elegant? Clipboard change listener could do: if(event.clipboardData.types.indexOf('application/pdf')))event.clipboarData.enableUI('paste'); Or something like that? We could also define DataTransfer.enableUI('copy')) to be called at any time without any event listener workarounds..) Similarly, shouldn't something such as if(event.clipboardData.isTypeSupported('foo/bar')) ... be callable before the event is triggered? That was my intention originally, but I'm also worried about fingerprinting potential when letting scripts access new data about an end user's config. Hence, I'm now intending to limit it to being called inside event listeners like the other DataTransfer methods. I also think that if the event is synthetic the method should not reveal actual config details. /Hallvord
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
On Sat, Feb 18, 2012 at 07:28, Hallvord R. M. Steen hallv...@opera.comwrote: On Fri, 17 Feb 2012 19:23:29 +0100, Daniel Cheng dch...@chromium.org wrote: Also, what does it mean to be supported? In new versions of Chrome, any kind of MIME type is supported in the sense that you can set data for any arbitrary type, and it can be understood by any browser that uses the same native conventions (I'd be happy to work with any other developers interested in making sure this works across different browsers on the same computer). That's interesting. How do you do that, on Windows in particular? Have you registered a clipboard format called MIME type of main clipboard part where you store type strings, or something? -- Hallvord R. M. Steen, Core Tester, Opera Software http://www.opera.com http://my.opera.com/hallvors/ Chrome registers a clipboard format on Windows called Chromium Web Custom MIME Data Format (we do the same thing on Mac and Linux at the moment to keep the various implementations similar), and when we see an attempt to add data of a type which we don't have a native translation for, we just place the string in there. We currently have a bug open to figure out how to standardize the serialization, so we can stop using the Chromium-specific types. I'd like to extend it to binary data at some point too. With respect to enabling copy/cut/paste in context menu items, IE/WebKit have beforecut/beforepaste (both) and beforecopy (WebKit only?) events that fire before a context menu is displayed. I'm not sure how widely used they are though. That being said, it'd be nice to have something that works well for native context menus and custom editing controls created with HTML elements like the one at http://ckeditor.com/demo. Daniel
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
On Fri, 17 Feb 2012 19:23:29 +0100, Daniel Cheng dch...@chromium.org wrote: Also, what does it mean to be supported? In new versions of Chrome, any kind of MIME type is supported in the sense that you can set data for any arbitrary type, and it can be understood by any browser that uses the same native conventions (I'd be happy to work with any other developers interested in making sure this works across different browsers on the same computer). I think this is the best solution. The draft should define the intermediate format I think so not just browsers, but also other applications can make use of it (at least in theory). Presumably also applies to drag drop, no? -- Anne van Kesteren http://annevankesteren.nl/
Re: (aside) MIME type (was Re: Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard)
WHEN I registered a media-type on the ietf list I have been quite much hit as the first comment one says media-type nowadays. And indeed MIME is meant for email originally. So I guess politically media-type is a requirement. Should I dig for a formal requirement? paul Le 18 févr. 2012 à 00:54, Hallvord Reiar Michaelsen Steen a écrit : Hallvord, it should be called media-types btw, or? IMHO the term MIME type is more widely used and also less ambiguous than media type, so I'd definitely prefer using the former if I can get away with it :) -- Hallvord R. M. Steen Core tester, Opera Software
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Does this include an ability for a page to say that a media-type is supported? (does it not appear natural?) I won't brag about native type names. I still find they're good but they're really not well ruled so it's ok. paul Le 18 févr. 2012 à 01:16, Hallvord Reiar Michaelsen Steen a écrit : Having thought about this some more, I see that there is a fingerprinting concern if we tie this closely into the available applications/OS capabilities. Also I understand that the API would be relevant for drag and drop (thought I'm not quite sure how it would work). Hence I think the method should be on the DataTransfer.prototype object, and usable only inside event handlers: if(event.clipboardData.isTypeSupported('foo/bar')) ... Supported means: if you give me data labelled as this type, I know the OS description and (if applicable) the relevant meta data and formatting for this type, and can write it to the clipboard so that other native apps can make use of it. If data labelled with the OS's native name or constant for this type is on the clipboard, I can map it to the mime type and extract the data in a way the script can use. Additionally, if a blob of data in this format is dropped on my window, I'll know what to do about it, and if you drag this data *from* my window to another application on the system, I can format and label it in a way the targe app will understand. Paul Libbrecht wrote: I have one concern: media-types are likely to be insufficient and flavour names, whatever they are on the host platform should be allowed I think. Almost arbitrary strings on Windows and Uniform Type Identifiers on Mac should be allowed, I think. No, I really don't think it is a good idea to make scripts on the web have to sniff the OS to determine what argument to pass to isTypeSupported().. If MIME types aren't sufficient, maybe we need to mint more MIME types as we go? :) -- Hallvord R. M. Steen Core tester, Opera Software
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
On Sat, 18 Feb 2012 13:48:52 +0100, Paul Libbrecht p...@hoplahup.net wrote: Does this include an ability for a page to say that a media-type is supported? (does it not appear natural?) Hm.. you mean a page should be able to say Hello web browser, I just lve processing application/pdf data from the clipboard ? I don't see the use case for that. If I'm misunderstanding you please clarify :-) I won't brag about native type names. I still find they're good but they're really not well ruled so it's ok. MIME types (ahem..) are sort of more web-like.. -- Hallvord R. M. Steen, Core Tester, Opera Software http://www.opera.com http://my.opera.com/hallvors/
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
On Fri, 17 Feb 2012 19:23:29 +0100, Daniel Cheng dch...@chromium.org wrote: Also, what does it mean to be supported? In new versions of Chrome, any kind of MIME type is supported in the sense that you can set data for any arbitrary type, and it can be understood by any browser that uses the same native conventions (I'd be happy to work with any other developers interested in making sure this works across different browsers on the same computer). That's interesting. How do you do that, on Windows in particular? Have you registered a clipboard format called MIME type of main clipboard part where you store type strings, or something? -- Hallvord R. M. Steen, Core Tester, Opera Software http://www.opera.com http://my.opera.com/hallvors/
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Le 18 févr. 2012 à 16:25, Hallvord R. M. Steen a écrit : Does this include an ability for a page to say that a media-type is supported? (does it not appear natural?) Hm.. you mean a page should be able to say Hello web browser, I just lve processing application/pdf data from the clipboard ? I don't see the use case for that. If I'm misunderstanding you please clarify :-) It means that if pdf is in the clipboard the paste command can stop being greyed out! Otherwise the user is left to try and be told by the web-page: please please do it! Similarly, shouldn't something such as if(event.clipboardData.isTypeSupported('foo/bar')) ... be callable before the event is triggered? This way the page can display the necessary invites to the user to make it clear that something can be copied? paul
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
I like the idea of letting authors dynamically detectthe supported mime types. But providing methods on the interface object seems rather unusual. Also, I'm thinking there are cases where the supported mime types change dynamically without reloading the page. - Ryosuke On Feb 17, 2012 5:25 AM, Hallvord R. M. Steen hallv...@opera.com wrote: Hi, I have an open issue noted in the spec - seems no implementor has worked on this yet so I'm throwing it out here to see what you all think. Given that a) We can't give a random script access to every format on the clipboard for security/privacy reasons (per earlier feedback from Chrome/WebKit devs and common sense) b) Certain clipboard formats have meta data or special formatting requirements that mean the implementation needs to do a bit of processing on reading/writing (thinking of Windows' CF_HTML) c) Text may apparently be placed on the clipboard in different encodings, an implementation may have to do conversions d) Clipboard contents can also be file references, in which case we'll make the *file* and not the actual clipboard data available - it's obvious that the implementation needs to do a bit of work to support reading/writing given types of data from/to the native clipboard. Hence, there should be a way a script can check if a clipboard format is registered and whether the implementation allows writing this format to the clipboard. My idea (at the moment) is to define a method on an interface (not on instances, on the interface object itself) like this: ClipboardEvent.**isTypeSupported('mime/type') or, perhaps: DataTransfer.isTypeSupported('**mime/type')?? (but it's more clipboard-specific and maybe it's confusing to mix it with the DnD stuff?) An application can then for example do something like if( ClipboardEvent.**isTypeSupported( 'vnd.supersoft/fancyformat' ) ){ // prepare fancyformat content and place on clipboard }else{ // fall back to for example put xml-data-as-plain-text or json-data-as-plain-text } I'm suggesting to define this method on the interface object because it helps web applications do feature detection early. It could of course be defined on instances addEventListener('copy', function(e){ if( e.isTypeSupported( 'vnd.supersoft/fancyformat' ) ){ // prepare fancyformat content and place on clipboard }else{ // fall back to for example put xml-data-as-plain-text or json-data-as-plain-text } }) but then a feature/compatibility test would have to be deferred until a point where the user potentially has done a lot of work in the web application. Thoughts? -- Hallvord R. M. Steen Core tester, Opera Software
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
I meant to say *if* there are cases. On Feb 17, 2012 9:53 AM, Ryosuke Niwa rn...@webkit.org wrote:
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Hello Hallvord, I think it is a very good idea if such a method would be available from the point of view of a web-app author. I have one concern: media-types are likely to be insufficient and flavour names, whatever they are on the host platform should be allowed I think. Almost arbitrary strings on Windows and Uniform Type Identifiers on Mac should be allowed, I think. Le 17 févr. 2012 à 18:53, Ryosuke Niwa a écrit : I like the idea of letting authors dynamically detectthe supported mime types. But providing methods on the interface object seems rather unusual. It sounds like ok to me. An alternative would be to have a Capabilities object... Also, I'm thinking if there are cases where the supported mime types change dynamically without reloading the page. (I hope I patched correctly) Yes, it does happen: I think I know that in Windows the supported flavour-names depend on the launched applications. On Mac it depends on the applications whose descriptor has been loaded (by the Finder I think, it might also be those that have been launched once). At least an application download and launch can cause a change in the supported media-types of the OS. However, would the browsers be informed of such a change? Would they be able to consider a given type as being safe and not needing a sanitization? paul Le 17 févr. 2012 à 14:23, Hallvord R. M. Steen a écrit : Hi, I have an open issue noted in the spec - seems no implementor has worked on this yet so I'm throwing it out here to see what you all think. Given that a) We can't give a random script access to every format on the clipboard for security/privacy reasons (per earlier feedback from Chrome/WebKit devs and common sense) b) Certain clipboard formats have meta data or special formatting requirements that mean the implementation needs to do a bit of processing on reading/writing (thinking of Windows' CF_HTML) c) Text may apparently be placed on the clipboard in different encodings, an implementation may have to do conversions d) Clipboard contents can also be file references, in which case we'll make the *file* and not the actual clipboard data available - it's obvious that the implementation needs to do a bit of work to support reading/writing given types of data from/to the native clipboard. Hence, there should be a way a script can check if a clipboard format is registered and whether the implementation allows writing this format to the clipboard. My idea (at the moment) is to define a method on an interface (not on instances, on the interface object itself) like this: ClipboardEvent.isTypeSupported('mime/type') or, perhaps: DataTransfer.isTypeSupported('mime/type')?? (but it's more clipboard-specific and maybe it's confusing to mix it with the DnD stuff?) An application can then for example do something like if( ClipboardEvent.isTypeSupported( 'vnd.supersoft/fancyformat' ) ){ // prepare fancyformat content and place on clipboard }else{ // fall back to for example put xml-data-as-plain-text or json-data-as-plain-text } I'm suggesting to define this method on the interface object because it helps web applications do feature detection early. It could of course be defined on instances addEventListener('copy', function(e){ if( e.isTypeSupported( 'vnd.supersoft/fancyformat' ) ){ // prepare fancyformat content and place on clipboard }else{ // fall back to for example put xml-data-as-plain-text or json-data-as-plain-text } }) but then a feature/compatibility test would have to be deferred until a point where the user potentially has done a lot of work in the web application. Thoughts? -- Hallvord R. M. Steen Core tester, Opera Software
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Any MIME type support restrictions that apply to clipboard MIME types will almost certainly apply to DnD MIME types as well. Therefore, it wouldn't make sense to tie it to ClipboardEvent. Also, what does it mean to be supported? In new versions of Chrome, any kind of MIME type is supported in the sense that you can set data for any arbitrary type, and it can be understood by any browser that uses the same native conventions (I'd be happy to work with any other developers interested in making sure this works across different browsers on the same computer). Daniel On Fri, Feb 17, 2012 at 10:10, Paul Libbrecht p...@hoplahup.net wrote: Hello Hallvord, I think it is a very good idea if such a method would be available from the point of view of a web-app author. I have one concern: media-types are likely to be insufficient and flavour names, whatever they are on the host platform should be allowed I think. Almost arbitrary strings on Windows and Uniform Type Identifiers on Mac should be allowed, I think. Le 17 févr. 2012 à 18:53, Ryosuke Niwa a écrit : I like the idea of letting authors dynamically detectthe supported mime types. But providing methods on the interface object seems rather unusual. It sounds like ok to me. An alternative would be to have a Capabilities object... Also, I'm thinking if there are cases where the supported mime types change dynamically without reloading the page. (I hope I patched correctly) Yes, it does happen: I think I know that in Windows the supported flavour-names depend on the launched applications. On Mac it depends on the applications whose descriptor has been loaded (by the Finder I think, it might also be those that have been launched once). At least an application download and launch can cause a change in the supported media-types of the OS. However, would the browsers be informed of such a change? Would they be able to consider a given type as being safe and not needing a sanitization? paul Le 17 févr. 2012 à 14:23, Hallvord R. M. Steen a écrit : Hi, I have an open issue noted in the spec - seems no implementor has worked on this yet so I'm throwing it out here to see what you all think. Given that a) We can't give a random script access to every format on the clipboard for security/privacy reasons (per earlier feedback from Chrome/WebKit devs and common sense) b) Certain clipboard formats have meta data or special formatting requirements that mean the implementation needs to do a bit of processing on reading/writing (thinking of Windows' CF_HTML) c) Text may apparently be placed on the clipboard in different encodings, an implementation may have to do conversions d) Clipboard contents can also be file references, in which case we'll make the *file* and not the actual clipboard data available - it's obvious that the implementation needs to do a bit of work to support reading/writing given types of data from/to the native clipboard. Hence, there should be a way a script can check if a clipboard format is registered and whether the implementation allows writing this format to the clipboard. My idea (at the moment) is to define a method on an interface (not on instances, on the interface object itself) like this: ClipboardEvent.isTypeSupported('mime/type') or, perhaps: DataTransfer.isTypeSupported('mime/type')?? (but it's more clipboard-specific and maybe it's confusing to mix it with the DnD stuff?) An application can then for example do something like if( ClipboardEvent.isTypeSupported( 'vnd.supersoft/fancyformat' ) ){ // prepare fancyformat content and place on clipboard }else{ // fall back to for example put xml-data-as-plain-text or json-data-as-plain-text } I'm suggesting to define this method on the interface object because it helps web applications do feature detection early. It could of course be defined on instances addEventListener('copy', function(e){ if( e.isTypeSupported( 'vnd.supersoft/fancyformat' ) ){ // prepare fancyformat content and place on clipboard }else{ // fall back to for example put xml-data-as-plain-text or json-data-as-plain-text } }) but then a feature/compatibility test would have to be deferred until a point where the user potentially has done a lot of work in the web application. Thoughts? -- Hallvord R. M. Steen Core tester, Opera Software
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
On Fri, Feb 17, 2012 at 10:10 AM, Paul Libbrecht p...@hoplahup.net wrote: I have one concern: media-types are likely to be insufficient and flavour names, whatever they are on the host platform should be allowed I think. Almost arbitrary strings on Windows and Uniform Type Identifiers on Mac should be allowed, I think. Realistically, I don't think we'll ever let the wild Web get/set arbitrary data like that. But maybe we can do that for privileged websites (ones that the user trusts). Le 17 févr. 2012 à 18:53, Ryosuke Niwa Software Engineer Google Inc. a écrit : Also, I'm thinking if there are cases where the supported mime types change dynamically without reloading the page. (I hope I patched correctly) Yes. Yes, it does happen: I think I know that in Windows the supported flavour-names depend on the launched applications. On Mac it depends on the applications whose descriptor has been loaded (by the Finder I think, it might also be those that have been launched once). At least an application download and launch can cause a change in the supported media-types of the OS. Right. These will become problems if we decide to expose all platform types. However, would the browsers be informed of such a change? Would they be able to consider a given type as being safe and not needing a sanitization? I don't think that's possible without some sort of pre-knowledge about how the data is processed. In practice, we always hard-code this kind of information somewhere so I'm even not sure if such an elaborate behavior can be implemented. - Ryosuke
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Le 17 févr. 2012 à 19:23, Daniel Cheng a écrit : Any MIME type support restrictions that apply to clipboard MIME types will almost certainly apply to DnD MIME types as well. Therefore, it wouldn't make sense to tie it to ClipboardEvent. Not sure to understand what lie means. Maybe you mean that it is a lie to use flavour names that are native names? Can you please explain? Also, what does it mean to be supported? In new versions of Chrome, any kind of MIME type is supported in the sense that you can set data for any arbitrary type, and it can be understood by any browser that uses the same native conventions (I'd be happy to work with any other developers interested in making sure this works across different browsers on the same computer). This sounds like a huge security leak. What if trusted web-site docs.google.com receives a paste of easy.cms.com that just got hacked with a hack that a kind html containing a script that reads the cookie when executed? paul
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Le 17 févr. 2012 à 19:25, Ryosuke Niwa a écrit : On Fri, Feb 17, 2012 at 10:10 AM, Paul Libbrecht p...@hoplahup.net wrote: I have one concern: media-types are likely to be insufficient and flavour names, whatever they are on the host platform should be allowed I think. Almost arbitrary strings on Windows and Uniform Type Identifiers on Mac should be allowed, I think. Realistically, I don't think we'll ever let the wild Web get/set arbitrary data like that. But maybe we can do that for privileged websites (ones that the user trusts). I'm easy here. I sure do not want to slow down progress of this spec. I think it makes sense but well... (I find UTIs rather sexy since they have inheritance which media-types do not have). Hallvord, it should be called media-types btw, or? Yes, it does happen: I think I know that in Windows the supported flavour-names depend on the launched applications. On Mac it depends on the applications whose descriptor has been loaded (by the Finder I think, it might also be those that have been launched once). At least an application download and launch can cause a change in the supported media-types of the OS. Right. These will become problems if we decide to expose all platform types. However, would the browsers be informed of such a change? Would they be able to consider a given type as being safe and not needing a sanitization? I don't think that's possible without some sort of pre-knowledge about how the data is processed. In practice, we always hard-code this kind of information somewhere so I'm even not sure if such an elaborate behavior can be implemented. If anyone is willing to consider trusted web-sites (and MSIE already does?) then it is worth included. paul
Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
On Fri, Feb 17, 2012 at 11:15 AM, Paul Libbrecht p...@hoplahup.net wrote: Le 17 févr. 2012 à 19:23, Daniel Cheng a écrit : Any MIME type support restrictions that apply to clipboard MIME types will almost certainly apply to DnD MIME types as well. Therefore, it wouldn't make sense to tie it to ClipboardEvent. Not sure to understand what lie means. tie, not lie. Also, what does it mean to be supported? In new versions of Chrome, any kind of MIME type is supported in the sense that you can set data for any arbitrary type, and it can be understood by any browser that uses the same native conventions (I'd be happy to work with any other developers interested in making sure this works across different browsers on the same computer). This sounds like a huge security leak. What if trusted web-site docs.google.com receives a paste of easy.cms.com that just got hacked with a hack that a kind html containing a script that reads the cookie when executed? In general, it's websites' responsibility to sanitize data they get from getData. On Fri, Feb 17, 2012 at 11:18 AM, Paul Libbrecht p...@hoplahup.net wrote: Le 17 févr. 2012 à 19:25, Ryosuke Niwa a écrit : Yes, it does happen: I think I know that in Windows the supported flavour-names depend on the launched applications. On Mac it depends on the applications whose descriptor has been loaded (by the Finder I think, it might also be those that have been launched once). At least an application download and launch can cause a change in the supported media-types of the OS. Right. These will become problems if we decide to expose all platform types. However, would the browsers be informed of such a change? Would they be able to consider a given type as being safe and not needing a sanitization? I don't think that's possible without some sort of pre-knowledge about how the data is processed. In practice, we always hard-code this kind of information somewhere so I'm even not sure if such an elaborate behavior can be implemented. If anyone is willing to consider trusted web-sites (and MSIE already does?) then it is worth included. Yeah, I think it's worth paying attention to this case. - Ryosuke
(aside) MIME type (was Re: Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard)
Hallvord, it should be called media-types btw, or? IMHO the term MIME type is more widely used and also less ambiguous than media type, so I'd definitely prefer using the former if I can get away with it :) -- Hallvord R. M. Steen Core tester, Opera Software
Re: Re: [Clipboard] checking if implementation allows reading/writing a given type to the OS clipboard
Having thought about this some more, I see that there is a fingerprinting concern if we tie this closely into the available applications/OS capabilities. Also I understand that the API would be relevant for drag and drop (thought I'm not quite sure how it would work). Hence I think the method should be on the DataTransfer.prototype object, and usable only inside event handlers: if(event.clipboardData.isTypeSupported('foo/bar')) ... Supported means: if you give me data labelled as this type, I know the OS description and (if applicable) the relevant meta data and formatting for this type, and can write it to the clipboard so that other native apps can make use of it. If data labelled with the OS's native name or constant for this type is on the clipboard, I can map it to the mime type and extract the data in a way the script can use. Additionally, if a blob of data in this format is dropped on my window, I'll know what to do about it, and if you drag this data *from* my window to another application on the system, I can format and label it in a way the targe app will understand. Paul Libbrecht wrote: I have one concern: media-types are likely to be insufficient and flavour names, whatever they are on the host platform should be allowed I think. Almost arbitrary strings on Windows and Uniform Type Identifiers on Mac should be allowed, I think. No, I really don't think it is a good idea to make scripts on the web have to sniff the OS to determine what argument to pass to isTypeSupported().. If MIME types aren't sufficient, maybe we need to mint more MIME types as we go? :) -- Hallvord R. M. Steen Core tester, Opera Software