Re: Enable compression of a blob to .zip file
On 2011-12-03 19:43, Charles Pritchard wrote: ... If there were a mime type that were more appropriate for compressed/non-compressed use, would it simply be: image/svg ? I think the obvious choice would be image/svgz, which should be really short, as it can be defined as gipped-wrapped image/svg+xml. Best regards, Julian
Re: Enable compression of a blob to .zip file
On 12/4/11 12:33 AM, Julian Reschke wrote: On 2011-12-03 19:43, Charles Pritchard wrote: ... If there were a mime type that were more appropriate for compressed/non-compressed use, would it simply be: image/svg ? I think the obvious choice would be image/svgz, which should be really short, as it can be defined as gipped-wrapped image/svg+xml. I'll try to get this continued in a decent manner on www-svg. As for the root of this topic, compressing a blob -- I don't think packaging should be done on the UA side. There are so many different ways that zip archives are presented, if the UA were responsible, I'd still need to go back to using work-arounds when I come across a zip archive that has a flag unrecognized by the UA. I'd like to revisit the issue later, as I would so very much like to see round trip base64 and deflate methods exposed to the scripting environment. -Charles
Re: Enable compression of a blob to .zip file
On 2011-12-03 01:38, Charles Pritchard wrote: ... Yes. So is HTML. What's the benefit of compressing SVG first and then BASE64-encoding it, over having it just URI-escaped, and gzip the whole HTML page (something most servers will automatically do for you)? SVG is primarily an image format. It's more appropriate to compare SVG and SVGZ to BMP and PNG. SVG files may reasonably be compressed by 80%. Yes. And they can, using Content-Encoding (and Transfer-Encoding). SVG files are useful for including inline, in both CSS and JS and HTML, to a lesser extent. My bringing this up is not about HTTP, the HTTP case is already working just fine. It's about all other cases, including Blob uris. Those are not working well at all, and the SVG spec requests that conforming implementations support deflated streams. Where does it say that? There's been a 9 years of lingering bug reports area: https://bugzilla.mozilla.org/show_bug.cgi?id=157514 https://bugs.webkit.org/show_bug.cgi?id=5246 http://www.ietf.org/mail-archive/web/pkix/current/msg27507.html http://lists.w3.org/Archives/Public/www-svg/2011May/0128.html http://code.google.com/p/chromium/issues/detail?id=76968 Indeed. It seems that except Opera all browsers do this right. Again: it can be done, but it should be done correctly. Defining a separate media type is the simplest thing to do here. Image formats have sniffing specified within the HTML5 specs. Even HTML has some sniffing. It seems completely reasonable that SVG be included in this. Adding a separate media type would work, but it's counter to what current specifications call for. It's also unnecessary for the HTTP protocol. We had a very long discussion about the svg+xml registration, because it was a bit vague on compressed SVG. The IETF rightfully (IMHO) insisted on removing the vagueness, because the base spec for +xml (RFC 3023) doesn't allow compression. If it turns out that svgz as a media format *is* needed, then we should define a type for it. ... Best regards, Julian
Re: Enable compression of a blob to .zip file
On 12/3/11 1:16 AM, Julian Reschke wrote: On 2011-12-03 01:38, Charles Pritchard wrote: ... Yes. So is HTML. What's the benefit of compressing SVG first and then BASE64-encoding it, over having it just URI-escaped, and gzip the whole HTML page (something most servers will automatically do for you)? SVG is primarily an image format. It's more appropriate to compare SVG and SVGZ to BMP and PNG. SVG files may reasonably be compressed by 80%. Yes. And they can, using Content-Encoding (and Transfer-Encoding). Those are HTTP semantics. SVG implementations that support HTTP must support [Content-Encoding and Transfer-Encoding]. My difficulties are with non-HTTP semantics. http://www.w3.org/TR/SVG/conform.html SVG files are useful for including inline, in both CSS and JS and HTML, to a lesser extent. My bringing this up is not about HTTP, the HTTP case is already working just fine. It's about all other cases, including Blob uris. Those are not working well at all, and the SVG spec requests that conforming implementations support deflated streams. Where does it say that? SVG implementations must correctly support gzip-encoded [RFC1952] and deflate-encoded [RFC1951] data streams, for any content type (including SVG, script files, images). http://www.w3.org/TR/SVG/conform.html That interpretation, though, doesn't apply as cleanly to my argument as I'd hoped. There's been a 9 years of lingering bug reports area: https://bugzilla.mozilla.org/show_bug.cgi?id=157514 https://bugs.webkit.org/show_bug.cgi?id=5246 http://www.ietf.org/mail-archive/web/pkix/current/msg27507.html http://lists.w3.org/Archives/Public/www-svg/2011May/0128.html http://code.google.com/p/chromium/issues/detail?id=76968 Indeed. It seems that except Opera all browsers do this right. Again: it can be done, but it should be done correctly. Defining a separate media type is the simplest thing to do here. Image formats have sniffing specified within the HTML5 specs. Even HTML has some sniffing. It seems completely reasonable that SVG be included in this. Adding a separate media type would work, but it's counter to what current specifications call for. It's also unnecessary for the HTTP protocol. We had a very long discussion about the svg+xml registration, because it was a bit vague on compressed SVG. The IETF rightfully (IMHO) insisted on removing the vagueness, because the base spec for +xml (RFC 3023) doesn't allow compression. Thank you very much for this information. If it turns out that svgz as a media format *is* needed, then we should define a type for it. Given that Blob has only one mechanism, mime types, and the IETF has rejected any possibility of +xml having compression, I'm going to back off the idea of sniffing the +xml mime type. If there were a mime type that were more appropriate for compressed/non-compressed use, would it simply be: image/svg ? -Charles
Re: Data uris, was: Re: Enable compression of a blob to .zip file
On Fri, Dec 2, 2011 at 6:06 PM, Charles Pritchard ch...@jumis.com wrote:
On 12/2/11 5:41 PM, Jonas Sicking wrote:
On Fri, Dec 2, 2011 at 5:05 PM, Charles Pritchardch...@jumis.com wrote:
On 12/2/11 4:52 PM, Jonas Sicking wrote:
On Fri, Dec 2, 2011 at 4:38 PM, Charles Pritchardch...@jumis.com
wrote:
As far as I can tell, vendors are trying to move away from data uris
and
toward blob uris.
Just to clear something up. I have not heard anything about vendors
trying to move away from data uris.
I didn't mean it in the sense of retiring the protocol. The URL, such
as
createObjectUrl and blob specs get around a lot of the issues present in
data uris. The simple action of copying and pasting a data uri from the
user-side is becoming more difficult.
So what specifically do you mean by trying to move away from data uris?
move away generally involves taking an action. You seem to be using
some alternative meaning?
I'll try to be more precise in the future. I do appreciate your constructive
criticism.
Specifically, vendors are disabling data uri access from the url bar and
replying with WONTFIX statuses to bug reports related to issues with long
data uris
Yes, that is the one area where I've heard of anyone moving away
from data. Though it's not specific to data, but also to javascript.
And possibly also blob in the future if needed. See below.
There have been no steps to expand or otherwise support base64 encoding,
nor
data uris, from a higher level.
What do you mean by this? base64 encoding has been supported in
datauris as long as data uris have been supported, and base64 encoding
in general has been supported far longer than that.
There are no binary base64 encoding methods available to the scripting
environment, there are no new types [btoa and atob already existing methods]
relating to base64 strings. There are no optimizations nor new helper
methods.
Why doesn't btoa and atob count just because they exist? If we exclude
all functions that exist then of course there is no support. For
anything!
Are you perhaps specifically pointing out that there are not built-in
methods that converts between ArrayBuffers and base64 encoded strings?
recently, Chrome started limiting paste into
address bar length, Firefox limited paste into address bar altogether.
This is due to a recent rise of social engineering attacks which
tricked users into pasting unknown contents into the URL bar. It might
apply to to blob uris as much as data uris. Though given that blob
uris still require script to be created, it isn't currently a problem.
But this might change in the future.
Yes, that's exactly what Firefox reports, now. I wouldn't call this a
bug.
It's also an intentional limitation.
What motivations are there for Firefox to block user-copying of blob
urls?
If/when we get to a point when blob-uris can be used to launch social
engineering attacks I don't see why we wouldn't block it. This could
for example happen if websites start running untrusted code in the
context of their origin, for example using technologies like Caja.
Are there means in Mozilla for running untrusted code outside of the context
of their origin?
I don't understand the question.
Thanks for providing the example, I understand that something in Caja might
create a blob url which would contain a same origin script untrusted by the
publisher, and that script would effectively break through the Caja
sandbox. I believe Caja is supposed to prohibit scripts from accessing APIs,
such as createObjectUrl. Regardless, I appreciate you coming up with an
example, and I'll continue to consider what social engineering attacks may
be present.
I'd like to allow users to easily click and drag and copy and paste URIs. I
hope we can still have that with blob:, but I understand that may not be
sustainable.
I don't believe Caja completely blocks access to any API. Instead it
offers a policy framework such that you can choose what APIs that you
want to expose.
The risk appears if a site like facebook were to allow javascript to
execute on their site while also wanting to allow such script to
access the File API. In such a scenario the page could do something
like:
var str = createObjectURL(new Blob(scriptfunction sendToServer(s)
{...} sendToServer(document.cookie)/script));
displayOnScreen(copy the following text to your url bar for free
smiley icons + str);
Of course, no websites do this today. But if it happens in the future
we might need to re-evaluate our policy for typing/pasting blob: urls
in the url-bar. Just like we did with data: and javascript:
At least at mozilla we have in fact been moving in the opposite
direction. Workers recently started supporting data uris and the new
architecture for uri loading specifically tries to make data uris more
consistently working.
There are some interesting semantics with blob and data uris in relation
to
file:/// and essentially
Re: Data uris, was: Re: Enable compression of a blob to .zip file
On 12/3/11 5:51 PM, Jonas Sicking wrote: On Fri, Dec 2, 2011 at 6:06 PM, Charles Pritchardch...@jumis.com wrote: On 12/2/11 5:41 PM, Jonas Sicking wrote: On Fri, Dec 2, 2011 at 5:05 PM, Charles Pritchardch...@jumis.comwrote: There have been no steps to expand or otherwise support base64 encoding, nor data uris, from a higher level. What do you mean by this? base64 encoding has been supported in datauris as long as data uris have been supported, and base64 encoding in general has been supported far longer than that. There are no binary base64 encoding methods available to the scripting environment, there are no new types [btoa and atob already existing methods] relating to base64 strings. There are no optimizations nor new helper methods. Why doesn't btoa and atob count just because they exist? If we exclude all functions that exist then of course there is no support. For anything! They don't count because they hurt my feelings every time I use them with binary data. And they're not new. Are you perhaps specifically pointing out that there are not built-in methods that converts between ArrayBuffers and base64 encoded strings? The methods convert DOMString. They fall apart across implementations when binary data is used from String. I was wrong in claiming there were no new types relating to base64 strings. FileReader.readAsDataURL does offer improved data: uri support for encoding binary data. recently, Chrome started limiting paste into address bar length, Firefox limited paste into address bar altogether. This is due to a recent rise of social engineering attacks which tricked users into pasting unknown contents into the URL bar. It might apply to to blob uris as much as data uris. Though given that blob uris still require script to be created, it isn't currently a problem. But this might change in the future. Yes, that's exactly what Firefox reports, now. I wouldn't call this a bug. It's also an intentional limitation. What motivations are there for Firefox to block user-copying of blob urls? If/when we get to a point when blob-uris can be used to launch social engineering attacks I don't see why we wouldn't block it. This could for example happen if websites start running untrusted code in the context of their origin, for example using technologies like Caja. Are there means in Mozilla for running untrusted code outside of the context of their origin? I don't understand the question. Sorry I asked it without more information, and more study. iframe sandbox was specifically created to run outside of the current origin and it has broad support. So, my question is dated, moot. Mozilla already supports the it. Background: The following nastiness is currently tolerated by Webkit and Mozilla, it appears: iframe.src = data:text/html,meta http-equiv='refresh' content=\0;URL='data:text/html,object /objectscriptalert(31337);/script'\; That breaks out of the current origin and goes into what amounts to a null origin. localStorage will throw an error in Webkit, but is tolerated in Mozilla (though I doubt it does much). iframe sandbox allows authors to run untrusted code outside of the current origin, in a safe manner. iframe sandbox=allow-scripts Microsoft does not allow data uris to be used as the source of an iframe. Microsoft is supporting iframe sandbox http://msdn.microsoft.com/en-us/library/hh673561.aspx I suspect vendors may retire data uris for iframe, in time. Vendors may suspect that users will retire IE, in time. But that's a different topic. As an aside, there's a good example here of using Blob urls for web workers, instead of data uris: http://www.html5rocks.com/en/tutorials/workers/basics/ Afaik, Microsoft has not yet chimed in about Blob urls and iframe. Thanks for providing the example, I understand that something in Caja might create a blob url which would contain a same origin script untrusted by the publisher, and that script would effectively break through the Caja sandbox. I believe Caja is supposed to prohibit scripts from accessing APIs, such as createObjectUrl. Regardless, I appreciate you coming up with an example, and I'll continue to consider what social engineering attacks may be present. I'd like to allow users to easily click and drag and copy and paste URIs. I hope we can still have that with blob:, but I understand that may not be sustainable. I don't believe Caja completely blocks access to any API. Instead it offers a policy framework such that you can choose what APIs that you want to expose. The risk appears if a site like facebook were to allow javascript to execute on their site while also wanting to allow such script to access the File API. In such a scenario the page could do something like: Wouldn't they simply limit access to the URL object, so that URL.createObjectURL can not be run? Or would they keep access on, so authors can use it? As I've noted, in Webkit, it seems that blob: urls can not
Re: Enable compression of a blob to .zip file
On 2011-11-30 19:42, Charles Pritchard wrote: On 11/30/2011 8:04 AM, Julian Reschke wrote: On 2011-11-30 16:50, Charles Pritchard wrote: Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code. It would mean that the semantics of a data URI depends on who's processing it. It would probably also cause lots of confusion about what types is applies to. It's already the case that data URIs depend on UA quirks. There's no reason to add more quirks. Instead we should try to remove the quirks. SVG support is highly implementation dependent. This issue would apply to one type, SVG. It's feature detectable through img src events. This would greatly improve the ability to use data:uris for SVG content. SVG can be highly compressible. Yes. So is HTML. What's the benefit of compressing SVG first and then BASE64-encoding it, over having it just URI-escaped, and gzip the whole HTML page (something most servers will automatically do for you)? There's been a 9 years of lingering bug reports area: https://bugzilla.mozilla.org/show_bug.cgi?id=157514 https://bugs.webkit.org/show_bug.cgi?id=5246 http://www.ietf.org/mail-archive/web/pkix/current/msg27507.html http://lists.w3.org/Archives/Public/www-svg/2011May/0128.html http://code.google.com/p/chromium/issues/detail?id=76968 Indeed. It seems that except Opera all browsers do this right. Again: it can be done, but it should be done correctly. Defining a separate media type is the simplest thing to do here. ... Best regards, Julian
Re: Enable compression of a blob to .zip file
On 12/2/11 5:22 AM, Julian Reschke wrote: On 2011-11-30 19:42, Charles Pritchard wrote: On 11/30/2011 8:04 AM, Julian Reschke wrote: On 2011-11-30 16:50, Charles Pritchard wrote: Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code. It would mean that the semantics of a data URI depends on who's processing it. It would probably also cause lots of confusion about what types is applies to. It's already the case that data URIs depend on UA quirks. There's no reason to add more quirks. Instead we should try to remove the quirks. This in no way changes the scheme of data URIs. Data uri quirks are mainly about string length. As far as I can tell, vendors are trying to move away from data uris and toward blob uris. IE has string length issues; recently, Chrome started limiting paste into address bar length, Firefox limited paste into address bar altogether. Webkit can not cope with data uris in any copy/paste text field (input type=text / textarea) if they are more than ~20k and have no spaces. These issues have nothing to do with SVG in context. SVG support is highly implementation dependent. This issue would apply to one type, SVG. It's feature detectable through img src events. This would greatly improve the ability to use data:uris for SVG content. SVG can be highly compressible. Yes. So is HTML. What's the benefit of compressing SVG first and then BASE64-encoding it, over having it just URI-escaped, and gzip the whole HTML page (something most servers will automatically do for you)? SVG is primarily an image format. It's more appropriate to compare SVG and SVGZ to BMP and PNG. SVG files may reasonably be compressed by 80%. SVG files are useful for including inline, in both CSS and JS and HTML, to a lesser extent. My bringing this up is not about HTTP, the HTTP case is already working just fine. It's about all other cases, including Blob uris. Those are not working well at all, and the SVG spec requests that conforming implementations support deflated streams. There's been a 9 years of lingering bug reports area: https://bugzilla.mozilla.org/show_bug.cgi?id=157514 https://bugs.webkit.org/show_bug.cgi?id=5246 http://www.ietf.org/mail-archive/web/pkix/current/msg27507.html http://lists.w3.org/Archives/Public/www-svg/2011May/0128.html http://code.google.com/p/chromium/issues/detail?id=76968 Indeed. It seems that except Opera all browsers do this right. Again: it can be done, but it should be done correctly. Defining a separate media type is the simplest thing to do here. Image formats have sniffing specified within the HTML5 specs. Even HTML has some sniffing. It seems completely reasonable that SVG be included in this. Adding a separate media type would work, but it's counter to what current specifications call for. It's also unnecessary for the HTTP protocol. I don't think either way is correct, but I hope that this issue is looked at. And I do think that SVGZ is a very reasonable use case for exposing a deflate method to the scripting environment. It would have been very simply to support SVGZ in non-HTTP contexts by simply looking at a few magic bytes and processing things from there. It's a shame that didn't happen. Afaik, there will not be progress on this issue any time soon. -Charles
Re: Enable compression of a blob to .zip file
On Fri, Dec 2, 2011 at 4:38 PM, Charles Pritchard ch...@jumis.com wrote: On 12/2/11 5:22 AM, Julian Reschke wrote: On 2011-11-30 19:42, Charles Pritchard wrote: On 11/30/2011 8:04 AM, Julian Reschke wrote: On 2011-11-30 16:50, Charles Pritchard wrote: Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code. It would mean that the semantics of a data URI depends on who's processing it. It would probably also cause lots of confusion about what types is applies to. It's already the case that data URIs depend on UA quirks. There's no reason to add more quirks. Instead we should try to remove the quirks. This in no way changes the scheme of data URIs. Data uri quirks are mainly about string length. As far as I can tell, vendors are trying to move away from data uris and toward blob uris. Just to clear something up. I have not heard anything about vendors trying to move away from data uris. IE has string length issues; Bugs are in general not a sign of moving away from a technology. It's rather a sign that there isn't a comprehensive test suite that the vendor is using. And last I checked IEs length issues were a generic uri limitation. Nothing data-uri specific. recently, Chrome started limiting paste into address bar length, Firefox limited paste into address bar altogether. This is due to a recent rise of social engineering attacks which tricked users into pasting unknown contents into the URL bar. It might apply to to blob uris as much as data uris. Though given that blob uris still require script to be created, it isn't currently a problem. But this might change in the future. Webkit can not cope with data uris in any copy/paste text field (input type=text / textarea) if they are more than ~20k and have no spaces. Again, just seems like a bug. Not a intended decision to move away from data uris. Also, is webkit really parsing uri contents of text fields?? That seems like a big waste. Is this not just a limitation in number of characters pasted? At least at mozilla we have in fact been moving in the opposite direction. Workers recently started supporting data uris and the new architecture for uri loading specifically tries to make data uris more consistently working. / Jonas
Re: Data uris, was: Re: Enable compression of a blob to .zip file
On Fri, Dec 2, 2011 at 5:05 PM, Charles Pritchard ch...@jumis.com wrote: On 12/2/11 4:52 PM, Jonas Sicking wrote: On Fri, Dec 2, 2011 at 4:38 PM, Charles Pritchardch...@jumis.com wrote: On 12/2/11 5:22 AM, Julian Reschke wrote: On 2011-11-30 19:42, Charles Pritchard wrote: On 11/30/2011 8:04 AM, Julian Reschke wrote: On 2011-11-30 16:50, Charles Pritchard wrote: Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code. It would mean that the semantics of a data URI depends on who's processing it. It would probably also cause lots of confusion about what types is applies to. It's already the case that data URIs depend on UA quirks. There's no reason to add more quirks. Instead we should try to remove the quirks. This in no way changes the scheme of data URIs. Data uri quirks are mainly about string length. As far as I can tell, vendors are trying to move away from data uris and toward blob uris. Just to clear something up. I have not heard anything about vendors trying to move away from data uris. I didn't mean it in the sense of retiring the protocol. The URL, such as createObjectUrl and blob specs get around a lot of the issues present in data uris. The simple action of copying and pasting a data uri from the user-side is becoming more difficult. So what specifically do you mean by trying to move away from data uris? move away generally involves taking an action. You seem to be using some alternative meaning? There have been no steps to expand or otherwise support base64 encoding, nor data uris, from a higher level. What do you mean by this? base64 encoding has been supported in datauris as long as data uris have been supported, and base64 encoding in general has been supported far longer than that. IE has string length issues; Bugs are in general not a sign of moving away from a technology. It's rather a sign that there isn't a comprehensive test suite that the vendor is using. And last I checked IEs length issues were a generic uri limitation. Nothing data-uri specific. It's an intentional limitation, and though it's certainly not data-uri specific (neither are the string length limitations of webkit), they are absolutely and primarily an issue in the use of data uris. I don't think it's fair to call it a bug. Indeed, but in the context you were using this it sounded like you were saying that this was an active step taken by IE. The limitation has always been there. recently, Chrome started limiting paste into address bar length, Firefox limited paste into address bar altogether. This is due to a recent rise of social engineering attacks which tricked users into pasting unknown contents into the URL bar. It might apply to to blob uris as much as data uris. Though given that blob uris still require script to be created, it isn't currently a problem. But this might change in the future. Yes, that's exactly what Firefox reports, now. I wouldn't call this a bug. It's also an intentional limitation. What motivations are there for Firefox to block user-copying of blob urls? If/when we get to a point when blob-uris can be used to launch social engineering attacks I don't see why we wouldn't block it. This could for example happen if websites start running untrusted code in the context of their origin, for example using technologies like Caja. Webkit can not cope with data uris in any copy/paste text field (input type=text / textarea) if they are more than ~20k and have no spaces. Again, just seems like a bug. Not a intended decision to move away from data uris. And a third browser, out of the top three, where data uris are less-operable. It's also intentional, for memory management in widgets. All three of these cases are intentional. I wrote them out to support my statement that vendors are moving away from data uris, opposed to trying to support them in additional manners. failing to move towards data uris at the speed you like is not the same as moving away from data uris. Additionally having limitations on string lengths in various situations is not the same as intentionally limiting data uris. I'm sure everybody is hoping for blob urls and dataTransfer to continue to gain more features. Also, is webkit really parsing uri contents of text fields?? That seems like a big waste. Is this not just a limitation in number of characters pasted? It's just a limitation of characters copied (not pasted). Holy crap man, why on earth would you express this as a limitation in pasting data uris then?? You are bordering on FUD here. Please stop. At least at mozilla we have in fact been moving in the opposite direction. Workers recently started supporting data uris and the new architecture for uri
Re: Data uris, was: Re: Enable compression of a blob to .zip file
On 12/2/11 5:41 PM, Jonas Sicking wrote: On Fri, Dec 2, 2011 at 5:05 PM, Charles Pritchardch...@jumis.com wrote: On 12/2/11 4:52 PM, Jonas Sicking wrote: On Fri, Dec 2, 2011 at 4:38 PM, Charles Pritchardch...@jumis.comwrote: As far as I can tell, vendors are trying to move away from data uris and toward blob uris. Just to clear something up. I have not heard anything about vendors trying to move away from data uris. I didn't mean it in the sense of retiring the protocol. The URL, such as createObjectUrl and blob specs get around a lot of the issues present in data uris. The simple action of copying and pasting a data uri from the user-side is becoming more difficult. So what specifically do you mean by trying to move away from data uris? move away generally involves taking an action. You seem to be using some alternative meaning? I'll try to be more precise in the future. I do appreciate your constructive criticism. Specifically, vendors are disabling data uri access from the url bar and replying with WONTFIX statuses to bug reports related to issues with long data uris There have been no steps to expand or otherwise support base64 encoding, nor data uris, from a higher level. What do you mean by this? base64 encoding has been supported in datauris as long as data uris have been supported, and base64 encoding in general has been supported far longer than that. There are no binary base64 encoding methods available to the scripting environment, there are no new types [btoa and atob already existing methods] relating to base64 strings. There are no optimizations nor new helper methods. That said, I know that Chrome displaying data:, truncated in the URL helps with performance and Webkit recently fixed a multi-year memory leak with referencing data uris in images. IE has string length issues; Bugs are in general not a sign of moving away from a technology. It's rather a sign that there isn't a comprehensive test suite that the vendor is using. And last I checked IEs length issues were a generic uri limitation. Nothing data-uri specific. It's an intentional limitation, and though it's certainly not data-uri specific (neither are the string length limitations of webkit), they are absolutely and primarily an issue in the use of data uris. I don't think it's fair to call it a bug. Indeed, but in the context you were using this it sounded like you were saying that this was an active step taken by IE. The limitation has always been there. Inaction has some meaning when reasons for action are pointed out and dismissed (as in, WONTFIX). I'm aware of the buffer size limitation in URL handlers in Windows. This sub-thread continues my assertion that data uris are already quite quirky with existing browsers. I don't think that data uri is responsible for the issues with SVGZ support in UAs. recently, Chrome started limiting paste into address bar length, Firefox limited paste into address bar altogether. This is due to a recent rise of social engineering attacks which tricked users into pasting unknown contents into the URL bar. It might apply to to blob uris as much as data uris. Though given that blob uris still require script to be created, it isn't currently a problem. But this might change in the future. Yes, that's exactly what Firefox reports, now. I wouldn't call this a bug. It's also an intentional limitation. What motivations are there for Firefox to block user-copying of blob urls? If/when we get to a point when blob-uris can be used to launch social engineering attacks I don't see why we wouldn't block it. This could for example happen if websites start running untrusted code in the context of their origin, for example using technologies like Caja. Are there means in Mozilla for running untrusted code outside of the context of their origin? Thanks for providing the example, I understand that something in Caja might create a blob url which would contain a same origin script untrusted by the publisher, and that script would effectively break through the Caja sandbox. I believe Caja is supposed to prohibit scripts from accessing APIs, such as createObjectUrl. Regardless, I appreciate you coming up with an example, and I'll continue to consider what social engineering attacks may be present. I'd like to allow users to easily click and drag and copy and paste URIs. I hope we can still have that with blob:, but I understand that may not be sustainable. Webkit can not cope with data uris in any copy/paste text field (input type=text / textarea) if they are more than ~20k and have no spaces. Again, just seems like a bug. Not a intended decision to move away from data uris. And a third browser, out of the top three, where data uris are less-operable. It's also intentional, for memory management in widgets. All three of these cases are intentional. I wrote them out to support my statement that vendors are moving away from data uris,
Re: Enable compression of a blob to .zip file
Fyi - it's been a long time request from Zynga to have a web packaging format to serve and store assets in large chunks. A relevant ticket with a discussion can be found here: https://bugzilla.mozilla.org/show_bug.cgi?id=681967 I'm all for it, of course. Am 31.10.11 12:20 schrieb Charles Pritchard unter ch...@jumis.com: On 10/31/11 11:34 AM, Boris Zbarsky wrote: On 10/31/11 1:42 PM, Charles Pritchard wrote: I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. I don't believe it is. In fact, doing that would contradict the specs as they stand, to my knowledge. I could be wrong, of course, but in that case citation needed... -Boris Well I've failed to find a citation. I will post one if I find it. In the meantime: Blob and Data uris can and should have sniffing for image/svg+xml. It's trivial, and it makes a whole lot more sense than extending both the Blob and data uri specs to include transfer encoding semantics. file: and filesystem: and widget urls are items that -might- be supported on an OS level, and thus out of scope here. Back to deflate use cases: PDF.js I'm sure implements deflate (for PDF FlateEncode), I've recently done some docx work which required deflate. Many servers do not host .svgz files well, and I use XHR with deflate to deal with that (though I would have just used blob urls if they worked). localStorage, indexedDB and WebSQL all require DOMString, as do most WebSocket implementations -- in practical use, this means base64 encoding data. It's another area where deflate could squeeze a few more bytes out of existing constraints. Especially with localStorage. As we continue to support more-and-more document formats, deflate support on the client side becomes more important. Apple and MS, two very large vendors, have pushed ahead with using deflate in various file formats they use. Adobe has been doing it for some time. Though HTTP works quite well for negotiated compression there are more and more cases that are not http bound. I would very much like to see deflate and inflate exposed to the scripting environment. From there I can easily interact with several file formats. Zip included. -Charles -Charles
Re: Enable Compression Of A Blob To .zip File
It would be great to have a native binding to Zlib and Snappy exposed to Javascript in the browser. Zlib covers the expensive disk use-cases, Snappy covers the expensive CPU use-cases. Also a native binding to basic crypto primitives, even if that means just SHA1 to start, and even if the Node.js crypto api is copied verbatim. TypedArrays are in current implementations are too slow to help with these, as far as I have tried.
Re: Enable compression of a blob to .zip file
On 2011-10-31 18:42, Charles Pritchard wrote: On 10/30/2011 5:56 PM, Cameron McCormack wrote: On 30/10/11 4:25 PM, Bjoern Hoehrmann wrote: Then he probably means file system files and not HTTP files, and support there has indeed been spotty in the past. Ah, yes. Regarding data: URIs, someone should really just amend the RFC to allow specifying a content encoding. That'll be tricky with respect to backwards compatibility; but it could be worth a try. That seems rather unnecessary. I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. It's minimal work. This also applies to the local file system; as in file:///. Hm, no. It's not specified, and seems like a bad idea to do. Best regards, Julian
Re: Enable compression of a blob to .zip file
On 2011-10-31 20:20, Charles Pritchard wrote: On 10/31/11 11:34 AM, Boris Zbarsky wrote: On 10/31/11 1:42 PM, Charles Pritchard wrote: I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. I don't believe it is. In fact, doing that would contradict the specs as they stand, to my knowledge. I could be wrong, of course, but in that case citation needed... -Boris Well I've failed to find a citation. I will post one if I find it. In the meantime: Blob and Data uris can and should have sniffing for image/svg+xml. ... Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Best regards, Julian
Re: Enable compression of a blob to .zip file
On Nov 30, 2011, at 7:05 AM, Julian Reschke julian.resc...@gmx.de wrote: On 2011-10-31 20:20, Charles Pritchard wrote: On 10/31/11 11:34 AM, Boris Zbarsky wrote: On 10/31/11 1:42 PM, Charles Pritchard wrote: I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. I don't believe it is. In fact, doing that would contradict the specs as they stand, to my knowledge. I could be wrong, of course, but in that case citation needed... -Boris Well I've failed to find a citation. I will post one if I find it. In the meantime: Blob and Data uris can and should have sniffing for image/svg+xml. ... Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code.
Re: Enable compression of a blob to .zip file
On 2011-11-30 16:50, Charles Pritchard wrote: Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code. It would mean that the semantics of a data URI depends on who's processing it. It would probably also cause lots of confusion about what types is applies to. Best regards, Julian
Re: Enable compression of a blob to .zip file
On 11/30/2011 8:04 AM, Julian Reschke wrote: On 2011-11-30 16:50, Charles Pritchard wrote: Nope. If you need gzipped SVG in data URIs, the right thing to do is to either extend data URIs to support that, or to mint a separate media type. Why? Seems like a lot of complexity for blob, data and file for something that could otherwise be handled by minimal code. It would mean that the semantics of a data URI depends on who's processing it. It would probably also cause lots of confusion about what types is applies to. It's already the case that data URIs depend on UA quirks. SVG support is highly implementation dependent. This issue would apply to one type, SVG. It's feature detectable through img src events. This would greatly improve the ability to use data:uris for SVG content. SVG can be highly compressible. There's been a 9 years of lingering bug reports area: https://bugzilla.mozilla.org/show_bug.cgi?id=157514 https://bugs.webkit.org/show_bug.cgi?id=5246 http://www.ietf.org/mail-archive/web/pkix/current/msg27507.html http://lists.w3.org/Archives/Public/www-svg/2011May/0128.html http://code.google.com/p/chromium/issues/detail?id=76968 There are all sorts of reasons to do it. It's a constant bummer that we need to include inflate code just to use inline svgz. This issue also comes up with Blob. It could have been fixed with the most minor of patches, years ago. I think that there are some theoretical purity issues here that have really things held-back. I don't need to revise the entire data: uri, I'm just looking to add a note for content-type sniffing of the first few bytes of a file, when SVG is loaded from a non-http resource, explicitly, file:, filesystem:, blob: and data:. -Charles
Re: Enable Compression Of A Blob To .zip File
I think crypto is supposed to be in scope of another WG that was being chartered nowish On 11/30/11, Joran Greef jo...@ronomon.com wrote: It would be great to have a native binding to Zlib and Snappy exposed to Javascript in the browser. Zlib covers the expensive disk use-cases, Snappy covers the expensive CPU use-cases. Also a native binding to basic crypto primitives, even if that means just SHA1 to start, and even if the Node.js crypto api is copied verbatim. TypedArrays are in current implementations are too slow to help with these, as far as I have tried. -- Sent from my mobile device
Re: Enable compression of a blob to .zip file
On 10/30/2011 5:56 PM, Cameron McCormack wrote: On 30/10/11 4:25 PM, Bjoern Hoehrmann wrote: Then he probably means file system files and not HTTP files, and support there has indeed been spotty in the past. Ah, yes. Regarding data: URIs, someone should really just amend the RFC to allow specifying a content encoding. That seems rather unnecessary. I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. It's minimal work. This also applies to the local file system; as in file:///.
Re: Enable compression of a blob to .zip file
On Oct 31, 2011, at 10:42 AM, Charles Pritchard wrote: On 10/30/2011 5:56 PM, Cameron McCormack wrote: On 30/10/11 4:25 PM, Bjoern Hoehrmann wrote: Then he probably means file system files and not HTTP files, and support there has indeed been spotty in the past. Ah, yes. Regarding data: URIs, someone should really just amend the RFC to allow specifying a content encoding. That seems rather unnecessary. I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. It's minimal work. Work in progress: http://mimesniff.spec.whatwg.org/ mh
Re: Enable compression of a blob to .zip file
On 31/10/11 10:50 AM, Mike Hanson wrote: Work in progress: http://mimesniff.spec.whatwg.org/ There's a section on Images in there which looks like it won't do sniffing for automatic gunzipping. Although perhaps if the SVG document were used as the top level document it would be, if it falls under the Web Pages case.
Re: Enable compression of a blob to .zip file
On 10/31/11 1:42 PM, Charles Pritchard wrote: I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. I don't believe it is. In fact, doing that would contradict the specs as they stand, to my knowledge. I could be wrong, of course, but in that case citation needed... -Boris
Re: Enable compression of a blob to .zip file
On 10/31/11 11:34 AM, Boris Zbarsky wrote: On 10/31/11 1:42 PM, Charles Pritchard wrote: I'm almost certain that somewhere, it is specified that the browser should sniff the first few bytes of the file to see if it is compressed. I don't believe it is. In fact, doing that would contradict the specs as they stand, to my knowledge. I could be wrong, of course, but in that case citation needed... -Boris Well I've failed to find a citation. I will post one if I find it. In the meantime: Blob and Data uris can and should have sniffing for image/svg+xml. It's trivial, and it makes a whole lot more sense than extending both the Blob and data uri specs to include transfer encoding semantics. file: and filesystem: and widget urls are items that -might- be supported on an OS level, and thus out of scope here. Back to deflate use cases: PDF.js I'm sure implements deflate (for PDF FlateEncode), I've recently done some docx work which required deflate. Many servers do not host .svgz files well, and I use XHR with deflate to deal with that (though I would have just used blob urls if they worked). localStorage, indexedDB and WebSQL all require DOMString, as do most WebSocket implementations -- in practical use, this means base64 encoding data. It's another area where deflate could squeeze a few more bytes out of existing constraints. Especially with localStorage. As we continue to support more-and-more document formats, deflate support on the client side becomes more important. Apple and MS, two very large vendors, have pushed ahead with using deflate in various file formats they use. Adobe has been doing it for some time. Though HTTP works quite well for negotiated compression there are more and more cases that are not http bound. I would very much like to see deflate and inflate exposed to the scripting environment. From there I can easily interact with several file formats. Zip included. -Charles -Charles
Enable compression of a blob to .zip file
Hi all I've tried to search the Webapps and WHATWG mailing lists for previous discussions about this request, but I haven't been able to find anything. My request can be explained easily: when uploading files to the server, many times the upload time can be improved greatly if the file is compressed instead of sent as-is; in my case I've adjusted some systems that requires the user to upload a data file so it can accept both the original file (mdb, csv, .xls, ), or a compressed version in .zip When the user remembers to compress the zip everything goes fine, but many times people might not be aware of that option, and so I think that it would be useful to have an option (whatever objects or proposal about methods are just quick ideas to explain the concept) on a Blob object to compress it to a .zip and so get a new Blob So the flow would be 1. User picks his data file with input type=file 2. JS starts a XHR upload to show a progress bar 3. JS checks that the file is uncompressed, so it creates a zipped blob: var file = input.files[0]; var extension = file.name.match( /\.([^\.]+)\s*$/ )[ 1 ]; if (extension.toLowerCase() != 'zip') file = file.toZip( 'compressed.zip' ); formData.append( file ); etc... 4. And the rest of the upload goes as usual. 5. The server gets the file and uncompresses it (the server doesn't really care if the file was compressed in js or because the user did it, but the js code can add a flag in the xhr upload to signal that it was autocompressed. Instead of a method on a Blob, it could be a separate object similar to Mozilla nsIZipWriter (https://developer.mozilla.org/en/nsIZipWriter), but with some simplifications to avoid extra verbose code. var zipWriter = new zipWriter( 'compressed.zip' ); // This way it could be possible to add several files into the zip zipWriter.addEntryFile( file); formData.append( zipWriter ); etc... Of course, in the files should be always added at the root of the zip without any info about the original path. The compression can (and probably should) be done asynchronously. Would it be possible to add this new feature to some spec? Thanks.
Re: Enable compression of a blob to .zip file
On 10/30/11 3:03 AM, Alfonso Martínez de Lizarrondo wrote: Instead of a method on a Blob, it could be a separate object similar to Mozilla nsIZipWriter (https://developer.mozilla.org/en/nsIZipWriter), but with some simplifications to avoid extra verbose code. var zipWriter = new zipWriter( 'compressed.zip' ); // This way it could be possible to add several files into the zip zipWriter.addEntryFile( file); formData.append( zipWriter ); etc... Of course, in the files should be always added at the root of the zip without any info about the original path. The compression can (and probably should) be done asynchronously. Would it be possible to add this new feature to some spec? I've repeatedly come across the need to have inflate support. There are a handful of libraries on the net implementing inflate / deflate. Licensing on them is spotty. One reason I've needed inflate is for svgz support. Browser vendors have consistently left bugs and/or ignored the spec for handling svgz files. SVG is really intended to be deflated. If we had basic inflate/deflate, it'd be trivial to support .zip, .docx and various other zip derivatives. As authors, we could also decide when to compress Blobs prior to storage. I give a big +1 to inflate/deflate. I'd also like support for base64 encoding and decoding of binary strings. Currently, base64 encoding is for DOMString and non-utf8 characters will lead to the browser throwing an error. There are work-arounds, such as using FileReader and Blob, they are indirect, practically obfuscated. I've been using a BlobString method to handle most of this work in my projects. It'd be great to have a spec to follow, so I can just use them as polyfill. These items don't require much for browser vendors. They already have support for inflate/deflate and base64 encoding. The issue, the work, is all about writing up a spec. -Charles
Re: Enable compression of a blob to .zip file
On 30/10/11 10:54 AM, Charles Pritchard wrote: One reason I've needed inflate is for svgz support. Browser vendors have consistently left bugs and/or ignored the spec for handling svgz files. SVG is really intended to be deflated. All major browsers have support for gzipped SVG documents through correct Content-Encoding headers, as far as I'm aware. gzipped SVG documents served as image/svg+xml without Content-Encoding:gzip should be rejected, according to the spec.
Re: Enable compression of a blob to .zip file
On 10/30/11 4:11 PM, Cameron McCormack wrote: On 30/10/11 10:54 AM, Charles Pritchard wrote: One reason I've needed inflate is for svgz support. Browser vendors have consistently left bugs and/or ignored the spec for handling svgz files. SVG is really intended to be deflated. All major browsers have support for gzipped SVG documents through correct Content-Encoding headers, as far as I'm aware. gzipped SVG documents served as image/svg+xml without Content-Encoding:gzip should be rejected, according to the spec. I've had issues with data uris and locally stored files in WebKit. -Charles
Re: Enable compression of a blob to .zip file
* Cameron McCormack wrote: On 30/10/11 10:54 AM, Charles Pritchard wrote: One reason I've needed inflate is for svgz support. Browser vendors have consistently left bugs and/or ignored the spec for handling svgz files. SVG is really intended to be deflated. All major browsers have support for gzipped SVG documents through correct Content-Encoding headers, as far as I'm aware. gzipped SVG documents served as image/svg+xml without Content-Encoding:gzip should be rejected, according to the spec. Then he probably means file system files and not HTTP files, and support there has indeed been spotty in the past. -- Björn Höhrmann · mailto:bjo...@hoehrmann.de · http://bjoern.hoehrmann.de Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de 25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/
Re: Enable compression of a blob to .zip file
On 30/10/11 4:25 PM, Bjoern Hoehrmann wrote: Then he probably means file system files and not HTTP files, and support there has indeed been spotty in the past. Ah, yes. Regarding data: URIs, someone should really just amend the RFC to allow specifying a content encoding.
