Re: [whatwg] Fetch Objects and scripts/stylesheets
On Jul 22, 2014, at 5:33 PM, Ian Hickson i...@hixie.ch wrote: On Tue, 22 Jul 2014, Ben Maurer wrote: To follow this up with a concrete suggestion: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); ... Why not: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ... ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? (whenneeded here is a placeholder. I don't expect to actually go with that. I just used it because it's what I had proposed the last time this came up.) That seems like it'd be no more complicated, but would involve less new API surface (not to mention fewer new ways to shoot yourself in the foot, e.g. getting the 'fetch-as' value wrong), and wouldn't require us to come up with a way to enumerate all the kinds of fetch in an API. That seems like a reasonable approach to the problem at hand. It still poses a question as to how we can add a similar API for other types of resources (videos, images, etc…). Any thoughts on that? - R. Niwa
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Wed, Aug 6, 2014 at 12:24 PM, Anne van Kesteren ann...@annevk.nl wrote: I filed https://www.w3.org/Bugs/Public/show_bug.cgi?id=26533 on investigating whether we can reuse the Request object (which is passed to fetch(), and used in service workers to expose requests from a window or worker). There are some complications given what can be set through HTML and what could be set through script and how to keep those best synchronized. We should be able to figure something out I think. Not sure how high of a priority this is though. As an update, Ian proposed a more concrete plan in the bug that I think will work. I recommend people copy themselves to the bug if they want to give further feedback. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Thu, Aug 14, 2014 at 5:28 PM, Nils Dagsson Moskopp n...@dieweltistgarnichtso.net wrote: Ben Maurer ben.mau...@gmail.com writes: Another concrete example with img tags: sometimes an abusive user will use a site like Facebook as a CDN -- they'll upload a picture and hotlink it from elsewhere. We could insert a time-stamped authentication token as a custom header. Today we sometimes do this via the query string -- giving the user a token that lasts for a few days. This means we bust the user's cache every time we rotate the token. With a custom header, the browser cache stays in tact. Why not just check the referer or origin header and act on that? That is not tied to the user. Images would also be a great example of where logging headers could be extremely helpful. For example, we could log what module within a page rendered an image and monitor bandwidth usage and CDN cache hit rate on a per module basis. In the past it's taken us a long time to debug issues that could easily be found with this method. This means more analytics and logging – privacy intrusions justified by the sheer complexity of systems created by several thousand monkeys on thousands of electronic typewriters. Incidentally, more fingerprinting. I do not see any immediate benefit to the user here. They can get this either way. E.g. the token could be put in the URL as well. Allowing custom headers makes the setup a bit nicer and actually allows developers to use the strengths of HTTP. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
Ben Maurer ben.mau...@gmail.com writes: Another concrete example with img tags: sometimes an abusive user will use a site like Facebook as a CDN -- they'll upload a picture and hotlink it from elsewhere. We could insert a time-stamped authentication token as a custom header. Today we sometimes do this via the query string -- giving the user a token that lasts for a few days. This means we bust the user's cache every time we rotate the token. With a custom header, the browser cache stays in tact. Why not just check the referer or origin header and act on that? Images would also be a great example of where logging headers could be extremely helpful. For example, we could log what module within a page rendered an image and monitor bandwidth usage and CDN cache hit rate on a per module basis. In the past it's taken us a long time to debug issues that could easily be found with this method. This means more analytics and logging – privacy intrusions justified by the sheer complexity of systems created by several thousand monkeys on thousands of electronic typewriters. Incidentally, more fingerprinting. I do not see any immediate benefit to the user here. -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, Jul 28, 2014 at 7:24 PM, Ian Hickson i...@hixie.ch wrote: It would presumably return the same object that was being proposed for the other way around (the proposal where you call as_stylesheet or whatever). I filed https://www.w3.org/Bugs/Public/show_bug.cgi?id=26533 on investigating whether we can reuse the Request object (which is passed to fetch(), and used in service workers to expose requests from a window or worker). There are some complications given what can be set through HTML and what could be set through script and how to keep those best synchronized. We should be able to figure something out I think. Not sure how high of a priority this is though. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, Jul 28, 2014 at 8:34 PM, Ian Hickson i...@hixie.ch wrote: What's the use case here? Why are we trying to send custom headers on a link? E.g. for img and such you want to turn authentication dialogs off. Cross-origin images can be fine, but not if they start spawning confusing dialogs to users making them leak passwords. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
Another concrete example with img tags: sometimes an abusive user will use a site like Facebook as a CDN -- they'll upload a picture and hotlink it from elsewhere. We could insert a time-stamped authentication token as a custom header. Today we sometimes do this via the query string -- giving the user a token that lasts for a few days. This means we bust the user's cache every time we rotate the token. With a custom header, the browser cache stays in tact. Images would also be a great example of where logging headers could be extremely helpful. For example, we could log what module within a page rendered an image and monitor bandwidth usage and CDN cache hit rate on a per module basis. In the past it's taken us a long time to debug issues that could easily be found with this method. On Mon, Jul 28, 2014 at 11:51 PM, Anne van Kesteren ann...@annevk.nl wrote: On Mon, Jul 28, 2014 at 8:34 PM, Ian Hickson i...@hixie.ch wrote: What's the use case here? Why are we trying to send custom headers on a link? E.g. for img and such you want to turn authentication dialogs off. Cross-origin images can be fine, but not if they start spawning confusing dialogs to users making them leak passwords. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Wed, Jul 23, 2014 at 2:33 AM, Ian Hickson i...@hixie.ch wrote: That seems like it'd be no more complicated, but would involve less new API surface (not to mention fewer new ways to shoot yourself in the foot, e.g. getting the 'fetch-as' value wrong), and wouldn't require us to come up with a way to enumerate all the kinds of fetch in an API. We have already sort of done that enumeration for service workers and CSP. See http://fetch.spec.whatwg.org/#concept-request-context -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Wed, Jul 23, 2014 at 1:07 AM, Ben Maurer ben.mau...@gmail.com wrote: To follow this up with a concrete suggestion: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); If you have 'fetch-as' you need a strong guarantee that the response can only be used for that purpose. How is that envisioned? asStyleSheet() is an idea that was shot down before (back then it was asHTML()) because it has the wrong layering semantics. We do not want the Fetch layer to depend on the Layout layer. You can only call asStyleShet if fetch-as=stylesheet. Passing this parameter would cause the browser to do all the things it would do if it were fetching a stylesheet. For example, it would specify an accept header of text/css unless otherwise specified. It would request at the same priority as the browser requests other stylesheets (again, unless overridden with a yet-to-be-defined syntax). Rules around CORS and tainting would be identical to a normal stylesheet fetch (namely that you could call.asStyleSheet on a request to a different origin but it would have whatever restrictions the browser has on stylesheets from different origins). Links in the stylesheet would be interpreted relative to the URL used to load it, etc. I have the feeling this should be a higher-level API as the browser might also want to progressively parse the CSS and start speculatively fetching the resources within. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Wed, Jul 23, 2014 at 2:33 AM, Ian Hickson i...@hixie.ch wrote: Why not: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ... ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? Because that does not give you access to the additional parameters of Fetch: http://fetch.spec.whatwg.org/#requestinit (And this will grow with parameters such as omitReferrer.) (Sorry for the multiple replies to different points in your email.) -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, 28 Jul 2014, Anne van Kesteren wrote: On Wed, Jul 23, 2014 at 2:33 AM, Ian Hickson i...@hixie.ch wrote: Why not: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ... ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? Because that does not give you access to the additional parameters of Fetch: http://fetch.spec.whatwg.org/#requestinit (And this will grow with parameters such as omitReferrer.) Why would you not be able to set them on the myfetch variable in the snippet above? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, Jul 28, 2014 at 7:07 PM, Ian Hickson i...@hixie.ch wrote: On Mon, 28 Jul 2014, Anne van Kesteren wrote: On Wed, Jul 23, 2014 at 2:33 AM, Ian Hickson i...@hixie.ch wrote: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? Apologies for missing myfetch. That could work potentially. Not entirely sure what kind of object it would return, but I guess we can think of something. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, 28 Jul 2014, Anne van Kesteren wrote: On Mon, Jul 28, 2014 at 7:07 PM, Ian Hickson i...@hixie.ch wrote: On Mon, 28 Jul 2014, Anne van Kesteren wrote: On Wed, Jul 23, 2014 at 2:33 AM, Ian Hickson i...@hixie.ch wrote: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? Apologies for missing myfetch. That could work potentially. Not entirely sure what kind of object it would return, but I guess we can think of something. It would presumably return the same object that was being proposed for the other way around (the proposal where you call as_stylesheet or whatever). -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, Jul 28, 2014 at 10:14 AM, Anne van Kesteren ann...@annevk.nl wrote: On Mon, Jul 28, 2014 at 7:07 PM, Ian Hickson i...@hixie.ch wrote: On Mon, 28 Jul 2014, Anne van Kesteren wrote: On Wed, Jul 23, 2014 at 2:33 AM, Ian Hickson i...@hixie.ch wrote: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? Apologies for missing myfetch. That could work potentially. Not entirely sure what kind of object it would return, but I guess we can think of something. Yep, this is roughly what I had in mind. The way Ian describes it is the imperative interface, but I assume element.fetch would be exposed for any DOM element with an associated resource request, not just DOM elements created via E(). If so, SGTM. -- http://annevankesteren.nl/
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, 28 Jul 2014, Ben Maurer wrote: What about initial parameters to fetch (vs modifications you could make in flight via the myfetch object). Would there be an attribute of link that you could use to pass parameters to fetch (eg a custom header)? What's the use case here? Why are we trying to send custom headers on a link? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
The idea is you could specify any parameter to fetch. For example, if fetch allowed you to specify an HTTP/2 priority, you could specify that. As a concrete example of why passing a header might be useful, Facebook uses an automated pipeline to decide what CSS/JS files to package together. If we could pass a custom header in the request for a CSS file (for example: what part of the page caused that file to be loaded) we could use this information in our packaging system to make smarter decisions. While we do have the referrer header, a piece of interactive javascript on a page might be the cause of the CSS file being required. A custom header would allow logging this. On Mon, Jul 28, 2014 at 11:34 AM, Ian Hickson i...@hixie.ch wrote: On Mon, 28 Jul 2014, Ben Maurer wrote: What about initial parameters to fetch (vs modifications you could make in flight via the myfetch object). Would there be an attribute of link that you could use to pass parameters to fetch (eg a custom header)? What's the use case here? Why are we trying to send custom headers on a link? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, 28 Jul 2014, Ben Maurer wrote: The idea is you could specify any parameter to fetch. For example, if fetch allowed you to specify an HTTP/2 priority, you could specify that. As a concrete example of why passing a header might be useful, Facebook uses an automated pipeline to decide what CSS/JS files to package together. If we could pass a custom header in the request for a CSS file (for example: what part of the page caused that file to be loaded) we could use this information in our packaging system to make smarter decisions. While we do have the referrer header, a piece of interactive javascript on a page might be the cause of the CSS file being required. A custom header would allow logging this. Ah, I see. Makes sense. Are there any cases where you'd know the headers you want to send at the time the markup is written, before JS is involved, or would you always be updating the fetch settings from script? Would this ever apply to things that are downloaded with the initial load, or even in the preloader, or is this only for loads that are triggered by script later? We can definitely expose the fetch settings in markup, e.g. in the trivialest sense just by embedding a JSON blob in an attribute, or maybe in a more friendly dedicated microsyntax. I just want to make sure that that's the right thing to do before we go down that path -- meaning, that it has sufficient use cases to justify it, rather than require consumers to set these settings from script. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, Jul 28, 2014 at 12:51 PM, Ian Hickson i...@hixie.ch wrote: Ah, I see. Makes sense. Are there any cases where you'd know the headers you want to send at the time the markup is written, before JS is involved, or would you always be updating the fetch settings from script? I think there are cases where we'd want to send data directly from the markup. For example logging the fact that the load was initiated from the preloader could be useful! Also, I'm hoping that one of the things we could eventually accomplish with exposing the fetch object is increasing the number of our scripts that are accessible to the preloader. Today, Facebook avoids sending script tags directly in the document -- there is quite a bit of JS on the Facebook home page, but not all of it is needed to make the document interactive. Therefore, we send JSON blobs of which JS files will be needed for the document to our client side javascript code. This code schedules script loading, only putting in non-critical JS after the critical JS is loaded. By exposing the fetch object, we could take advantage of whatever functionality the browser gives us to control prioritization. For example, for clients using SPDY, if fetch exposed a way to set priority of the request, we could expose all of our JS to the preloader and let our CDN order it's response. We can definitely expose the fetch settings in markup, e.g. in the trivialest sense just by embedding a JSON blob in an attribute, or maybe in a more friendly dedicated microsyntax. I just want to make sure that that's the right thing to do before we go down that path -- meaning, that it has sufficient use cases to justify it, rather than require consumers to set these settings from script. I'd argue we're already expressing many of the fetch settings already. For example, the crossorigin attribute exposes the mode of a fetch ( http://fetch.spec.whatwg.org/#concept-request-mode). The sub-resource integrity spec provides a way to pass an integrity attribute to the fetch algorithm (http://www.w3.org/TR/SRI/). Directly exposing fetch parameters to the user could allow specification of this type of feature with adding new attributes to every type of request. As mentioned earlier in the thread, it could also help apply these attributes in difficult contexts (eg, the source of a background image in a CSS file). While providing fetch settings might not be the friendliest interface, I think this would be a huge win for advanced web sites. I'd point out that for sites that are advanced enough to benefit from this feature, they are highly unlikely to be writing their code by hands -- their server side code is likely generating the script tags to be inserted in their document.
Re: [whatwg] Fetch Objects and scripts/stylesheets
Hey, Yeah, I think that API would be perfect (assuming that whenneeded=true initiates the fetch, but not the load into the document). In combination with allowing the user to specify an attribute with a set of parameters to the fetch algorithm (eg, a custom header) I think that would cover the use cases I was thinking of. -b On Tue, Jul 22, 2014 at 5:33 PM, Ian Hickson i...@hixie.ch wrote: On Tue, 22 Jul 2014, Ben Maurer wrote: To follow this up with a concrete suggestion: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); ... Why not: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ... ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? (whenneeded here is a placeholder. I don't expect to actually go with that. I just used it because it's what I had proposed the last time this came up.) That seems like it'd be no more complicated, but would involve less new API surface (not to mention fewer new ways to shoot yourself in the foot, e.g. getting the 'fetch-as' value wrong), and wouldn't require us to come up with a way to enumerate all the kinds of fetch in an API. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Jul 21, 2014 9:55 AM, Ben Maurer ben.mau...@gmail.com wrote: I was walking with Will about how the browser prioritizes the loading of scripts and stylesheets. An idea that came up in our conversation was allowing the user to directly access Fetch objects associated with scripts and stylesheets. Some examples of how I could see this working: (1) Allowing the user to specify parameters to Fetch. For example, a user could say: script src=/my.js params={'headers':{'myheader':'value'}} id=myscript / This would allow the user to make any type of request customization that they could for a direct call to Fetch. This would indeed be cool. Though one downside is that sending custom headers cross-origin requires the use of CORS and preflight requests. This isn't to say that we shouldn't do it, but it will expose the author to more complexity. (2) Allowing the user to access the in progress fetch object. For example, a user could say: $(myscript).fetch.then(function(resp) { console.log(resp.status); }) Over time, the fetch object might add new functionality that would be useful to the application developer. For example, I could imagine the fetch object gaining a way to change HTTP/2 priorities mid-request. Developers could take advantage of these functions for browser-initiated fetches. As long as we can do this without requiring that the UA holds on to the unprocessed byte stream forever. I.e. after the UA has parsed the script, both the undecided byte stream and the decoded text stream should be possible to throw away. (3) Allowing the user to construct a script or stylesheet from a fetch. Complex sites often want precise control over when scripts and stylesheets are loaded and inserted into the DOM. For example, today inserting a stylesheet blocks the execution of inline script tags. Some sites may know that this dependency is not needed. If one could construct stylesheets and scripts directly from a fetch, the author could separate the act of fetching the stylesheet from inserting into the DOM: var myfetch = window.fetch(...); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); By calling asStyleSheet, the user could preserve the devtools experience that exists today while still having complete control over the fetching of their content. I like the general idea, but I don't really love the syntax. / Jonas
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Mon, 21 Jul 2014, Ben Maurer wrote: (1) Allowing the user to specify parameters to Fetch. For example, a user could say: script src=/my.js params={'headers':{'myheader':'value'}} id=myscript / This would allow the user to make any type of request customization that they could for a direct call to Fetch. I think this would make a lot of sense. It's similar to earlier suggestions for controlling Referer on a per-link basis, but more generic. The problem I had with the proposals for controlling Referer was that while there's an obvious way to do it for script and link, it quickly starts breaking down don't have a 1:1 mapping of URL to element. For example, how do you get the fetches of URLs in a style sheet? Or of the poster frame of a video element? Or EventSource objects? Should we just have different ways to set the options for each one? (2) Allowing the user to access the in progress fetch object. For example, a user could say: $(myscript).fetch.then(function(resp) { console.log(resp.status); }) Over time, the fetch object might add new functionality that would be useful to the application developer. For example, I could imagine the fetch object gaining a way to change HTTP/2 priorities mid-request. Developers could take advantage of these functions for browser-initiated fetches. This would make sense to me also, but has the same API explosion problem where there's no obvious way to expose this consistently for everything. (3) Allowing the user to construct a script or stylesheet from a fetch. Complex sites often want precise control over when scripts and stylesheets are loaded and inserted into the DOM. For example, today inserting a stylesheet blocks the execution of inline script tags. Some sites may know that this dependency is not needed. This specific use case is something I'm already working on. My long-term plan is to integrate something like this with the HTML Imports dependency model and the ES Module dependency model so that we have a single model that can handle everything. If one could construct stylesheets and scripts directly from a fetch, the author could separate the act of fetching the stylesheet from inserting into the DOM: If we have a way to give dependency information for the link or script, we don't need to manually manage the dependencies at the Fetch object level. var myfetch = window.fetch(...); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); By calling asStyleSheet, the user could preserve the devtools experience that exists today while still having complete control over the fetching of their content. We could do this too. It's mostly just a convenience method, right? (4) Giving the user explicit access to initiate fetches from the preload scanner. One downside of the approach in (3) is that it doesn't take advantage of the preload scanner to initiate fetches. I could imagine offering the user an explicit way to trigger a Fetch from the preload scanner: link id=foo rel=fetch href=/my.css / $('foo').then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); This is exactly the kind of thing I'm looking at addressing with the aforementioned work on dependencies. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Tue, Jul 22, 2014 at 10:26 AM, Ian Hickson i...@hixie.ch wrote: On Mon, 21 Jul 2014, Ben Maurer wrote: (1) Allowing the user to specify parameters to Fetch. For example, a user could say: script src=/my.js params={'headers':{'myheader':'value'}} id=myscript / This would allow the user to make any type of request customization that they could for a direct call to Fetch. I think this would make a lot of sense. It's similar to earlier suggestions for controlling Referer on a per-link basis, but more generic. The problem I had with the proposals for controlling Referer was that while there's an obvious way to do it for script and link, it quickly starts breaking down don't have a 1:1 mapping of URL to element. For example, how do you get the fetches of URLs in a style sheet? Or of the poster frame of a video element? Or EventSource objects? Should we just have different ways to set the options for each one? I guess the upshot of of doing this through fetch is that once we added the ability to specify fetch metadata it would hopefully reduce the need for future modification of all the different ways one could load an object. http://lists.w3.org/Archives/Public/public-webappsec/2014Jan/0129.html suggests the idea of using fetch for stylesheet subresources in the context of subresource integrity. (3) Allowing the user to construct a script or stylesheet from a fetch. Complex sites often want precise control over when scripts and stylesheets are loaded and inserted into the DOM. For example, today inserting a stylesheet blocks the execution of inline script tags. Some sites may know that this dependency is not needed. This specific use case is something I'm already working on. My long-term plan is to integrate something like this with the HTML Imports dependency model and the ES Module dependency model so that we have a single model that can handle everything. This sounds interesting. The way we render pages at Facebook seems very similar to what you are suggesting. We essentially flush sections of HTML with a list of CSS and JS dependencies for each section. The HTML is only rendered once these resources are loaded. In essence, it seems like we're doing inline HTML imports. We have a few other use cases for this type of dependency (for example, we have a method where you can say call this callback once resources X, Y and Z are requested. var myfetch = window.fetch(...); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); By calling asStyleSheet, the user could preserve the devtools experience that exists today while still having complete control over the fetching of their content. We could do this too. It's mostly just a convenience method, right? One advantage of doing this is that if there is some use case a site has that isn't met by the dependency model they can still manually separate the fetch of an object from its insertion into the DOM. Giving the author the freedom to manually separate the fetch of a resource from it's use in the document gives them a powerful ability to experiment with different approaches to resource loading. asStyleSheet/asScript wouldn't merely be a helper method to construct a stylesheet from a given string of text. For example, line numbers in developer tools (or JS stack traces) as if they were loaded directly from the given URL. A browser might decide to parse the JS/CSS in a background thread as soon as the document was fetched (or cache a pre-parsed version of the file). This model also might be beneficial for sites that were written prior to the dependency model you are describing. For example, at Facebook we already schedule JS/CSS loading ourselves. This API would allow us to do things like fetch a CSS stylesheet without blocking inline script execution that follows while making a minimal number of changes to the site. -b
Re: [whatwg] Fetch Objects and scripts/stylesheets
On 7/22/14, 2:01 PM, Ben Maurer wrote: One advantage of doing this is that if there is some use case a site has that isn't met by the dependency model they can still manually separate the fetch of an object from its insertion into the DOM. One issue worth considering here: there are various situations (CSP, extension) in which a browser would like to know what sort of resource is being loaded, or more precisely how it will be consumed, before loading it. From that point of view, adding a way to customize fetch paramaters on link rel=stylesheet or some other mechanism that tells the UA up front what is being fetched is vastly preferable to taking an existing fetch and synthesizing a stylesheet from it, because it provides a lot more control to the user agent and user's extensions. -Boris
Re: [whatwg] Fetch Objects and scripts/stylesheets
Nothing prevents a website from downloading content via fetch/XHR and simply inserting that text into the DOM. For example, Gmail seems to fetch many of its resources this way today. This API seems strictly better than a site that fetches text and just inserts it into the DOM. Also, it seems like CSP or extensions could still hook into this API, maybe not as early as before. For example, CSP would still know the URL of the resource that had been loaded as a script / stylesheet. While it wouldn't be able to block the loading, it could block the document from being turned into a script or stylesheet element. One could also imagine a flag passed to fetch saying fetch this document as if it were the src of a script tag. Perhaps that flag would be mandatory for one to use the asScript API. On Tue, Jul 22, 2014 at 11:34 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/22/14, 2:01 PM, Ben Maurer wrote: One advantage of doing this is that if there is some use case a site has that isn't met by the dependency model they can still manually separate the fetch of an object from its insertion into the DOM. One issue worth considering here: there are various situations (CSP, extension) in which a browser would like to know what sort of resource is being loaded, or more precisely how it will be consumed, before loading it. From that point of view, adding a way to customize fetch paramaters on link rel=stylesheet or some other mechanism that tells the UA up front what is being fetched is vastly preferable to taking an existing fetch and synthesizing a stylesheet from it, because it provides a lot more control to the user agent and user's extensions. -Boris
Re: [whatwg] Fetch Objects and scripts/stylesheets
On 7/22/14, 2:57 PM, Ben Maurer wrote: Nothing prevents a website from downloading content via fetch/XHR and simply inserting that text into the DOM. Yes, I know that. But we're trying to develop a better API so sites won't need/want to do that anymore, right? All I'm saying is that we should make the new API play nicer with CSP and extensions than the XHR and stick it in approach does. This won't stop _malicious_ sites, obviously, but it'll help with user control for normal sites who actually want to play nice with the user's settings. This API seems strictly better than a site that fetches text and just inserts it into the DOM. Sure. Also, it seems like CSP or extensions could still hook into this API, maybe not as early as before. For example, CSP would still know the URL of the resource that had been loaded as a script / stylesheet. While it wouldn't be able to block the loading, it could block the document from being turned into a script or stylesheet element. Again, sure. One could also imagine a flag passed to fetch saying fetch this document as if it were the src of a script tag. Right, exactly. That would actually simplify things for UAs as well; for example they have to do different kinds of sniffing on different request types, so knowing ahead of time what sort of thing you're requesting is quite helpful. -Boris
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Tue, Jul 22, 2014 at 11:01 AM, Ben Maurer ben.mau...@gmail.com wrote: On Tue, Jul 22, 2014 at 10:26 AM, Ian Hickson i...@hixie.ch wrote: On Mon, 21 Jul 2014, Ben Maurer wrote: (1) Allowing the user to specify parameters to Fetch. For example, a user could say: script src=/my.js params={'headers':{'myheader':'value'}} id=myscript / This would allow the user to make any type of request customization that they could for a direct call to Fetch. I think this would make a lot of sense. It's similar to earlier suggestions for controlling Referer on a per-link basis, but more generic. The problem I had with the proposals for controlling Referer was that while there's an obvious way to do it for script and link, it quickly starts breaking down don't have a 1:1 mapping of URL to element. For example, how do you get the fetches of URLs in a style sheet? Or of the poster frame of a video element? Or EventSource objects? Should we just have different ways to set the options for each one? I guess the upshot of of doing this through fetch is that once we added the ability to specify fetch metadata it would hopefully reduce the need for future modification of all the different ways one could load an object. http://lists.w3.org/Archives/Public/public-webappsec/2014Jan/0129.html suggests the idea of using fetch for stylesheet subresources in the context of subresource integrity. That's intriguing. Does fetch() have any dependencies on the javascript context? IIUC, a big motivator for element attributes for headers is allowing the preloader to issue resource requests without bouncing back through the main thread. Without knowing too much about fetch(), I'd be curious if that achieved your goals. The kind of translation I imagine happening is the DOM explaining resource requests by exposing the Request object that Fetch is defining as an attribute on the DOM element. And then you could have both a declarative and imperative interface for modifying Request.Headers. (3) Allowing the user to construct a script or stylesheet from a fetch. Complex sites often want precise control over when scripts and stylesheets are loaded and inserted into the DOM. For example, today inserting a stylesheet blocks the execution of inline script tags. Some sites may know that this dependency is not needed. This specific use case is something I'm already working on. My long-term plan is to integrate something like this with the HTML Imports dependency model and the ES Module dependency model so that we have a single model that can handle everything. This sounds interesting. The way we render pages at Facebook seems very similar to what you are suggesting. We essentially flush sections of HTML with a list of CSS and JS dependencies for each section. The HTML is only rendered once these resources are loaded. In essence, it seems like we're doing inline HTML imports. We have a few other use cases for this type of dependency (for example, we have a method where you can say call this callback once resources X, Y and Z are requested. var myfetch = window.fetch(...); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); By calling asStyleSheet, the user could preserve the devtools experience that exists today while still having complete control over the fetching of their content. We could do this too. It's mostly just a convenience method, right? One advantage of doing this is that if there is some use case a site has that isn't met by the dependency model they can still manually separate the fetch of an object from its insertion into the DOM. Giving the author the freedom to manually separate the fetch of a resource from it's use in the document gives them a powerful ability to experiment with different approaches to resource loading. asStyleSheet/asScript wouldn't merely be a helper method to construct a stylesheet from a given string of text. For example, line numbers in developer tools (or JS stack traces) as if they were loaded directly from the given URL. A browser might decide to parse the JS/CSS in a background thread as soon as the document was fetched (or cache a pre-parsed version of the file). This model also might be beneficial for sites that were written prior to the dependency model you are describing. For example, at Facebook we already schedule JS/CSS loading ourselves. This API would allow us to do things like fetch a CSS stylesheet without blocking inline script execution that follows while making a minimal number of changes to the site. -b
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Tue, Jul 22, 2014 at 12:03 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/22/14, 2:57 PM, Ben Maurer wrote: Nothing prevents a website from downloading content via fetch/XHR and simply inserting that text into the DOM. Yes, I know that. But we're trying to develop a better API so sites won't need/want to do that anymore, right? All I'm saying is that we should make the new API play nicer with CSP and extensions than the XHR and stick it in approach does. This won't stop _malicious_ sites, obviously, but it'll help with user control for normal sites who actually want to play nice with the user's settings. +1 Also, I'd like to note that, at least for now without a better prioritization system (I know you'd like to do client=server prior knowledge based prioritization mechanism, smuggling prioritization metadata via opaque-to-the-UA HTTP headers, using the headers attribute), browsers rely on resource type as a key input to their prioritization heuristics. Gmail and G+ both found that this interacted poorly with their XHR based resource loading [1] [2] since XHRs hide the true resource type. This API seems strictly better than a site that fetches text and just inserts it into the DOM. Sure. Also, it seems like CSP or extensions could still hook into this API, maybe not as early as before. For example, CSP would still know the URL of the resource that had been loaded as a script / stylesheet. While it wouldn't be able to block the loading, it could block the document from being turned into a script or stylesheet element. Again, sure. One could also imagine a flag passed to fetch saying fetch this document as if it were the src of a script tag. Right, exactly. That would actually simplify things for UAs as well; for example they have to do different kinds of sniffing on different request types, so knowing ahead of time what sort of thing you're requesting is quite helpful. -Boris [1] - https://insouciant.org/tech/spdy-prioritization-case-study-gmail/ [2] - https://plus.google.com/+ShubhiePanicker/posts/Uw87yxQFCfY
Re: [whatwg] Fetch Objects and scripts/stylesheets
I'm pretty new to this spec overall, so please tell me if I'm totally off track here -- the fetch spec states that it's goal is To unify fetching across the web platform by providing consistent handling of things such as CSP or priorities. It seems like the ultimate goal here is that the specification for something like script loading should be able to be explained in the terminology of Fetch. A corollary of this seems to be that a user should be able to simulate the behavior of a fetch initiated by the browser upon seeing a script or stylesheet. This could mean passing custom parameters to the fetch api (eg, asking for a specific priority or asking for CSP to be applied to the element). I understand that these things aren't there today -- and any site using a fetch based system might face the same poor interactions as an XHR based system would today. But in the long term, wouldn't the goal be to have parity between the parameters a user can pass to the fetch api and the parameters the browser passes internally to the fetch API? If the fetch api offers the same set of functionality available to the browser internally, users could experiment with different types of dependency management APIs. On Tue, Jul 22, 2014 at 12:13 PM, William Chan (陈智昌) willc...@chromium.org wrote: On Tue, Jul 22, 2014 at 12:03 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/22/14, 2:57 PM, Ben Maurer wrote: Nothing prevents a website from downloading content via fetch/XHR and simply inserting that text into the DOM. Yes, I know that. But we're trying to develop a better API so sites won't need/want to do that anymore, right? All I'm saying is that we should make the new API play nicer with CSP and extensions than the XHR and stick it in approach does. This won't stop _malicious_ sites, obviously, but it'll help with user control for normal sites who actually want to play nice with the user's settings. +1 Also, I'd like to note that, at least for now without a better prioritization system (I know you'd like to do client=server prior knowledge based prioritization mechanism, smuggling prioritization metadata via opaque-to-the-UA HTTP headers, using the headers attribute), browsers rely on resource type as a key input to their prioritization heuristics. Gmail and G+ both found that this interacted poorly with their XHR based resource loading [1] [2] since XHRs hide the true resource type. This API seems strictly better than a site that fetches text and just inserts it into the DOM. Sure. Also, it seems like CSP or extensions could still hook into this API, maybe not as early as before. For example, CSP would still know the URL of the resource that had been loaded as a script / stylesheet. While it wouldn't be able to block the loading, it could block the document from being turned into a script or stylesheet element. Again, sure. One could also imagine a flag passed to fetch saying fetch this document as if it were the src of a script tag. Right, exactly. That would actually simplify things for UAs as well; for example they have to do different kinds of sniffing on different request types, so knowing ahead of time what sort of thing you're requesting is quite helpful. -Boris [1] - https://insouciant.org/tech/spdy-prioritization-case-study-gmail/ [2] - https://plus.google.com/+ShubhiePanicker/posts/Uw87yxQFCfY
Re: [whatwg] Fetch Objects and scripts/stylesheets
To follow this up with a concrete suggestion: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); You can only call asStyleShet if fetch-as=stylesheet. Passing this parameter would cause the browser to do all the things it would do if it were fetching a stylesheet. For example, it would specify an accept header of text/css unless otherwise specified. It would request at the same priority as the browser requests other stylesheets (again, unless overridden with a yet-to-be-defined syntax). Rules around CORS and tainting would be identical to a normal stylesheet fetch (namely that you could call.asStyleSheet on a request to a different origin but it would have whatever restrictions the browser has on stylesheets from different origins). Links in the stylesheet would be interpreted relative to the URL used to load it, etc. Essentially, fetch-as=stylesheet would be the canonical definition of the browser's behavior around loading stylesheets. It would define the behavior if the proposals I suggested in the original email were implemented -- namely if the user chose to pass parameters to the fetch algorithm or if the user accessed the result of a stylesheet's fetch. Boris, Will -- would this setup address the concerns you have about the problems websites that use XHR to load resources encounter? On Tue, Jul 22, 2014 at 12:39 PM, Ben Maurer ben.mau...@gmail.com wrote: I'm pretty new to this spec overall, so please tell me if I'm totally off track here -- the fetch spec states that it's goal is To unify fetching across the web platform by providing consistent handling of things such as CSP or priorities. It seems like the ultimate goal here is that the specification for something like script loading should be able to be explained in the terminology of Fetch. A corollary of this seems to be that a user should be able to simulate the behavior of a fetch initiated by the browser upon seeing a script or stylesheet. This could mean passing custom parameters to the fetch api (eg, asking for a specific priority or asking for CSP to be applied to the element). I understand that these things aren't there today -- and any site using a fetch based system might face the same poor interactions as an XHR based system would today. But in the long term, wouldn't the goal be to have parity between the parameters a user can pass to the fetch api and the parameters the browser passes internally to the fetch API? If the fetch api offers the same set of functionality available to the browser internally, users could experiment with different types of dependency management APIs. On Tue, Jul 22, 2014 at 12:13 PM, William Chan (陈智昌) willc...@chromium.org wrote: On Tue, Jul 22, 2014 at 12:03 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/22/14, 2:57 PM, Ben Maurer wrote: Nothing prevents a website from downloading content via fetch/XHR and simply inserting that text into the DOM. Yes, I know that. But we're trying to develop a better API so sites won't need/want to do that anymore, right? All I'm saying is that we should make the new API play nicer with CSP and extensions than the XHR and stick it in approach does. This won't stop _malicious_ sites, obviously, but it'll help with user control for normal sites who actually want to play nice with the user's settings. +1 Also, I'd like to note that, at least for now without a better prioritization system (I know you'd like to do client=server prior knowledge based prioritization mechanism, smuggling prioritization metadata via opaque-to-the-UA HTTP headers, using the headers attribute), browsers rely on resource type as a key input to their prioritization heuristics. Gmail and G+ both found that this interacted poorly with their XHR based resource loading [1] [2] since XHRs hide the true resource type. This API seems strictly better than a site that fetches text and just inserts it into the DOM. Sure. Also, it seems like CSP or extensions could still hook into this API, maybe not as early as before. For example, CSP would still know the URL of the resource that had been loaded as a script / stylesheet. While it wouldn't be able to block the loading, it could block the document from being turned into a script or stylesheet element. Again, sure. One could also imagine a flag passed to fetch saying fetch this document as if it were the src of a script tag. Right, exactly. That would actually simplify things for UAs as well; for example they have to do different kinds of sniffing on different request types, so knowing ahead of time what sort of thing you're requesting is quite helpful. -Boris [1] - https://insouciant.org/tech/spdy-prioritization-case-study-gmail/ [2] - https://plus.google.com/+ShubhiePanicker/posts/Uw87yxQFCfY
Re: [whatwg] Fetch Objects and scripts/stylesheets
On Tue, 22 Jul 2014, Ben Maurer wrote: To follow this up with a concrete suggestion: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); ... Why not: var mystyle = E('link', { rel: 'stylesheet', href: 'my.css', whenneeded: true }); document.body.appendChild(mystyle); var myfetch = mystyle.fetch; ... ...where E() is some mechanism to easily create new elements (we need one of those regardless), and whenneeded is some attribute that controls the load policy (and in this case, tells it to not load yet, since I presume that's what you're going to do next with the myfetch variable)? (whenneeded here is a placeholder. I don't expect to actually go with that. I just used it because it's what I had proposed the last time this came up.) That seems like it'd be no more complicated, but would involve less new API surface (not to mention fewer new ways to shoot yourself in the foot, e.g. getting the 'fetch-as' value wrong), and wouldn't require us to come up with a way to enumerate all the kinds of fetch in an API. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Fetch Objects and scripts/stylesheets
I like how fetch is a small primitive right now. We should not overload it with more functionality, but instead create new APIs on top of it. It's important to be able to polyfill those APIs on top of fetch, instead of adding more stuff to the base substrate of fetch itself. From: Ben Maurermailto:ben.mau...@gmail.com Sent: 2014-07-22 19:07 To: William Chan (陈智昌)mailto:willc...@chromium.org Cc: whatwg@lists.whatwg.orgmailto:whatwg@lists.whatwg.org; Boris Zbarskymailto:bzbar...@mit.edu Subject: Re: [whatwg] Fetch Objects and scripts/stylesheets To follow this up with a concrete suggestion: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); You can only call asStyleShet if fetch-as=stylesheet. Passing this parameter would cause the browser to do all the things it would do if it were fetching a stylesheet. For example, it would specify an accept header of text/css unless otherwise specified. It would request at the same priority as the browser requests other stylesheets (again, unless overridden with a yet-to-be-defined syntax). Rules around CORS and tainting would be identical to a normal stylesheet fetch (namely that you could call.asStyleSheet on a request to a different origin but it would have whatever restrictions the browser has on stylesheets from different origins). Links in the stylesheet would be interpreted relative to the URL used to load it, etc. Essentially, fetch-as=stylesheet would be the canonical definition of the browser's behavior around loading stylesheets. It would define the behavior if the proposals I suggested in the original email were implemented -- namely if the user chose to pass parameters to the fetch algorithm or if the user accessed the result of a stylesheet's fetch. Boris, Will -- would this setup address the concerns you have about the problems websites that use XHR to load resources encounter? On Tue, Jul 22, 2014 at 12:39 PM, Ben Maurer ben.mau...@gmail.com wrote: I'm pretty new to this spec overall, so please tell me if I'm totally off track here -- the fetch spec states that it's goal is To unify fetching across the web platform by providing consistent handling of things such as CSP or priorities. It seems like the ultimate goal here is that the specification for something like script loading should be able to be explained in the terminology of Fetch. A corollary of this seems to be that a user should be able to simulate the behavior of a fetch initiated by the browser upon seeing a script or stylesheet. This could mean passing custom parameters to the fetch api (eg, asking for a specific priority or asking for CSP to be applied to the element). I understand that these things aren't there today -- and any site using a fetch based system might face the same poor interactions as an XHR based system would today. But in the long term, wouldn't the goal be to have parity between the parameters a user can pass to the fetch api and the parameters the browser passes internally to the fetch API? If the fetch api offers the same set of functionality available to the browser internally, users could experiment with different types of dependency management APIs. On Tue, Jul 22, 2014 at 12:13 PM, William Chan (陈智昌) willc...@chromium.org wrote: On Tue, Jul 22, 2014 at 12:03 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/22/14, 2:57 PM, Ben Maurer wrote: Nothing prevents a website from downloading content via fetch/XHR and simply inserting that text into the DOM. Yes, I know that. But we're trying to develop a better API so sites won't need/want to do that anymore, right? All I'm saying is that we should make the new API play nicer with CSP and extensions than the XHR and stick it in approach does. This won't stop _malicious_ sites, obviously, but it'll help with user control for normal sites who actually want to play nice with the user's settings. +1 Also, I'd like to note that, at least for now without a better prioritization system (I know you'd like to do client=server prior knowledge based prioritization mechanism, smuggling prioritization metadata via opaque-to-the-UA HTTP headers, using the headers attribute), browsers rely on resource type as a key input to their prioritization heuristics. Gmail and G+ both found that this interacted poorly with their XHR based resource loading [1] [2] since XHRs hide the true resource type. This API seems strictly better than a site that fetches text and just inserts it into the DOM. Sure. Also, it seems like CSP or extensions could still hook into this API, maybe not as early as before. For example, CSP would still know the URL of the resource that had been loaded as a script / stylesheet. While it wouldn't be able to block the loading, it could block the document from being turned into a script or stylesheet element
Re: [whatwg] Fetch Objects and scripts/stylesheets
On 7/22/14, 7:07 PM, Ben Maurer wrote: var myfetch = window.fetch('my.css', {'fetch-as': 'stylesheet'}); myfetch.then(function(resp) { document.body.appendChild(resp.body.asStyleSheet()); }); Boris, Will -- would this setup address the concerns you have about the problems websites that use XHR to load resources encounter? It would. So would some API on top of fetch (as Domenic and Ian propose). -Boris