Re: [webkit-dev] X-Purpose on prefetching requests
Noone has commented on this thread in some time; is this less controversial now? On 24 September 2010 21:27, Alexey Proskuryakov a...@webkit.org wrote: Eric has made another good point in Bugzilla - we don't explain the purpose in lots of other cases, from loading into a display:none frame to XMLHttpRequest. Why should Prefetch be all the different? Those requests are still part of loading a page as requested by the user, and failing them or returning a 500 will result in undefined behaviour; the result of failing part of a page is unpredictable. In the case of prefetches, we're in a very different space. The prefetch load has no effects on the DOM of the current resource (unlike display:none elements). There is no javascript event, or history impact, etc... (unlike XMLHttpRequest). A subsequent navigation will occur correctly whether there was no prefetch, a prefetch that failed, a prefetch that requires revalidation, and a prefetch that can be used without revalidation. So that, in my mind, leaves the balance in favour of emitting the header, as Mozilla Safari currently do in similar situations. Thoughts? - Gavin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
28.09.2010, в 07:58, Gavin Peters (蓋文彼德斯) написал(а): Noone has commented on this thread in some time; is this less controversial now? I've presented some concerns about the effect of this on enterprise network monitors. It it also not really clear what the benefits of the proposed approach are. It's beginning to sound like this header doesn't even help statistics much, and the only practical use is refusing to serve a request under high load. But I suspect that there is not much difference between returning a 5xx response and serving a cached pseudo-static version of a resource. It should be much easier to implement the latter (which lots of servers obviously already do) than to change behavior based on current load. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
On 28 September 2010 11:58, Alexey Proskuryakov a...@webkit.org wrote: 28.09.2010, в 07:58, Gavin Peters (蓋文彼德斯) написал(а): Noone has commented on this thread in some time; is this less controversial now? I've presented some concerns about the effect of this on enterprise network monitors. It it also not really clear what the benefits of the proposed approach are. It's beginning to sound like this header doesn't even help statistics much, and the only practical use is refusing to serve a request under high load. But I suspect that there is not much difference between returning a 5xx response and serving a cached pseudo-static version of a resource. It should be much easier to implement the latter (which lots of servers obviously already do) than to change behavior based on current load. The header allow high load opt out, as you suggested. It also allows general opt-out of prefetching by servers. Additionally, it is the only way you'd gather statistics about who has prefetch links targeting you (from Referer when provided with prefetch). Without this header, there is no way at all for servers to gather statistics on user initiated navigations vs prefetches; with it, you can use cache-control to make this determination. I see those as benefits, and I think they're nonzero. On the cost side, I see a small increase in the size of prefetch requests (and that cost is born only by this type of request, not by any other kind) and the hassle you mentioned in network monitoring. We can probably agree that the cost of the extra request headers is minimal, so hopefully that's not a very high hurdle to overcome. The remaining question then is: how much weight should be given to the difficulties of network monitoring software authors? - Gavin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
Forgive the second follow up, I want to just clarify something. See below. On 28 September 2010 11:58, Alexey Proskuryakov a...@webkit.org wrote: I've presented some concerns about the effect of this on enterprise network monitors. I've thought about this some more, and and I think I don't get this actually. Could you clarify for me? What effect does this header have on network monitoring software, distinct from prefetching itself? If we lack the request header, then network monitoring software can't attribute browsing action to users at all; any request might be a prefetch. In the presence of this header, the network monitoring software can determine which requests began as prefetches, and, depending on the nature of the response, it may or may not be able to determine if a later user initiated navigation to that resource occured. Doesn't this header actually help network monitoring software? - Gavin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
28.09.2010, в 9:43, Gavin Peters (蓋文彼德斯) написал(а): I've presented some concerns about the effect of this on enterprise network monitors. I've thought about this some more, and and I think I don't get this actually. Could you clarify for me? I think that it changes false positives to false negatives. Without the header, it will complain about prefetch requests made for Google search results. But once the monitoring software learns to ignore prefetch requests, then it will be easy to circumvent it by adding X-Purpose to every request (e.g. with a browser extension). Doomed both ways. It seems that the only real way to make prefetch safe may be to limit it to same origin URLs. Yes, one can always do their own prefetching via a hidden frame, but the purpose of explicit prefetch was to make it semantically clean, and that doesn't seem to work without imposing a same origin restriction. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
2010/9/28 Alexey Proskuryakov a...@webkit.org: 28.09.2010, в 9:43, Gavin Peters (�w文彼德斯) написал(а): I've presented some concerns about the effect of this on enterprise network monitors. I've thought about this some more, and and I think I don't get this actually. Could you clarify for me? I think that it changes false positives to false negatives. Without the header, it will complain about prefetch requests made for Google search results. But once the monitoring software learns to ignore prefetch requests, then it will be easy to circumvent it by adding X-Purpose to every request (e.g. with a browser extension). Doomed both ways. It seems that the only real way to make prefetch safe may be to limit it to same origin URLs. Yes, one can always do their own prefetching via a hidden frame, but the purpose of explicit prefetch was to make it semantically clean, and that doesn't seem to work without imposing a same origin restriction. Hum... Restricting prefetching to a single origin would seem to defeat much of the point of the feature in the first place. Folks who want to do prefetch will just end up using an image tag instead. Alexey, it seems like your objection to the X-Purpose header is based on the belief that servers and network monitors won't use the information correctly. Is there evidence that these folks make these mistakes with the similar X-Moz header used by Firefox? What about the X-Purpose header as used by Safari already? To counterbalance these concerns, we'll gotten feedback from server operators that they're sad that WebKit lacks the X-Moz header on prefetch requests, which they find useful. From talking with these folks, it seems like they'll live happier lives if we include this information in prefetch requests. Given that the cost of including the information is low and there's already ample implementation experience from Firefox and Safari, including the information seems like the right call, on balance. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
I guess I don't understand your perspective. Are you arguing that prefetch is a misfeature and folks should use image tags instead? That doesn't really make sense to me. For example, using an explicit prefect API means the prefetch gets properly prioritized and doesn't delay the load event. Below you characterize prefetch as a hack, but it seems like a useful feature that's used by a bunch of sites. As for X-Purpose, sometimes you seem to be saying that it can't be used correctly, whereas other times you seem to be saying that it can be used correctly, just that sites will screw it up. At the beginning of the discussion, you seemed worried about the size increase to requests. Reading your messages, it feels like you don't like X-Purpose, but you're not quite sure why. As the discussion progresses, your reasons for disliking it seem to shift. Server operators seem to be sad that we're don't have the same feature as Firefox. I don't understand why we shouldn't make them happier. Adam 2010/9/28 Alexey Proskuryakov a...@webkit.org: 28.09.2010, в 12:53, Adam Barth написал(а): Hum... Restricting prefetching to a single origin would seem to defeat much of the point of the feature in the first place. Folks who want to do prefetch will just end up using an image tag instead. Yes, that sounds likely. But why was link prefetch added if it's unnecessary? I have two guesses: - syntactic sugar; - trying to do it semantically right, providing a way to solve the set of problems with prefetch we've just discussed. I think that the discussion that we're having now shows that the second goal cannot be achieved, at least not without some new ideas. Telling servers what part of WebCore initiated the request only shifts the blame on someone else (we've provided complete information to the server, and washed our hands). Alexey, it seems like your objection to the X-Purpose header is based on the belief that servers and network monitors won't use the information correctly. Is there evidence that these folks make these mistakes with the similar X-Moz header used by Firefox? It actually appears that it cannot be used correctly, not just that it won't be. What about the X-Purpose header as used by Safari already? As far as Safari implementation experience goes, I'd like to re-iterate that preview is quite different in this respect, I don't think it's sufficiently related to be given as an example. There is no expectation that fetching a page for preview will make navigating to it faster in the future - and fulfilling that expectation is one of the central problem with prefetch. There is no question of whether fetching for preview means access for network security monitors either - you only prefetch for sites that were previously visited, roughly speaking. To counterbalance these concerns, we'll gotten feedback from server operators that they're sad that WebKit lacks the X-Moz header on prefetch requests, which they find useful. It's undoubtedly a problem for server operators when another site prefetches their resources indiscriminately (exactly as it would be if someone used display:none or img tag prefetch indiscriminately). Unfortunately, link prefetch looks as if it were less of a hack, encouraging people to use it. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
On Sep 28, 2010, at 4:16 PM, Adam Barth wrote: I guess I don't understand your perspective. Are you arguing that prefetch is a misfeature and folks should use image tags instead? That doesn't really make sense to me. For example, using an explicit prefect API means the prefetch gets properly prioritized and doesn't delay the load event. Below you characterize prefetch as a hack, but it seems like a useful feature that's used by a bunch of sites. As for X-Purpose, sometimes you seem to be saying that it can't be used correctly, whereas other times you seem to be saying that it can be used correctly, just that sites will screw it up. At the beginning of the discussion, you seemed worried about the size increase to requests. Reading your messages, it feels like you don't like X-Purpose, but you're not quite sure why. As the discussion progresses, your reasons for disliking it seem to shift. Server operators seem to be sad that we're don't have the same feature as Firefox. I don't understand why we shouldn't make them happier. I'll try to find out the reasons we added X-Purpose: preview to Safari's snapshot fetcher. If similar use cases apply to prefetch, then that seems like a strong argument to add X-Purpose: prefetch. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
28.09.2010, в 16:16, Adam Barth написал(а): I guess I don't understand your perspective. Are you arguing that prefetch is a misfeature and folks should use image tags instead? That doesn't really make sense to me. For example, using an explicit prefect API means the prefetch gets properly prioritized and doesn't delay the load event. This wasn't a preconceived idea of mine when starting this discussion. However, I'm actually beginning to think that cross-origin prefetch may be a misfeature indeed. It's one thing to optimize one's own site loading, and another to use someone else's bandwidth. The effects this has on statistics and network monitoring are secondary reasons to doubt that this is a good idea. The benefit of cross origin prefetching seems to be limited, too. In most cases, you can only prefetch the main resource, which likely requires revalidation anyway. Below you characterize prefetch as a hack, but it seems like a useful feature that's used by a bunch of sites. As for X-Purpose, sometimes you seem to be saying that it can't be used correctly, whereas other times you seem to be saying that it can be used correctly, just that sites will screw it up. At the beginning of the discussion, you seemed worried about the size increase to requests. Reading your messages, it feels like you don't like X-Purpose, but you're not quite sure why. As the discussion progresses, your reasons for disliking it seem to shift. Yes, this is a correct impression. I'm not sure why you look at this as something negative. I didn't know about the consequences of link prefetch much in the beginning, and reacted to this patch simply because it smelled wrong (disclose more information without saying a word about why that's beneficial, and why it's not dangerous). The new arguments that are being provided are worth answering, and it's only fair that each gets its own answer. Others have also noticed that there were major questions that weren't answered in https://bugs.webkit.org/show_bug.cgi?id=46529, or in any discussion linked from it. Server operators seem to be sad that we're don't have the same feature as Firefox. I don't understand why we shouldn't make them happier. This is a good argument for spending time on designing a feature, not necessarily so for shipping it. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
28.09.2010, в 17:07, Gavin Peters (蓋文彼德斯) написал(а): If you're questioning if prefetch can make page loads faster, I can say that on the web today it does. Thank you for the information. Does the performance increase come at the cost of correctness? I.e., are these sites requiring revalidation on each load? The samples of prefetching on the web today are mostly cross site. Can you provide any examples? - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
On 28 September 2010 20:25, Alexey Proskuryakov a...@webkit.org wrote: 28.09.2010, в 17:07, Gavin Peters (蓋文彼德斯) написал(а): If you're questioning if prefetch can make page loads faster, I can say that on the web today it does. Thank you for the information. Does the performance increase come at the cost of correctness? I.e., are these sites requiring revalidation on each load? That's a great question, and I don't have data on it in aggregate. The only examples I know of for pages varying behaviour based on the X-Moz header are sites that deny prefetches. I hope that revalidation will end up being somewhat rare: adding a RTT to the root document of a page of course slows down first paint, onload, etc.. by pretty much exactly one RTT. If correctness requires it, so be it, but if a resource would have been cacheable on a user-initiated load, I hope site operators would allow it to be cacheable for prefetchers over the same period. Of course, this is the default behaviour now, and it's the one that webkit enforces. The samples of prefetching on the web today are mostly cross site. Can you provide any examples? Yes. Google Web Search provides a prefetch link to the first search result in many cases. These links are (almost?) always cross site. - Gavin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
24.09.2010, в 22:26, Gavin Peters (蓋文彼德斯) написал(а): I think I might be missing this point. For the purposes of prefetching, I think that once you send out a Vary: X-Purpose header, you've basically opted out of permitting prefetching. I've been working from an overly broad idea that servers might want to do something different for prefetch requests. But it seems that the only reasonable responses to a prefetch request are: - break the connection; - send a 404 or 5xx response; - send exactly the same response as for a non-prefetch request. And if the server bothers to count accesses to this resource, it must have Cache-control set to must-revalidate. There is no need to send a Vary header field in response for these. (Note that if any server won't send Cache-control: must-revalidate, then this creates a way to potentially fool enterprise network monitoring tools. One one hand, they'll need to ignore prefetch resources (since prefetching doesn't imply viewing), but on the other hand, viewing won't generate a request at all!) - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] X-Purpose on prefetching requests
When the prefetch feature was originally added to WebKit, the first patch (bug 3652), it had a header X-Moz, for backwards compatibility with Firefox. Currently, when Firefox follows link prefetch requests, it adds a header X-Moz: prefetch to the request. However, before 3652 was landed, we removed this. The X-Moz header is pretty ugly, it has a very bad name. I felt a bit unsatisfied about this, so today I circled back and landed a patch to cause WebKit, when following prefetches, to send an X-Purpose: prefetch header, which is modelled closely on the X-Purpose: preview header that is sent by Safari when it does its preview generating loads (for those pictures of websites you see on the new tab screen). That patch is in bug 46529. I've also started a discussion in the HTTPbis working group about the three incompatible methods of notification that exist (X-Moz, X-Purpose, and do nothing), to see if a new header Purpose makes sense. The X-Purpose patch was controversial though; so Alexey Proskuryakov suggested we start a discussion in the mailing list. Right now, as I see it (and please chime in with what I'm missing), points in favour of sending X-Purpose are: - Conformity. Existing browsers that do prefetching/previewing notify servers, WebKit browsers should follow this. - Differentiation. Allows some kinds of differential treatment. A heavily loaded server may not want to serve prefetches, and 500 them. Some servers today 404 all X-Moz requests; while I think that's a shame, it is good to let people opt out of a feature. - Statistics. Server authors may wish to distinguish the origins of their traffic; although of course this is tricky, since prefetch loads are not always viewed and not always ignored. (send Cache-Control: must-revalidate) to prefetch requests, and you'll accurately know about uses, although at some cost in latency for your users. - Monitoring. Perhaps your web monitoring software or network operations department would like to know the difference between user navigation, and browser-initiated navigation. However, against X-Purpose, we have: - Acceptance. We want prefetching to succeed, and this header will promote opt-out. Besides, opt-out after getting the request is slow anyway. - Cost. Why make requests longer than they need to be? - Dangerous. If people vary content based on this header, they could cause all sorts of caching troubles (for instance a proxy might cache a 404 from a server that's differentiating traffic). What do people think? From I think we should follow existing practice, and send the header. I don't think the header cost is very big, and I am not sure that caching 404s is defensible anyway. Lots of headers shouldn't be used to mutate content; this is not new. Thoughts? - Gavin For the original landing of prefetching, see: https://bugs.webkit.org/show_bug.cgi?id=3652 For this new bug, see: https://bugs.webkit.org/show_bug.cgi?id=46529 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
24.09.2010, в 17:31, Gavin Peters (蓋文彼德斯) написал(а): - Cost. Why make requests longer than they need to be? If used correctly, this will make responses longer, too (due to Vary: X-Purpose). Due to the nature of Vary, it will need to be sent with every response for the resource, not just those that are in response to prefetch requests. Eric has made another good point in Bugzilla - we don't explain the purpose in lots of other cases, from loading into a display:none frame to XMLHttpRequest. Why should Prefetch be all the different? - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] X-Purpose on prefetching requests
On 24 September 2010 21:27, Alexey Proskuryakov a...@webkit.org wrote: 24.09.2010, в 17:31, Gavin Peters (蓋文彼德斯) написал(а): - Cost. Why make requests longer than they need to be? If used correctly, this will make responses longer, too (due to Vary: X-Purpose). Due to the nature of Vary, it will need to be sent with every response for the resource, not just those that are in response to prefetch requests. I think I might be missing this point. For the purposes of prefetching, I think that once you send out a Vary: X-Purpose header, you've basically opted out of permitting prefetching. Why wouldn't such a site send out a non-cacheable failure to prefetching requests? A 500 or 404 would likely not cause a cache discard, and so an eventual user navigation will just cause cache validation; but that's OK: the prefetch wouldn't have resulted in network activity if the cache was fresh anyway. Is there another case I'm forgetting or not understanding? I can see how this might be different in the X-Purpose: preview case, but that cat's out of the bag. Eric has made another good point in Bugzilla - we don't explain the purpose in lots of other cases, from loading into a display:none frame to XMLHttpRequest. Why should Prefetch be all the different? That I don't have a good answer to; we don't. For prefetching browsers traditionally have, but for these cases they haven't. Prefetching is not all that different than a display:none frame I suppose. - Gavin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
