Re: [whatwg] Improvement of the Application Cache
Sounds related to Programmable HTTP Caching and Serving (formerly titled DataCache API): http://dev.w3.org/2006/webapi/DataCache/ [[[ This document defines APIs for off-line serving of requests to HTTP resources using static and dynamic responses. It extends the function of application caches defined in HTML5. ]]] - Joe On Mar 3, 2011, at 1:50 PM, Michael Nordman wrote: Sounds like there are at least two feature requests here... 1) Some way of not adding the 'master' entry to the cache. This is a common feature request, there's another thread about it specificially titled Application Cache for on-line sites. 2) The ability to add(), remove(), and enumerate() individual urls in the appcache. Long ago, there were interfaces on the appcache drawing board to allow that. They got removed for a variety of reasons including to start simpler. A couple of years later, it may make sense to revisit these kind of features, although there is another repository also capable of storing ad-hoc collection of resources now (FileSystem), so i'm not sure this feature really needs to be in the appcache. @Hixie... any idea when the appcache feature set will be up for a growth spurt? I think there's an appetite for another round of features in the offline app developers that i communicate with. There's been some recent interest here in pursuing a means of programatically producing a response instead of just returning static content. On Wed, Mar 2, 2011 at 7:40 AM, Edward Gerhold edward.gerh...@googlemail.com wrote: Hello, i would like to suggest an improvement for the Offline Web applications. Problem: I´ve found out, that i can not Cache my Joomla! Content Management System. Of course i´ve read and heard about, that the application cache is for static pages. But with a little change to the spec and the implementations, it would be possible to cache more than static pages. I would like to cache my Joomla! system. To put the scripts, css and images into the cache. I would like to add the appcache manifest to the index.php file of the Joomla Template. What happens is, that the index.php is cached once and not updated again. I can not view new articles. The problem is, that i can neither update the Master File, nor whitelist it. And this is, what my request or suggestion is about. I would like to whitelist the Master file, where the appcache manifest is installed in. Or i would like to update this file, or any file else, i would like to update, on demand. If there is any possibility, to do that already, please tell me. But i think that is not the case. Caching the CMS by making it possible to update or to whitelist certain files, the always dynamic frontpage or /index.php, would be the hammer to nail the board on the storage. Rules: The things, which should be considered are: *To allow to fetch the Master file, e.g. index.php* *in Joomla! over the NETWORK,* while any other file in the manifest get´s fetched or cached like before. Which is the most important for me, to get Joomla! into the cache. Javascript: For the script i would like to add *applicationCache.updateMaster()*, which forces the browser to fetch the file again. I think, this is impossible today, to update exactly this file. For the function, i could add a button to my page, to let the user choose to update the file. The second function would be *applicationCache.updateFile(url)*, which could be triggered by a button and script, too. I could let the user update certain articles. With that i would like to suggest* applicationCache.addToCache(url)* to add files manually or programmatic, which can not be determined by the manifest. Urls like new articles (*), i would like to read offline. I would like to add them to the cache, if the link appears, maybe on the frontpage. I would have to add the manifest to the CMS anyways, so i could add a few more functions to the page, of course. * applicationCache.removeFromCache(url)* should be obvious and helpful with the other functions. Good would be, to be able to iterate through the list of cached objects and even the manifest, with the update, add, remove functions, it would be very useful to work with the filenames and parameters. [(*) I could let the user decide wether he wants to download my mp3 files to the appcache or not, and fulfill the wish with the javascript functions. Maybe he´s got no bytes left or wants only the lyrics.] Conclusion: The application cache is very powerful. But it is very disappointing, that it is only useful for static pages. With a little improvement to the Offline Web applications chapter, and of course to the browsers, it would be possible to cache any Content Manager or dynamic page. And that would let the appcache become one of the most powerful things in the world. I could read my Joomla! offline, could update the cached files, if i want
Re: [whatwg] Appcache feedback (various threads)
On Aug 12, 2010, at 3:29 PM, Ian Hickson wrote: These quotas are often global, some kind of user setting, or are per-origin. Application Caches are missing such a quota. The entire Disk Space section of Web SQL Databases could equally apply to Application Caches: http://dev.w3.org/html5/webdatabase/#disk-space I suppose that's not unreasonable. I've added a similar section to the appcache section. Excellent. The text [1][2] looks great. The likely behavior would be the user agent emits an error event. However, storage limits are not specified in the spec as an error condition: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#event-appcache-error In general the specification does not specify behaviour in handling constraints of the operating environment (e.g. out of memory, out of disk space). I've added some generic handling for this. We can add more detailed error reporting in a future version if there are good reasons to do this. Good to know. Thanks. - Joe [1]: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#disk-space [2]: http://html5.org/tools/web-apps-tracker?from=5286to=5287
[whatwg] Should window.name be [Replaceable]?
Should window.name be [Replaceable]?
There are a number of [Replaceable] [1] properties on the window
object. However, window.name is not marked as such [2]. Not being
marked as replaceable means that if authors use a global variable
named name it will be coerced into a string. For many this causes
confusion. It seems some browsers have opted to make it replaceable.
Test case. You can test your browser at [3].
(function() {
var arr = [];
arr.push( typeof window.name );
window.name = 1;
arr.push( typeof window.name );
name = 1;
// number if [Replaceable], string otherwise
arr.push( typeof window.name );
alert( arr );
})();
Browser Results:
- Safari 5 / WebKit Nightly and Chrome 5
NOT [Replaceable], printing string,string,string
- Firefox 3.6.3 / 3.7a5 and Opera 10.5.3 / 10.6
[Replaceable], printing string,string,number
- I didn't have access to IE to test. I'd appreciate
someone giving it a shot to see how they act.
Has there been discussion on this in the past? I searched and
didn't find any discussion.
- Joe
[1]: http://dev.w3.org/2006/webapi/WebIDL/#Replaceable
[2]:
http://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#the-window-object
[3]: https://bug-19967-attachments.webkit.org/attachment.cgi?id=60491
Re: [whatwg] Should window.name be [Replaceable]?
On Jul 4, 2010, at 9:35 PM, Biju wrote: On Mon, Jul 5, 2010 at 12:05 AM, Joseph Pecoraro pecor...@apple.com wrote: - I didn't have access to IE to test. I'd appreciate someone giving it a shot to see how they act. On IE8 it is printing string,string,string Thanks for checking! So its not Replaceable. And it looks like in Firefox 1) name = with_a_number changes type to matched the assigned value 2) window.name = with_a_number keeps the previous type Yes. So there is no way to access the real window.name once it has been replaced. - Joe
[whatwg] Form Validity Ambiguity - patternMismatch
The section describing the pattern attribute does not explain that it only applies to particular input types [1]. However, one of the tables showing which content attributes apply to which input types shows the pattern attribute only applies to particular input types. Namely text, search, url, telephone, email, and password but not dates, times, numbers, hidden and others: This disconnect could lead to ambiguous implementations of ValidityState's patternMismatch which is simply described in multiple places as: [[ When a control has a value that doesn't satisfy the pattern attribute. ]] [3] [[ returns true if the element's value doesn't match the provided pattern; false otherwise. ]] [4] These two definitions of patternMismatch do not mention that they only apply on input type's that the pattern attribute applies to. If that is the case then I recommend clarifying that constraint in the description of the pattern attribute [1] and when describing patternMismatch [3][4]. If that is not that the case then I don't see the purpose of the table showing that the pattern attribute only applies to particular input types, and I would like clarification there (like the row being removed or all checkboxes?) [2]. This may also apply to other validity states as well, I haven't checked others yet. Cheers! - Joe [1]: http://www.whatwg.org/specs/web-apps/current-work/multipage/common-input-element-attributes.html#attr-input-pattern [2]: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-input-element.html#the-input-element [3]: http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#suffering-from-a-pattern-mismatch [4]: http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#the-constraint-validation-api
Re: [whatwg] Form Validity Ambiguity - patternMismatch
On Apr 20, 2010, at 6:16 AM, Mounir Lamouri wrote: Actually, if you are looking for something more normative (the table is non-normative), at the end of each input element states description, you can found the list of attributes which apply/don't apply. AFAIK, the attributes/methods never mention in which states they apply. I suppose it is to prevent mentioning things twice thus doubling the needed changes and the chances of incoherence/mistakes. You're correct! Thanks for pointing out that section. I still think this could be clarified. I feel that a majority of the time someone is going to read patternMismatch and say oh, I just need a pattern attribute and not think to check that the pattern attribute only applies to particular types. Does anyone else agree or should this be left alone? - Joe
Re: [whatwg] Appcache feedback
On Thu, 17 Dec 2009 22:44:01 +0100, Ian Hickson i...@hixie.ch wrote: We could delay the application cache download process so that it doesn't start until after the 'load' event has fired. Does anyone have an opinion on this? On Dec 17, 2009, at 5: 24PM, Michael Nordman wrote: I don't think we'd have to delay the update job, just the delivery of any events associated with the appcache until 'load' has happened to get the desired effect. On Dec 18, 2009, at 7: 51AM, Anne van Kesteren wrote: I believe we (Opera) found this is what at least some implementations are doing now and we do this as well. What Michael suggested makes sense and I suppose as the effects are indistinguishable in the end that (delaying either the events or the download process) should be fine. Mike's suggestion sounds good. Is this in the queue to be added or could this have been forgotten about? There was a change around the time this email thread occurred that clarified that the download process should be performed in the background, but nothing about the events. - Joseph Pecoraro
Re: [whatwg] Appcache feedback
On Dec 17, 2009, at 4: 44PM, Ian Hickson wrote: Another conforming sequence of events would be: 1. The parser's first parsing task begins. 2. As soon as the manifest= attribute is parsed, the application cache download process begins. It queues a task to dispatch the 'checking' event. 3. The parser's first parsing task ends. 4. The event loop spins, and runs the next task, which is the 'checking' event. No scripts have yet run, so no handlers are registered. Nothing happens. 5. The parser's second parsing task begins. It will parse the script, etc. [snip moved below..] We could delay the application cache download process so that it doesn't start until after the 'load' event has fired. Does anyone have an opinion on this? From an application developer standpoint I think that would be very nice. I cannot comment on a this from an implementors perspective (yet). It seems important considering the following text from the Spec: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#downloading-or-updating-an-application-cache [[ Certain events fired during the application cache download process allow the script to override the display of such an interface. The goal of this is to allow Web applications to provide more seamless update mechanisms, hiding from the user the mechanics of the application cache mechanism. User agents may display user interfaces independent of this, but are encouraged to not show prominent update progress notifications for applications that cancel the relevant events. ]] It seems pointless to provide hooks in the API that allow for a custom interface, when fundamentally they may never be triggered in an otherwise compliant user agent. You can work around all this by checking the .status attribute when you first hook up the event listeners. This is even worse. To me, this means the application developer cannot rely on certain (any?) events, so he/she would need to build a completely new API on top? Developers will still likely be caught searching for bugs in their own code (like I did) wondering why behavior is different between browsers. Thanks for the quick response, - Joe
[whatwg] [ApplicationCache] Clarify When Events Fire
I'd like some clarification on exactly when the ApplicationCache events should
fire. Specifically the events that are likely to fire early, such as the
checking or downloading events.
In 6.9.4 Downloading or updating an application cache, the specification
currently says the following for the checking event (step 4):
http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#dfnReturnLink-0
[[
If these steps were invoked with a cache host, and the status of
cache group is checking or downloading, then queue a task to fire
a simple event named checking that is cancelable at the
ApplicationCache singleton of that cache host. ...
]]
What should the proper behavior be in the following scenario:
!DOCTYPE html
html manifest=appcache.manifest
head
titleApplication Cache Bug/title
script src=EXTERNAL_SCRIPT type=text/javascript/script
script type=text/javascript
window.applicationCache.onchecking = function() {
alert('checking');
}
/script
/head
body
h1You should get an alert on reloads!/h1
/body
/html
Firefox triggers the alert.
WebKit does not trigger the alert.
There is an open WebKit bug on this topic [1] which describes the thought
process behind WebKit's behavior. Taken from the comments:
1) As soon as body manifest attribute is processed, application
cache update begins, and a zero-timer task to dispatch a checking
event is posted for later execution.
2) script src=EXTERNAL_SCRIPT begins to load, blocking the inline
script below that sets applicationCache.onchecking.
3) While EXTERNAL_SCRIPT is being requested, the task from step 1
fires, and the checking event gets dispatched, even though there is
no listener set for it yet.
4) Loading finally finishes with a failure, so main document parsing
resumes. Onchecking listener gets set, but it's too late now.
I can see both interpretations as valid. As a developer, I prefer Firefox's
implementation, because that would allow me put my applicationCache handlers
inside an external script and still be sure that the handlers work properly.
Otherwise, with WebKit's interpretation, it seems I am forced to use an inline
script to ensure the main event thread doesn't fire the events before my
handlers are added.
Can someone from Mozilla comment on how Firefox's implementation handles this?
It seems it defers Offline events until the body element is reached (I did a
search and found nsGlobalWindow::FireOfflineStatusEvent). It looks like many of
Firefox's tests depend on this type of behavior.
Thanks,
Joseph Pecoraro
[1]: https://bugs.webkit.org/show_bug.cgi?id=29690
[whatwg] Reliably Minimize Reflows
A task that developers are often faced with is applying many style
updates to multiple Nodes in the DOM. An example would be showing/
hiding all divs with the foo style class:
// Hide multiple divs
var foos = document.getElementsByClassName('foo');
for (var i=0, len=foos.length; i len; ++i)
foos[i].style.display = 'none';
As I understand it, if there are 20 .foo elements then updating the
style on each _could_ cause 20 individual reflows. It would be best
if the developer can narrow that down to a single reflow. Minimizing
reflows is a well known suggestion for speeding up web pages [1][2].
The most natural and generalized API that I can think of would be
something like:
// Single reflow would be triggered at the end of batchUpdate
document.batchUpdate(function() {
...
});
Opera mentions a little about its reflow algorithms at [2] with raises
some interesting points. Here is the relevant portion:
Browsers may choose to wait until the end of a script thread before
reflowing to show the changes. Opera will wait until enough changes
have been made, enough time has elapsed, or the end of the thread is
reached. This means that if the changes happen quickly enough in the
same thread, they may only produce one reflow. However, this cannot be
relied on, especially considering the various different speeds of
devices that Opera runs on.
The idea here is that batchUpdate would be a reliable way to
minimize reflows. Does this sound like a worthwhile improvement?
Cheers,
Joseph Pecoraro
[1]: http://code.google.com/speed/articles/reflow.html
[2]: http://dev.opera.com/articles/view/efficient-javascript/?page=3
Re: [whatwg] Reliably Minimize Reflows
// Single reflow would be triggered at the end of batchUpdate
document.batchUpdate(function() {
...
});
As a UA developer, I'd not be all that happy implementing this,
since you can stay inside the batchUpdate more or less forever (e.g.
by putting up alerts or doing sync XHR).
In any case, it seems unnecessary.
I was aware of the potential for a thread to sit inside a
batchUpdate. I guess the basic idea would have been that batchUpdate
would be a hint to the UA. However, I can't think of any other
situations where there are API functions that are hints so maybe
this isn't appropriate for a spec?
Browsers may choose to wait until the end of a script thread before
reflowing to show the changes. Opera will wait until enough changes
have
been made, enough time has elapsed, or the end of the thread is
reached.
This means that if the changes happen quickly enough in the same
thread,
they may only produce one reflow. However, this cannot be relied on,
especially considering the various different speeds of devices that
Opera runs on.
Note that Gecko will not perform layout while your script is
running, unless you ask for layout information. If Opera has a
different behavior, I assume they have a good reason for it, right?
Why are you sure you want to override that reason?
I don't think of this as overriding any reasons/behaviors. I thought
that this would do more to justify those reasons/behaviors.
- Joe
[whatwg] document.head
Was there any discussion for including document.head in HTML5?
Searching the mailing list shows document.head show up a few times in
example code [1][2]. However, there has been no proposal, and it is
not mentioned in the document's IDL [3] in the Spec.
Developers often do the following to get a reference to the head
element (top 2 google results) [4]:
var head = document.getElementsByTagName('head')[0];
Its useful for developers to use that reference to manipulate/append
stylesheets, scripts, and style elements which are only valid inside
the head when the scope attribute is absent [5].
I feel this would be useful to add. It is confusing that there is a
document.body but no document.head. If added, code that interacts
with the head would be much clearer then the current popular
approach (shown above). Also, the current approach, barring
optimizations in the engine, could search the entire DOM tree.
Cheers,
Joseph Pecoraro
[1]: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-March/018835.html
[2]: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2007-May/011561.html
[3]: http://www.whatwg.org/specs/web-apps/current-work/#documents-in-the-dom
[4]:
http://www.google.com/search?hl=enclient=safarirls=enq=get+head+element+with+javascriptaq=foq=aqi=
[5]: http://www.whatwg.org/specs/web-apps/current-work/#the-style-element
Re: [whatwg] document.head
On Sep 20, 2009, at 3:57 PM, Michael A. Puls II wrote:
I think it'd be cool to have to complement document.documentElement
and document.body.
On Sep 20, 2009, at 4: 00PM, Juriy Zaytsev wrote:
Surely better than abominable – `document.getElementsByTagName
('head')[0]` :)
I agree. Unfortunately that is the most popular method I've seen.
There are better solutions, but they haven't caught on. I think a
smarter idea would be to look at the children of the html element.
Something like this almost always works:
var head = document.documentElement.firstChild
- Joe
Re: [whatwg] document.head
On Sep 20, 2009, at 4: 24PM, Juriy Zaytsev wrote: Speaking of `document.head`, I think Mootools does exactly that. Good thinking. I took a look at some JavaScript Libraries / Frameworks. Here are some quick counts of how many times I could see that they use the getElementsByTagName method of getting the head element: - MooTools 1.2.3 has: 2 - jQuery 1.3.2 has: 2 - Ext 3.0.0 has: 5 - YUI 3.0.0b1 has: 10 - Prototype 1.6.1, MochiKit 1.4.2, Dojo 1.3.2 have: 0 I am not making any criticism of the libraries. I'm just re-enforcing the fact that it is a popular method of getting the head element, and adding document.head would be a worthwhile improvement. - Joe
Re: [whatwg] document.head
On Sep 20, 2009, at 10: 29PM, Garrett Smith wrote: On Sun, 20 Sep 2009 16:15:11 -0400, Joseph Pecoraro joepec...@gmail.com Something like this almost always works: var head = document.documentElement.firstChild The documentElement.firstChild cannot be expected to be head. It could be a text node. For example:- html head ... the first child node of HTML looks like a textNode with the value \n\n\u0020\u0020. document.getElementsByTagName(head)[0] could be expected to produce (in a valid HTML document) a result that is more consistent than document.firstChild. Even despite the white space, I got the head element in Safari, Firefox, and Opera. But, you're absolutely right, I'm not saying my suggestion is bulletproof, only that it can be expected to perform far better! However, I am interested in seeing a solution that is bulletproof and one that performs well! Namely, document.head. Cheers, - Joe
Re: [whatwg] Access the Response Headers for the Current Document
On Jul 28, 2009, at 9: 21PM, Ian Hickson wrote: Use Cases: Any that apply to XHR accessing their response headers would certainly apply here. Some thoughts are accessing the Content-Type header or Custom Headers and acting accordingly. You can just include the data straight into the page, for now. It's really clear what the use cases would actually be in practice. True, but that feels like a hack. If the HTTP protocol contains the data you need, then a server-side script may try to provide the data and may possibly provide an incorrect value. Likewise at the very least its a duplication of data being sent. This is certainly better then the current method, but not optimal. Come up with a clear description of the problem that needs to be solved: Cannot access the Response Headers for the current document in Javascript. Any there Browser Implementors out there that agree with this? If so, any thoughts on the best ways to expose the current page's request headers to Javascript? Certainly they are readonly, modifying them seems to be useless. How about keeping consistent with the XHR interface with something like: document.getAllResponseHeaders() and document.getResponseHeader(header) This is something that might make sense for a future version, but in the absence of a compelling need for this, I'm going to skip adding this in this version. I originally helped someone in an IRC channel with this question. He wanted to check a Date header being sent from his server, via Javascript. I don't know what his exact reason was. We provided him the same solutions mentioned here. However, like Adam de Boor suggested, a use case could be detecting proxies. The use case that I thought of was using custom headers to ensure requests go to a certain server in a cluster, perhaps to maintain a session with a reasonable cache. But that isn't really compelling and probably not very common.
Re: [whatwg] A New Way Forward for HTML5
I think we need an approach that doesn't involve in-flow links... I'm just not sure what the right solution is. Maybe alt-double-clicking should show a menu with two options, submit comment here or change section status? Alt-Double Click doesn't sound very discoverable. Even if I knew that shortcut I'd probably forget at some point. Maybe having something position:fixed would be better because there is something visible and reachable at all times. The problem then would be automatically determining which section is being commented on. Its certainly not as straight-forward as clicking on the section. Just some ideas. I'm very interested in seeing a commenting system. - Joe
[whatwg] Access the Response Headers for the Current Document
It seems like an oversight that Javascript can read response headers off of XHR but not for the current document. So in order to find out the headers for the current document you would need to make another request, refetching the current page, to find that out [1]. Use Cases: Any that apply to XHR accessing their response headers would certainly apply here. Some thoughts are accessing the Content-Type header or Custom Headers and acting accordingly. Come up with a clear description of the problem that needs to be solved: Cannot access the Response Headers for the current document in Javascript. Any there Browser Implementors out there that agree with this? If so, any thoughts on the best ways to expose the current page's request headers to Javascript? Certainly they are readonly, modifying them seems to be useless. How about keeping consistent with the XHR interface with something like: document.getAllResponseHeaders() and document.getResponseHeader(header) Cheers, Joseph Pecoraro [1] Example: http://bogojoker.com/x/xhr/headers.html
Re: [whatwg] Browser Bundled Javascript Repository
On Jul 13, 2009, at 3: 01PM, Aryeh Gregor wrote: How about you have an extra HTTP header like X-Content-Hash? This could provide a SHA256 hash (or something else that looks safe for now, progressively upgradeable) of the content. The browser can keep its cached copies of these files indexed by hash. If it tries downloading a file, and notices that the hash is the same as a file already downloaded, it can terminate the HTTP connection and use the existing file (even if it's from a different site). It will then proceed as though it had actually downloaded the file: e.g., it will respect the Expires headers separately (two sites might serve the same file but have different expectations about how likely it is to change). I think thats brilliant. Its a cache that works across all sites. On Jul 13, 2009, at 3: 01PM, Aryeh Gregor wrote: The most obvious place to solve this seems to be HTTP, not HTML. HTTP is closer to the resource itself. If you do something with HTML, like an extra link attribute, then you're going to get authors updating the HTML but not the thing it points to or vice versa. An ETag-like solution would be implemented either in the web server or whatever script is serving the content, and those should always know whether the file has changed. (Modulo pathological behavior like something changing the file and then forging the mtime/ctime.) I agree. On Jul 13, 2009, at 4: 20PM, Aryeh Gregor wrote: Does anyone have statistics on how useful this would be in real life? I suspect only marginally. I think this is the most important aspect of this idea. We don't yet know if this is worth doing yet. It is likely only preventing the initial download of some files. However some of the initial framework sizes are getting hefty: (These were the sizes just pulled from google's hosted libraries for the latest versions) 91K dojo.xd.js 79K ext-core.js 182K jquery-ui.min.js 56K jquery.min.js 127K prototype.js 2.6K scriptaculous.js 10K swfobject.js 27K yuiloader-min.js I'm guessing that mobile browsers would benefit from not needing to download a few of those 100KB downloads and use up an HTTP Connection to do so. And speaking of mobile, there are some mobile specific web application frameworks (I'm thinking of ones for iPhone web apps) that don't have a single cache point (like google) that weigh in pretty heavily. Real statistics would make it obvious wether or not this is a good idea. But even still, I like the HTTP idea, because at that point its just a more efficient way to cache files, by content, across all the entire web, rather then site specific and by name/URL. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Some quick feedback before I heed Ian's advice and try to formalize things a little more. As a browser vendor (or at least someone working for one, specifically on script-loading), I would definitely be interested in something like this. However there are some hard problems to be solved: I looked through the mailing list and I take it you're from Mozilla. Correct me if I'm wrong. =) 1. How do we choose which libraries to pre-package? I'd rather not be in charge of ruling which libraries are popular enough or cool enough to warrant inclusion. It'd be very nice if the library cache were populated dynamically as the user used the web. Yes this point did come up. I believe that the cache can be grown dynamically, but I don't think that the cache could be reliably used without some indication or opt-in from the html (and thus the developer or some automated process). Files can be compared with each other very quickly, and SHA hashes even quicker, to determine that indeed the same content is being sent from multiple URLs, thus allowing potential for the cache to grow dynamically with frequently encountered content. But, again, I think the problem is then on the developer to opt-in to the service, otherwise the cache is just a speculation. 2. How do we deal with identifying libraries. As Aaron Boodman pointed out, SHA hashes means that you can't make upgrades for security problems etc. I still don't see this as an issue. As long as it is a SHA hash of the content, then its the content that matters. If a developer is using a script with a security problem then they should change their script. At this point a browser could go the extra mile and attempt to fix it for them by updating the script but thats another speculation. The idea of canonical names could run into problems though. 3. Compat when the browser doesn't have a library cached. A solution that includes using a different uri for browsers that do support caching than browsers that don't is scary since there is a big risk that the author will forget to update one but not the other. Yes, I do see this as a problem. I gave a response to this before, but I do still see this as being a potential headache point. Tools can easily solve this but thats not a perfect solution. I haven't come up with a good solution to this, but I do have some ideas. I believe all these problems are solvable, but they do need to be solved before considering inclusion in any spec. I'm also not convinced this needs to be included in the HTML5 spec, but that might depend on what the ultimate solution looks like. I agree again. I concluded this feature makes the most sense for a restricted environment, such as developing mobile web apps. Those developers are already faced with some different conditions, and I think iPhone web development even has some custom syntax. A feature like this could make a much bigger impact in that kind of environment where it could gain traction. / Jonas Thanks for the feedback Jonas. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
We could make Storage into an EventTarget and also fire events on there, sure. As Jeremy said, this is something that might best be done in the next version; we're still trying to get the current features implemented in a stable and consistent fashion as it is. Sounds great. Although wouldn't now be the best time to introduce something new? Its not a breaking change (theoretically), its an additional feature. Its my understanding that most of Web Storage is undergoing changes (B-trees and SQL orthogonally), but the Storage interface is not going to change much? Is that because of current browser support? In general, though, I don't think it's critical, since you can easily work around it by checking which was changed based on the event data Fair enough. =) (I agree that your proposal would have made more sense. I'm sorry my learning experiences have such a detrimental effect on the Web! :-) ) We're all learning. =P -- Ian Hickson U+1047E) \._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _ \ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'-- (,_..'`-.;.' - Joe
Re: [whatwg] Browser Bundled Javascript Repository
I would recommend approaching the browser vendors directly and seeing if they would be interested in implementing this idea, as discussed in this FAQ entry on introducing new features: http://wiki.whatwg.org/wiki/FAQ#Is_there_a_process_for_adding_new_features_to_the_spec.3F I see, thanks Ian. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
This doesn't break anything in the current spec. So it wouldn't break any existing implementations. I'm also guessing that the groundwork for implementing a feature like this is already in place due to the ubiquity of addEventListener. To be frank, it seems like a lot of bloat though to avoid a simple comparison. Its my thinking that the simple comparison shouldn't be necessary to begin with. Thats why I suggested this. This is a probably a bad analogy, but would it make sense to put a click function on just the window object and check every time it fires to see if it fired in the div you were interested in? Imagine every single click listener you register fires every time you click on something... that doesn't scale. (The reason this is a bad is because these click events bubble. Maybe bubbling is appropriate here)? - Less Listener Functions Fired - Instead of every registered listener getting fired on every storage event, only those applicable will be fired. This may mean overall less listeners getting fired, and code that doesn't have to continually check the affected storageArea, leading to potential performance improvements. Actually, assuming only one of the two solutions would exist the same amount of events would fire. A change to localStorage causes an event to be dispatched and likewise a change to sessionStorage causes an event to be dispatched. Having said that, with your solutions more events will fire since the legacy event will have to be dispatched too. What do you mean when you say more events will fire? Why not just fire the same storage event but now put the if statement that the web developer had to always write and put it into the browser's dispatching code. This may be my inexperience with the implementations, but I honestly don't see how this could end up firing more events. I think looking at it this way (more events) is looking at it sideways and maybe a bit pessimistically. If my first thought is this is going to be really complex then my next thought is how can I simplify this and if I can't find a way to simplify it I would state why it can't or shouldn't be done. I don't see that here, I just see the this is really complex part. I'm guessing that there is a simpler way to look at this, and if not I'd be interested to know why this seems so complex. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
Jeremy Orlow: Is it too late? It seems as though Joseph's suggestion could be in addition to what's already in the spec. This doesn't break anything in the current spec. So it wouldn't break any existing implementations. I'm also guessing that the groundwork for implementing a feature like this is already in place due to the ubiquity of addEventListener. Pros: - Less Developer Confusion: When a web developer is focusing on a particular Storage area and set a storage listener, unless they check the storageArea they may not realize their listener is getting fired for scripts dealing with a different Storage area. I'm not sure if this can be considered a security threat but it would certainly limit the potential for problems. - Less Listener Functions Fired - Instead of every registered listener getting fired on every storage event, only those applicable will be fired. This may mean overall less listeners getting fired, and code that doesn't have to continually check the affected storageArea, leading to potential performance improvements. Cons: - This is similar to implementing the full EventTarget Interface on Storage areas. However, the only event of importance on a Storage area is storage not click or hover etc. Invalid event types should probably throw an error or something... or maybe this is more in the scope of DOM Events. Anne: By the way, change requests for this specification should be made to public-weba...@w3.org. localStorage and friends are no longer part of HTML5. I read from your (Anne's) blog that Web Storage has split off of HTML5, and I see it on the WebApps homepage [1] but the document itself said to send feedback to either mailing list or the bugzilla [2]. I just subscribed. In the future I'll use the other list for whats on the WebApps homepage. [1]: http://www.w3.org/2008/webapps/charter/ [2]: http://dev.w3.org/html5/webstorage/#status-of-this-document - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
Seems like a reasonable idea. I don't know if it's been brought up before. My guess is that Ian will say this should be re-examined for HTML6, though. Very cool. Let me know if I can help at all to push this idea forward. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
It sounds like there wasn't any discussion on this. I recently heard
talk of other potential Storage areas [2]. That would make this idea
even more appealing to me. Does this sound like something worth
adding? Any comments?
[2]: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020485.html
On Jun 17, 2009, at 1: 44PM, Joseph Pecoraro wrote:
The storage event [1] fires for both sessionStorage and
localStorage. To me, this means if you only want to interact with
localStorage you will have to manually ensure that it is the storage
area being modified:
window.addEventListener('storage', function(e) {
if ( e.storageArea === localStorage ) {
// ...
}
}
Was there any discussion about creating events specific to the
storage object, or should that already be possible? I've been
playing around with WebKit's Storage implementation, and the
following (understandably) is not possible:
localStorage.addEventListener
undefined
Is there any way to listen to events for a single specific storage
area or is the previously mentioned approach preferred?
Cheers,
Joe
[1]: http://dev.w3.org/html5/webstorage/#the-storage-event
[whatwg] Storage Events for a Specific Storage Area
The storage event [1] fires for both sessionStorage and
localStorage. To me, this means if you only want to interact with
localStorage you will have to manually ensure that it is the storage
area being modified:
window.addEventListener('storage', function(e) {
if ( e.storageArea === localStorage ) {
// ...
}
}
Was there any discussion about creating events specific to the storage
object, or should that already be possible? I've been playing around
with WebKit's Storage implementation, and the following
(understandably) is not possible:
localStorage.addEventListener
undefined
Is there any way to listen to events for a single specific storage
area or is the previously mentioned approach preferred?
Cheers,
Joe
[1]: http://dev.w3.org/html5/webstorage/#the-storage-event
[whatwg] Browser Bundled Javascript Repository
Hey Guys, This is my first time on the list, I searched the Archives but I didn't see anything like this so I apologize if I missed any earlier discussion on something like this. A while back I came across this two paragraph blog post titled Browsers Should Bundle JS Libraries: http://fukamachi.org/wp/2009/03/30/browsers-should-bundle-js-libraries/ The premise is basically that browsers are repeatedly downloading the same javascript frameworks from different domains over and over every day. In the author's own words: All popular, stable Javascript libraries, all open source. All downloaded tens of millions of times a day, identical code each time. Below is a summary and expansion of my comments/ideas from the discussion on the above blog article. A typical solution to the problem, and one that works right now in browsers, is that if you require a javascript library on your website you can point to a publicly available version of that library. If enough sites use this public URI then the browser will continually be using that URI and it will be cached and reused by the browser. This is the idea behind Google's Hosted Libraries: http://code.google.com/apis/ajaxlibs/ There are some arguments against using Google's Hosted Libraries: http://www.derekallard.com/blog/post/jquery-hosted-on-google-and-some-implications-for-developers/ However, I think the author makes a good point. Bundling the JS Libraries in the Browser seems like it would require very little space, could even be stored in a more efficient representation (compiled bytecode for example), and would prevent an extra HTTP Request. The problem then becomes how does a browser know example.com's jquery.js is the same as other.com's jquery.js. The developer should opt-in to telling the browser it wants to use a certain JS Library version that the browser may already know about. The way I thought about it was by adding an attribute to the script tag. In my comments, I used the rel attribute because of developer's familiarity with it in other tags, but it could (and probably should) be an entirely new attribute. The value inside of this attribute would need to be a unique identifier for a possible script available in the browser's repository. The src attribute should still point to a hosted version of the script in case this attribute is unsupported (ignored) or the script is not found in the repository (not-bundled). For Example: !-- SHA1 hash as identifier for jquery-1.2.3 -- script rel=A56F2CED6... src=... / !-- Canonical name as an identifier for a JS lib and version -- script rel=jquery-1.2.3 src=... / Here the rel attribute's value is a standard identifier for a particular version of the JQuery JS Library. The browser could check its Repository to see if it has it. If found, no request is needed and it can load its local version. If not found it can proceed like normal using the src attribute to download the script. Pros: - Future-Proof: Adding a new attribute, or using a currently ignored attribute, on the script tag would make this a safe addition that works fine in older browsers (backwards compatible) and works instantly in supported browsers. - Developer Opt-In: Developers that choose not to use this feature could just ignore it. - Pre-Compiled: By bundling known JS Libraries with the browser, the browser could store a more efficient representation of the file. For instance pre-compiled into Bytecode or something else browser specific. - Less HTTP Requests / Cache Checks: If a library is in the repository no request is needed. Cache checks don't need to be performed. Also, for the 100 sites you visit that all send you the equivalent jquery.js you now would send 0 requests. I think this would be enticing to mobile browsers which would benefit from this Space vs. Time tradeoff. - No 3rd Party is Gathering Statistics: One of the arguments against using Google's Hosted Libraries is that you send them some data if you are indeed using their scripts and a client downloads from them (Referrer, etc.). Here there is no 3rd party, its just between the client browser and domain. - Standardizing Identifier For Libraries: Providing a common identifier for libraries would be open for discussion. The best idea I've had would be to provide the SHA1 Hash of the Desired Release of a Javascript Library. This would ensure a common identifier for the same source file across browsers that support the feature. This would be useful for developers as well. A debug tool can indicate to a developer that the script they are using is available in the Browser Repository with a certain identifier. - Repository Can Grow Dynamically - Assuming this is a desirable feature that shows some promise, the browser repository can grow dynamically. Browsers can count the number of times they have seen equivalent
Re: [whatwg] Browser Bundled Javascript Repository
Pros: - Pre-Compiled: By bundling known JS Libraries with the browser, the browser could store a more efficient representation of the file. For instance pre-compiled into Bytecode or something else browser specific. I think something needs to be clarified wrt to compile times and the like. In the WebKit project we do a large amount of performance analysis and except in the most trivial of cases compile time just doesn't show up as being remotely significant in any profiles. Additionally the way JS works, certain forms of static analysis result in behaviour that cannot reasonably be cached. Finally the optimised object lookup and function call behaviour employed by JavaScriptCore, V8 and (i *think*) TraceMonkey is not amenable to caching, even within a single browser session, so for modern engines i do not believe caching bytecode or native is really reasonable -- i suspect the logic required to make this safe would not be significantly cheaper than just compiling anyway. I noticed this came up on the WebKit mailing list recently and they said the same thing as you, that compile time was insignificant. Thanks for expanding on this: https://lists.webkit.org/pipermail/webkit-dev/2009-May/007657.html Although this response sounded slightly more promising: https://lists.webkit.org/pipermail/webkit-dev/2009-May/007682.html - Less HTTP Requests / Cache Checks: If a library is in the repository no request is needed. Cache checks don't need to be performed. Also, for the 100 sites you visit that all send you the equivalent jquery.js you now would send 0 requests. I think this would be enticing to mobile browsers which would benefit from this Space vs. Time tradeoff. I believe http can specify how long you should wait before validating the cached copy of a resource so i'm not know if this is a real win, but i'm not a networking person so am not entirely sure of this :D I believe you're correct. HTTP E-Tags and Expires headers can influence browsers to cache resources for very long times. Unfortunately I don't think that a lot of developers take advantage of these tags (this is a bad argument but worth mentioning). Although Yahoo's Y-Slow and Google's Page-Speed extensions have opened many developers eyes to ways they can improve their site's performance. The real gain however, would be if you visited 100 different websites that all needed the same script and you wouldn't have to make a single request, or cache a single resource, due to using the script in the repository. I think this sounds better. - Standardizing Identifier For Libraries: Providing a common identifier for libraries would be open for discussion. The best idea I've had would be to provide the SHA1 Hash of the Desired Release of a Javascript Library. This would ensure a common identifier for the same source file across browsers that support the feature. This would be useful for developers as well. A debug tool can indicate to a developer that the script they are using is available in the Browser Repository with a certain identifier. This isn't a pro -- it's additional work for the standards body You are correct. I'm not too familiar with the process behind the specifications (although I would like to learn). Maybe including this as a Pro was premature, but having a hash value like a SHA1 be the identifier has a number of advantages. Maybe there are better solutions out there that have the same advantages that a SHA1 would provide. Cons: - May Not Grow Fast Enough: If JS Libraries change too quickly the repository won't get used enough. - May Not Scale: Are there too many JS Libraries, versions, etc making this unrealistic? Would storage become too large? - Adds significant spec complexity - Adds developer complexity, imagine a developer modifies their servers copy of a given script but forgets to update the references to the script, now they get inconsistent behaviour between browsers that support this feature and browsers that don't. Significant spec complexity? I'm too inexperienced to know. =( As for the developer scenario this is similar to modifying any single attribute on a tag and not appropriately modifying the others. Change the src on an img and not changing the alt. Change the href on a link and not changing the media. I think this could be easily avoided with validation. If the unique identifier were a SHA1 hash and the referenced script src does not hash to the provided value, then the page would invalidate with an error/warning. However, you raise a good point, and I can't come up with any truly equivalent analogy to any other existing developer specific problem. --Oliver Thanks for the points Oliver. Looks like a few of the Pros may have been eliminated. Do you think this could produce any noticeable improvements? - Joe
Re: [whatwg] Browser Bundled Javascript Repository
The common JavaScript libraries should be identified using urn scheme with JavaScript namespace, as in script src=urn:JavaScript:cool-acme-lib:1.0 /script Chris Chris, this was what I was originally thinking of when I read about tag-uri's in the Atom Publishing Protocol (which I think are now defunct). URN Schemes: http://www.w3.org/TR/uri-clarification/ I don't know too much about URN Scheme's but I don't think they provide as much benefit as a hash would be (ugly as they are). In the original email I alluded to canonical or easy to use identifiers that would be alias's for the hash: !-- Canonical name as an identifier for a JS lib and version -- script rel=jquery-1.2.3 src=... / Maybe a URN Scheme would be optimal for a canonical name, but I still like the SHA1 hash for verification/validation and dynamic growth. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Dion: The problem here is that isn't backwards compatible and thus no-one will really be able to use it. I thought the original idea was backwards compatible. Maybe not the URN Schemes. If the original idea is not, could you point out the issues? Dion: You then also get into the how do I get my library into the browser? Enough widespread usage of a library is a clear indicator for adoption into a browser bundle. Dynamically growing repositories could optimize per computer for the particular user's browsing habits (assuming developers would mark their scripts with the identifiers). You can have the same problem with what libraries will Google include in its CDN. Although it may be easier for Google to host just about any library if it already has a CDN setup. Dion: After mulling this over with the Google CDN work, I think that using HTTP and the browser mechanisms that we have now gives us a lot without any of these issues. I was afraid of this. This is a completely valid point. I guess it sounds like too much work for too little gain? - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Chris Holland: But you're right, this is all a lot of end-user
intervention: it would
be a slightly, err, very painful process of installing a browser
plugin, which is currently very-much of a user opt-in process, and not
something very practical.
[...]. I'm just trying to find ways to leverage a lot of what's
already there.
Yes, I don't like the idea of requiring users to act. But I see what
you were tying to do with reuse.
Interesting thing is the same scheme could be leveraged for local CSS
extensions:
I thought about this as well, but CSS is far less likely to be
duplicated across multiple sites. There are plenty of CSS Frameworks
but I feel that none have picked up enough dominance for this kind of
optimization to be useful.
You do mention what looks like urn schemes and extending this idea to
CSS. I was specifically thinking of javascript because of its
widespread use of libraries/frameworks. Using URN Schemes could let
this repository idea extend to more then just javascript, however I
don't think any other type of resource (CSS, Images, Etc.) have this
unique pattern of the exact same content being served on thousands of
different domains.
link rel=local:extension type=text/css href=ext:ibdom.
0.2.js /
To handle users who don't have the ibdom javascript extension
installed, developers could add something like this to their document:
(assuming a decent library which declares a top-level
object/namespace):
script type=text/javascript
if (!window.IBDOM) {
var newScript = document.createElement(script);
scr.setAttribute(type,text/javascript);
scr.setAttribute(src,/path/to/ibdom.0.2.js);
document.appendChild(newScript);
}
/script
Although the idea is the same (have a fall back plan if a repository
lookup is ignored or fails), I think this is needlessly complex
compared to just adding a new attribute on the script tag. By
extending the script tag you already have fall back behavior to just
download the script from the src attribute.
If you take the link approach then you're practically requiring that
you need to write fault tolerant code like you showed above, and that
is no fun for web developers.
--
The more I think about it, the more I think this might not necessarily
be a web standards idea. More of a browser optimization, however it
would never take off unless it was standardized. I don't know what to
do about this... CDN Caching, like Google's Hosted Libraries, is more
generic but less optimized. Maybe this is just a special case?
- Joe
Re: [whatwg] Browser Bundled Javascript Repository
On Mon, Jun 15, 2009 at 1:09 PM, Joseph Pecoraro joepec...@gmail.com wrote: Dion: The problem here is that isn't backwards compatible and thus no-one will really be able to use it. I thought the original idea was backwards compatible. Maybe not the URN Schemes. If the original idea is not, could you point out the issues? The URN schemes isn't compatible. The SHA hash idea is do-able, but as Oliver pointed out is impractical: a) devs will forget to update it, b) looks ugly, c) fun things would happen with a SHA collision! ;) a) Solved by Validation - I can't think of anything much better then that. =( b) Canonical Listing - This shouldn't be too difficult to distribute from a central source or some convention. c) Hehe, I think I detect a hint of sarcasm. If there is a SHA1 collision then you'd probably make a lot of money! Dion: You then also get into the how do I get my library into the browser? Enough widespread usage of a library is a clear indicator for adoption into a browser bundle. Dynamically growing repositories could optimize per computer for the particular user's browsing habits (assuming developers would mark their scripts with the identifiers). You can have the same problem with what libraries will Google include in its CDN. Although it may be easier for Google to host just about any library if it already has a CDN setup. This was a real problem for us. How much is enough ? We started to get inundated with requests for people to put libraries up there. Lets the browsers decide. And I can't make any reasonable suggestions without getting real world data, something I haven't tried to do yet. But yes, this is a good point, something that is extremely flexible / variable. Dion: After mulling this over with the Google CDN work, I think that using HTTP and the browser mechanisms that we have now gives us a lot without any of these issues. I was afraid of this. This is a completely valid point. I guess it sounds like too much work for too little gain? I don't want to stop you from working on these ideas. The core problem that we tend to download the same crap all the time is real, and I look forward to seeing people come up with interesting solutions. Thanks for the support. My thoughts are beginning to look like this: - Javascript Frameworks are downloaded all the time on many domains. This is a special case. - Those who benefit the most are the ones that can't space the extra request or large caches. This makes me think mobile browsers would get the biggest benefit. - I think the iPhone had some special html syntax for its mobile webpages, maybe they can sneak this in if it proves useful to them. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
c) fun things would happen with a SHA collision! ;) c) Hehe, I think I detect a hint of sarcasm. If there is a SHA1 collision then you'd probably make a lot of money! C is a serious concern. SHA-1 collisions are now 2^51 - http://eprint.iacr.org/2009/259.pdf This time I didn't detect sarcasm =) I was actually aware of that paper. I saw it on Reddit this past week, and although they complained about the fact that it has not yet been reviewed I think it could very well be valid. Its been known that SHA1 has been theoretically broken (not perfect 2**80) for some time now: (2005) http://www.schneier.com/blog/archives/2005/02/sha1_broken.html However, its application in this Repository idea is not to be a cryptographically secure hash, it would just be to perform a quick, reliable, hash of the contents and to produce a unique identifier. There would be no security concerns in the impossibly rare chance that two scripts hashes collide. Just add some whitespace to the text somewhere! It would even be easy to debug when with standard tools such as Firefox's Firebug and Webkit's Web Inspector. Hahaha =) Also, Git and Mercurial (distributed version control systems) have been using SHA1 for the exact same purpose for years. I'm more familiar with Git's use of SHA1 and it uses it everywhere in the internals (file contents, directory listings, commit history). Finally, if anyone here is seriously concerned with SHA1 just move to SHA-256 or SHA-512. With a repository unlikely to grow into the thousands, much less the millions, the chances of a collision even in 2**51 (2251799813685248 base 10) is bold thinking ;) I'm not attacking anyone here, I'm just clarifying why I think SHA1 is not a bad choice. Collision will always be an issue when a infinite number of things gets reduced to a finite set of values, but the concern negligible when done right. Cheers - Joe
Re: [whatwg] Browser Bundled Javascript Repository
In the event of a collision there would be huge issues - imagine running someone else's script in your application. Basically XSS - someone could take over your app, steal passwords, do bank transactions on your behalf, etc. Collisions are made easier in plain text than in certs given that your input is not constrained. Aaron Boodman: I think the idea was for browser vendors to select and include these libraries in the browser. So there isn't an obvious (to me) way for an attacker to use hash collisions to create an XSS. Yes, thanks for clearing that up. That said, I don't think content hashes are the right identifier. Using a sha-1 of a specific jquery version would prevent anyone from ever fixing critical bugs in it. There's be all this legacy content out there referring to an outdated version. Assuming there is a buggy version of a JS library: 1. It probably shouldn't be used. 2. The browser vendor can (and should) eliminate it from their repository and proceed with the usual fallback of downloading the script from the script's src attribute would take effect. I could see where this could get confusing. JS Library X gets released and tells its users to use SHA1 ABC Thousands of people download it, and later that day they fix an issue, update their site and say use this new version with SHA1 FDE Canonical listings would makes this easier. - Joe
