Re: [whatwg] Offline Web Apps
On Oct 11, 2007, at 6:47 PM, Robert O'Callahan wrote: On Oct 12, 2007 12:53 PM, Ian Hickson [EMAIL PROTECTED] wrote: The problem with isLocallyAvailable() -- as noted by Maciej on IRC -- is mostly one of race conditions. What if the resource was removed in between you asking for it and using it? Or added? In the contexts for which it was requested, race conditions are tolerable. For example in a mapping application, if you choose the wrong tile as a template for zooming, then you get an ugly transition but that's all. Seems like wrongly choosing one that is not actually locally available would leave a hole in your map. Or you might have a button that opens another offline application, and you want to disable the button if you think that application is not available; here a race condition would probably mean that the button is enabled but pressing it gives you an error page. In these cases isLocallyAvailable can be a useful hint even if it's occasionally wrong. Or what if you go offline (thus changing the rules) in between checking for and using the resource? That's why we have the 'whenOffline' parameter. I don't see how the whenOffline parameter addresses that problem that your online/offline state may change between the time you do the check and the time you try to load the resource. In fact, I'm not really sure how it is helpful, compared to just basing the check on the current online/offline state always. I'm going to punt on this for now, pending implementation and author experience. I'm certainly open to suggestions though. That's fine. A race-free API seems difficult to design. Because loads are typically asynchronous, I doubt you can do much better than start a regular load, and set a timer to go off and cancel the load and take other action if the load doesn't complete fast enough. The race-free approach would be to provide APIs to load resources only from the cache and to error out immediately if the the request can't be served locally. The problem with this is that there are a huge number of APIs to trigger a load: frame src, iframe src, window.location assignment, window.location.replace, img src, script src, link for a stylesheet, XMLHttpRequest, and that's just the obvious ones. I think a way to do an XMLHttpRequest that will only succeed if the data is cached would cover many cases, and is probably a reasonable extension to XHR (given the text of an html document, script or stylesheet, you can insert it into an appropriate element for use). The problem is binary formats, since XHR has no good way of returning the data yet and the elements that would consume them (img, video, audio, object) have no way to pass in raw bytes (using a data: URL to encode it would be poor form for image and probably unworkable for video). However, if those elements were extended with some way to set raw data then such an approach could work. At that point, isLocallyAvailable might possibly be useful solely as a UI hint, much as the POSIX access() / stat() calls are only useful as UI hints for openability. In fact, this is pretty much analogous to the potential pitfalls of access() vs. open(). You need to check whether open() succeeds, and pass it the right parameters to make sure it will fail if it shouldn't succeed, rather than trusting a pre-check with access() or stat(). Regards, Maciej
Re: [whatwg] Offline Web Apps
On Fri, 12 Oct 2007, Maciej Stachowiak wrote: The race-free approach would be to provide APIs to load resources only from the cache and to error out immediately if the the request can't be served locally. That's what the offline caching system does right now for any URI not on the whitelist. We could have some sort of API (I'm not sure exactly what it would look like) to modify the whitelist, adding items temporarily or something? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Oct 12, 2007, at 12:16 PM, Ian Hickson wrote: On Fri, 12 Oct 2007, Maciej Stachowiak wrote: The race-free approach would be to provide APIs to load resources only from the cache and to error out immediately if the the request can't be served locally. That's what the offline caching system does right now for any URI not on the whitelist. We could have some sort of API (I'm not sure exactly what it would look like) to modify the whitelist, adding items temporarily or something? That's a good point. However, it seems like the most likely candidate for a resource to use only if local is one that's in an opportunistic caching namespace in the manifest. At least, that is how I see Robert's map scenario working. And you wouldn't want either a real version loaded from the network or the fallback resource. One possible approach would be an API to temporarily turn an opportunistic caching namespace into cache-only, such that any loads initiated while in this mode are served only if cached and fail otherwise, bypassing the possibility of network load or getting the fallback resources. Then you can handle the error in case the load fails. Regards, Maciej
Re: [whatwg] Offline Web Apps
On Oct 12, 2007 9:39 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Oct 11, 2007, at 6:47 PM, Robert O'Callahan wrote: On Oct 12, 2007 12:53 PM, Ian Hickson [EMAIL PROTECTED] wrote: The problem with isLocallyAvailable() -- as noted by Maciej on IRC -- is mostly one of race conditions. What if the resource was removed in between you asking for it and using it? Or added? In the contexts for which it was requested, race conditions are tolerable. For example in a mapping application, if you choose the wrong tile as a template for zooming, then you get an ugly transition but that's all. Seems like wrongly choosing one that is not actually locally available would leave a hole in your map. This case was for an online app. The assumption is that either tile chosen would load eventually; the app would choose to base its transition effect on whichever tile was most likely to be available. I'm told that right now, they use heuristics based on a completely naive model of the browser cache! Or what if you go offline (thus changing the rules) in between checking for and using the resource? That's why we have the 'whenOffline' parameter. I don't see how the whenOffline parameter addresses that problem that your online/offline state may change between the time you do the check and the time you try to load the resource. In fact, I'm not really sure how it is helpful, compared to just basing the check on the current online/offline state always. Some apps may find it helpful to be able to predict their offline capabilities even while they're online. For example, an app might want to indicate what the user will be able to do if they go offline right now. A race-free API seems difficult to design. Because loads are typically asynchronous, I doubt you can do much better than start a regular load, and set a timer to go off and cancel the load and take other action if the load doesn't complete fast enough. The race-free approach would be to provide APIs to load resources only from the cache and to error out immediately if the the request can't be served locally. The problem with this is that there are a huge number of APIs to trigger a load: frame src, iframe src, window.location assignment, window.location.replace, img src, script src, link for a stylesheet, XMLHttpRequest, and that's just the obvious ones. Indeed. Another problem is that if you want to offer a hard guarantee that the resource will load, you probably want to check not just whether it's in the cache but also that no other error will occur before the load completes (e.g. due to a change in the browser state during a load). This might be simple, or it might not, depending on the browser implementation. In other words, the race window doesn't end when the load starts, it actually ends when the load ends IMHO. I think a way to do an XMLHttpRequest that will only succeed if the data is cached would cover many cases, and is probably a reasonable extension to XHR (given the text of an html document, script or stylesheet, you can insert it into an appropriate element for use). The problem is binary formats, since XHR has no good way of returning the data yet and the elements that would consume them (img, video, audio, object) have no way to pass in raw bytes (using a data: URL to encode it would be poor form for image and probably unworkable for video). However, if those elements were extended with some way to set raw data then such an approach could work. Yes, but I think you'd want this API to be synchronous for the above reasons. At that point, isLocallyAvailable might possibly be useful solely as a UI hint, much as the POSIX access() / stat() calls are only useful as UI hints for openability. In fact, this is pretty much analogous to the potential pitfalls of access() vs. open(). You need to check whether open() succeeds, and pass it the right parameters to make sure it will fail if it shouldn't succeed, rather than trusting a pre-check with access() or stat(). Agreed. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Thu, 11 Oct 2007, Robert O'Callahan wrote: -- Several Web app authors have asked for the ability to test whether a resource is cached, for their online apps. For example, when you're zooming in and out of a map, the application could choose which tile(s) to use for the animation by scaling them up or down. This would also be convenient for offline use, where a resource might not necessarily be in the offline cache but you could use it if it happened to be available. Interesting. This isn't covered yet, should we add it immediately? I think it should be added *somewhere*, but it doesn't have to be part of the offline spec itself (although it is particularly useful when you're offline). The problem with isLocallyAvailable() -- as noted by Maciej on IRC -- is mostly one of race conditions. What if the resource was removed in between you asking for it and using it? Or added? Or what if you go offline (thus changing the rules) in between checking for and using the resource? I'm going to punt on this for now, pending implementation and author experience. I'm certainly open to suggestions though. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Oct 12, 2007 12:53 PM, Ian Hickson [EMAIL PROTECTED] wrote: The problem with isLocallyAvailable() -- as noted by Maciej on IRC -- is mostly one of race conditions. What if the resource was removed in between you asking for it and using it? Or added? In the contexts for which it was requested, race conditions are tolerable. For example in a mapping application, if you choose the wrong tile as a template for zooming, then you get an ugly transition but that's all. Or you might have a button that opens another offline application, and you want to disable the button if you think that application is not available; here a race condition would probably mean that the button is enabled but pressing it gives you an error page. In these cases isLocallyAvailable can be a useful hint even if it's occasionally wrong. Or what if you go offline (thus changing the rules) in between checking for and using the resource? That's why we have the 'whenOffline' parameter. I'm going to punt on this for now, pending implementation and author experience. I'm certainly open to suggestions though. That's fine. A race-free API seems difficult to design. Because loads are typically asynchronous, I doubt you can do much better than start a regular load, and set a timer to go off and cancel the load and take other action if the load doesn't complete fast enough. Overkill if all you want is the hint, and since authors can already do that but there's still demand for isLocallyAvailable, I suspect that's not really what's wanted. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On 10/10/07, Ian Hickson [EMAIL PROTECTED] wrote: On Fri, 21 Sep 2007, Robert O'Callahan wrote: -- Several Web app authors have asked for the ability to test whether a resource is cached, for their online apps. For example, when you're zooming in and out of a map, the application could choose which tile(s) to use for the animation by scaling them up or down. This would also be convenient for offline use, where a resource might not necessarily be in the offline cache but you could use it if it happened to be available. Interesting. This isn't covered yet, should we add it immediately? I think it should be added *somewhere*, but it doesn't have to be part of the offline spec itself (although it is particularly useful when you're offline). We have an experimental implementation: http://mxr.mozilla.org/seamonkey/source/dom/public/idl/base/nsIDOMClientInformation.idl#55 That's ... underdocumented ... but the idea is this: boolean http://mxr.mozilla.org/seamonkey/ident?i=boolean isLocallyAvailable http://mxr.mozilla.org/seamonkey/ident?i=isLocallyAvailable(in DOMString http://mxr.mozilla.org/seamonkey/ident?i=DOMString uri http://mxr.mozilla.org/seamonkey/ident?i=uri, in boolean http://mxr.mozilla.org/seamonkey/ident?i=boolean whenOffline); It should return true when the browser will load the resource given by the URI without contacting the network. If 'whenOffline' is true then this putative load is assumed to happen while the browser is offline/using its offline caching functionality, if any. This is intended to allow an app to predict, while the user is online, what loads will succeed when the user goes offline, and in Gecko, for example, we do not perform HTTP cache validation while the browser is in offline mode, so a page might be locally available in offline mode but not in online mode. But we could probably get rid of this parameter if people think it doesn't make sense. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Fri, 24 Aug 2007, Maciej Stachowiak wrote: Could multi-page apps be addressed by letting applications specify that other applications should be cached (using a similar api to the one that lets applications programatically cache resources)? I don't think that works very well - you'd have to parse all the HTML, CSS and scripts associated with those other pages just to do the caching. That's a huge cost compared to just downloading the resources. Consider web apps like flickr and upcoming which consist of many many pages. Obviously these specific examples can't cache all of their pages offline but they may well want to cache a significant subset that is interesting to the user. I took this into account and the spec now uses a manifest with no automatic adding to the cache (except for the top-level page itself). I think it's easy to extend Ian's idea in a way that keeps it really simple for the simple case, but that works better for the multi-page case or other complex cases where pages load some resources dynamically. html application=manifest-file The manifest file would indicate all resources used by the web app, including other pages, and other resources that may be loaded by the current page but normally would not be at startup (another problem with Ian's proposal IMO). Multiple pages that refer to the same manifest are considered part of the same web app and share the same cache. If you give an empty value for the application attribute, then the implicit thing that Ian describes happens - the resources that the page actually loads are the ones cached. This is somewhat what the spec says now. Do people think we should rename the application= attribute to manifest= or cache-manifest= or something, by the way? On Fri, 24 Aug 2007, Dimitri Glazkov wrote: Intuitively, I think I agree with Maciej. Manifest is not as elegant as participation by association approach, but it allows for better packaging an application. I am thinking about scripts/stylesheets that are typically a limited set of resources, reused throughout an application. Agreed. I also don't yet understand why Ian wants to store multiple versions of resource representations, if same representation is used by multiple apps. What is the motivation here? If App1 and App2 both use the same library J, and J is then updated to J2 and App2 is updated to use J2, we don't want App1 suddenly to be using J2. It should keep on using J until such time as App1 is updated. ... and how would one make an app like Twitter or Facebook available offline? Perhaps a markup attribute is not a good idea in this case, where every profile page is potentially an application. I am thinking that only _your_ profile page is an offline app. Right? I'm not sure what you would do, exactly. The spec has fallback pages for this kind of thing now though. It'll be interesting to see how well this works. On Thu, 13 Sep 2007, Aaron Boodman wrote: On Sep 6, 2007 5:46 PM, Ian Hickson [EMAIL PROTECTED] wrote: We provide an API that can add files to the cache, and that can be queried to determine if we are in upgrader mode or not, and that can swap in a new cache without reloading the page, during the 'upgrading' event. Given this, and the hidden context that is used while upgrading (not the 'upgrader context', just the one that the page you are viewing is loaded into), isn't it possible to simplify by just doing something like this? addFileToCache(otherTopLevelPage.html); addFileToCache(yetAnotherTopLevelPage.html); addFileToCache(imageThatWasntReferenced.png); Then you don't need the manifest anymore, do you? The manifest is now used in a more critical role than in that proposal. I've been thinking about this, and it seems like an interesting idea, but to me it creates more complexity than it's worth. I've dropped the upgrader idea. On Thu, 20 Sep 2007, Maciej Stachowiak wrote: Is there any need to treat top-level resources differently? If the user directly loads a PNG, JPG or for that matter PDF that's part of an offline manifest, I think it makes sense to serve it from the app cache. The spec now does this. I assume any resource that's not found in the cache can be loaded normally (it would have to be if this is a brand new cache). Actually, I'm not sure from the cache makes sense here given the next sentence. In the current design, once the cache is primed, resources that aren't on the whitelist and aren't in the cache will return errors, and will not be loaded normally. This aids development significantly. Is it really the right thing for XMLHttpRequest to bypass reading from the cache? It makes sense to me that they wouldn't be implicitly stored in the cache, but I don't think the data you get for a URI should depend on whether you used XMLHttpRequest or loaded it in a frame. To be fair, I'm not sure why you'd want to
Re: [whatwg] Offline Web Apps
On Fri, 14 Sep 2007, Aaron Boodman wrote: In order to offline-enable bugzilla, you would first need to turn it into an ajax-style application. Where you separate the UI template from the data. If you still want to keep the old URLs working, this basically means capturing (and periodically updating) every single possible entry point, even though they will all have the exact same content (the ui template). This doesn't seem reasonable to me. You could probably do it with the fallback support now. I think the problem here isn't necessarily just the query parameters though. The problem is more that the application has an open-ended URI space, and we want to capture the whole thing, without actually downloading a near-infinite amount of data per user. Another example would be flickr, where there are bazillions of images, each with their own permalink. Those, though, aren't query parameters. That's true, this was a compromise for Gears. Perhaps what's needed is the concept of aliases. The developer could specify every possible address that could be used to access this application. That's kinda what we have now. I still would think you would want some sort of simple pattern matching so you don't have to list every single bug separately, but it would also work for the flickr case you describe. If by pattern matching you accept prefix matching, then that's what the spec says. :-) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
Ooops On 9/25/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Sep 24, 2007, at 10:45 PM, Robert O'Callahan wrote: So I suspect that, much like synchronous XMLHttpRequest, synchronous file reads will lead to excessive UI lockups in bad circumstances unanticipated by the app author. The file size is accessible to the app author. But OK, lets add an asynchronous API. What's the best way to do that? Add an onread event to File and, say, void read(long offset, long length)? Or something with an explicit callback such as setTimeout has? (Although I'm not sure how we write such things in standard IDL.) Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On 24/09/2007, at 10:45 PM, Robert O'Callahan wrote: snip For small files, synchronous reading is OK. Perhaps there should be a separate whiz-bang asynchronous API ... it could support partial reads too. I would be concerned about this -- for many people async APIs seem scary compared to synchronous ones, and people will end up doing silly things (excessive reads, etc) using a synchronous API, eg. synchronous api for large reads, or many sequential small synchronous reads -- either of which will block the main thread in excitingly bad ways. I would make the argument that anyone making a real webapp (online or offline) is likely to have at least *some* experience with async-xhr, so should have to much difficulty with an async api for reading/writing. Also, I'm not sure how a web app can be expected to know the encoding of a text file on disk. The same way that any other app does --- guess based on the extension and expected usage? --- now that we've all standardized on meta-data-less file systems :-(. I suppose an app could examine the first chunk of the file and then re-read the file with a better guess. The MacOS fs appears to store encoding, but then maybe editors are more careful about including the BOM, etc in files that they save. Surely it would be possible for the browser to transparently store the encoding in the event that none was defined by the developer? Rob --Oliver
Re: [whatwg] Offline Web Apps
On Sep 24, 2007, at 10:45 PM, Robert O'Callahan wrote: On 9/23/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: Obviously, if the way to get the contents as text requires providing the encoding, then it has to be a method. My comment was about the no- argument methods. But you have a point that reading from disk is not a simple get operation. Probably the methods should have names based on read or the like (read(), readAsText(), etc) to indicate this. Also, they should arguably be asynchronous since reading from the disk can be slow, especially for large files, and it is undesirable to block the main thread. For small files, synchronous reading is OK. Perhaps there should be a separate whiz-bang asynchronous API ... it could support partial reads too. What kind of file is small enough is a matter of judgment and depends on device performance characteristics. I tried the following experiment to estimate how much time could be taken by synchronous cold reads of a moderate number of files (assuming multi-file support in input type=file and naiive use of the synchronous read API): $ time cat ~/Pictures/*.jpg /dev/null real0m1.135s user0m0.007s sys 0m0.076s This is on a pretty fast machine with a local filesystem. I have 76 .jpg files totaling about 19M in size. 1.13 seconds seems like an unacceptable length of time to block the UI, and it could easily be much worse for, say, a batch photo upload or an upload of a moderately large video file. So I suspect that, much like synchronous XMLHttpRequest, synchronous file reads will lead to excessive UI lockups in bad circumstances unanticipated by the app author. Also, I'm not sure how a web app can be expected to know the encoding of a text file on disk. The same way that any other app does --- guess based on the extension and expected usage? --- now that we've all standardized on meta-data-less file systems :-(. I suppose an app could examine the first chunk of the file and then re-read the file with a better guess. The OS and the UA can often make a better guess, so I think the option to let the UA decide the encoding should at least be provided. Here are some sources of info that the UA has but the web app doesn't (at least without doing a separate binary read of the file first and possibly significant computation): 1) OS-level metadata, as for example in Mac OS X: $ xattr -l plan.txt com.apple.TextEncoding: UTF-8;134217984 2) Checking for a BOM. 3) Heuristics for specific file types, like looking for meta charset in HTML files or the encoding pseudo-attribute in an XML declaration. 4) General character set autodetection algorithms through statistical methods or similar. 5) Knowledge of the user's locale (useful for some legacy systems where default text encoding is determined by locale). 6) Knowledge of platform encoding conventions. Regards, Maciej
Re: [whatwg] Offline Web Apps
On 9/25/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: The OS and the UA can often make a better guess, so I think the option to let the UA decide the encoding should at least be provided. Here are some sources of info that the UA has but the web app doesn't (at least without doing a separate binary read of the file first and possibly significant computation): Our File implementation lets the browser choose the encoding if an empty encoding is passed by the app, but there might be a nicer/more obvious way to do that. -dave
Re: [whatwg] Offline Web Apps
On Sep 25, 2007, at 8:55 AM, Dave Camp wrote: On 9/25/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: The OS and the UA can often make a better guess, so I think the option to let the UA decide the encoding should at least be provided. Here are some sources of info that the UA has but the web app doesn't (at least without doing a separate binary read of the file first and possibly significant computation): Our File implementation lets the browser choose the encoding if an empty encoding is passed by the app, but there might be a nicer/more obvious way to do that. I think it would be nice to make the argument optional so you can just omit it instead of passing . That or have two different methods, since I would not expect requesting a specific encoding to be the common case. Regards, Maciej
Re: [whatwg] Offline Web Apps
Which other application exactly do you have in mind? The applications I know mostly ask the opener. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robert O'Callahan Sent: Tuesday, September 25, 2007 7:46 AM To: Maciej Stachowiak Cc: Křištof Želechovski; Dave Camp; [EMAIL PROTECTED]; Ian Hickson; Aaron Boodman Subject: Re: [whatwg] Offline Web Apps On 9/23/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: Also, I'm not sure how a web app can be expected to know the encoding of a text file on disk. The same way that any other app does --- guess based on the extension and expected usage? --- now that we've all standardized on meta-data-less file systems :-(. I suppose an app could examine the first chunk of the file and then re-read the file with a better guess. Rob
Re: [whatwg] Offline Web Apps
In that case we go back to structured files. Structured files should be read using specialized parsers, not plain character streams. Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Oliver Hunt Sent: Tuesday, September 25, 2007 8:00 AM To: [EMAIL PROTECTED] Cc: Aaron Boodman; Dave Camp; Maciej Stachowiak; [EMAIL PROTECTED]; Křištof Želechovski; Ian Hickson Subject: Re: [whatwg] Offline Web Apps Surely it would be possible for the browser to transparently store the encoding in the event that none was defined by the developer?
Re: [whatwg] Offline Web Apps
On 9/23/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: Obviously, if the way to get the contents as text requires providing the encoding, then it has to be a method. My comment was about the no- argument methods. But you have a point that reading from disk is not a simple get operation. Probably the methods should have names based on read or the like (read(), readAsText(), etc) to indicate this. Also, they should arguably be asynchronous since reading from the disk can be slow, especially for large files, and it is undesirable to block the main thread. For small files, synchronous reading is OK. Perhaps there should be a separate whiz-bang asynchronous API ... it could support partial reads too. Also, I'm not sure how a web app can be expected to know the encoding of a text file on disk. The same way that any other app does --- guess based on the extension and expected usage? --- now that we've all standardized on meta-data-less file systems :-(. I suppose an app could examine the first chunk of the file and then re-read the file with a better guess. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
Why would you need to preserve the technical details of the encoding, which is entirely platform-dependent? If you nevertheless want to upload the binary representation of text data, just read the file as an octet stream. Burdening the text interface with support for such exotic demands is not a good idea. Cheers, Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Maciej Stachowiak Sent: Sunday, September 23, 2007 3:20 AM To: Křištof Želechovski Cc: [EMAIL PROTECTED]; 'Aaron Boodman'; 'Ian Hickson'; [EMAIL PROTECTED]; 'Dave Camp' Subject: Re: [whatwg] Offline Web Apps On Sep 22, 2007, at 8:48 AM, Křištof Želechovski wrote: An arbitrary file is a sequence of bytes and it is up to the application how these bytes are interpreted as characters. Moreover, a text file conceptually does not contain a string; it contains, by convention, lines of text. The result of reading a file as text should be a sequence of lines, not a string, with the line breaking characters removed. Disagree. If the goal is to later upload the text file, then you want to preserve line endings and other special characters as-is. Regards, Maciej
Re: [whatwg] Offline Web Apps
I do not share your reservations. The file contents does not constitute its property and, unlike properties, much work is actually needed to extract it. Therefore the name chosen seems very appropriate. How is the character encoding determined when the file is read as text? An arbitrary file is a sequence of bytes and it is up to the application how these bytes are interpreted as characters. Moreover, a text file conceptually does not contain a string; it contains, by convention, lines of text. The result of reading a file as text should be a sequence of lines, not a string, with the line breaking characters removed. A text file can a printout; the difference is that a printout consists of pages that consist of lines. Printouts are rare these times, but they do appear from time to time. They can be distinguished from ordinary text files by the presence of the form characters form feed and vertical tab. The form feed is used to separate pages, the vertical tab is for presentation only and should be converted to an empty line or two. Cheers Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Maciej Stachowiak Sent: Saturday, September 22, 2007 1:07 AM To: Dave Camp Cc: [EMAIL PROTECTED]; Ian Hickson; [EMAIL PROTECTED]; Aaron Boodman Subject: Re: [whatwg] Offline Web Apps On Sep 21, 2007, at 10:49 AM, Dave Camp wrote: On 9/21/07, Robert O'Callahan [EMAIL PROTECTED] wrote: Actually we have an experimental API for this now. See here: http://mxr.mozilla.org/seamonkey/source/dom/public/idl/html/nsIDOMNSHTMLInpu tElement.idl#55 http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFileList.i dl http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFile.idl The core is: readonly attribute DOMString fileName; readonly attribute unsigned long long fileSize; It would be nice if this was designed to handle the possibility of multiple file selection (which I think Web Forms 2 enables). DOMString getAsText(in DOMString encoding); // raises(FileException) on retrieval DOMString getAsDataURL(); // raises(FileException) on retrieval DOMString getAsBinary(); // raises(FileException) on retrieval These should be self-explanatory. Minor nitpicks: For the ones that don't take a parameter, I think a read-only attribute would be more appropriate than a get function. It's relatively rare for JS APIs to use no-arg getter instead of an attribute, and even when it does it's rare for the function name to start with get.
Re: [whatwg] Offline Web Apps
On Sep 22, 2007, at 8:48 AM, Křištof Želechovski wrote: I do not share your reservations. The file contents does not constitute its property and, unlike properties, much work is actually needed to extract it. Therefore the name chosen seems very appropriate. How is the character encoding determined when the file is read as text? Obviously, if the way to get the contents as text requires providing the encoding, then it has to be a method. My comment was about the no- argument methods. But you have a point that reading from disk is not a simple get operation. Probably the methods should have names based on read or the like (read(), readAsText(), etc) to indicate this. Also, they should arguably be asynchronous since reading from the disk can be slow, especially for large files, and it is undesirable to block the main thread. Also, I'm not sure how a web app can be expected to know the encoding of a text file on disk. An arbitrary file is a sequence of bytes and it is up to the application how these bytes are interpreted as characters. Moreover, a text file conceptually does not contain a string; it contains, by convention, lines of text. The result of reading a file as text should be a sequence of lines, not a string, with the line breaking characters removed. Disagree. If the goal is to later upload the text file, then you want to preserve line endings and other special characters as-is. Regards, Maciej
Re: [whatwg] Offline Web Apps with Open-Ended URI Spaces (was Re: Offline Web Apps)
On 9/20/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: 2) Many offline web apps will let you want to make changes, including not just changing existing items, but also creating new items. To do this, at minimum there needs to be an API to inject a new resource into the offline cache programatically, with the data explicitly provided. This violates the principle that the offline cache is a cache. It creates the issue of how to reconcile server pages with client-stored pages. 6) It's potentially costly to download data mulltiple times, so if you pull the remote data into a local database, you won't also want to pull every page reflecting that data, instead you will want to generate templates client-side and insert them into the offline cache. However, it seems like a relatively small step from there to having a single fallback page to be used for all URIs that are part of the app but haven't gotten downloaded in the course of normal use. And this would be a huge optimization, since it would save the client the need to manually generate each page for a resource of interest that has not yet been visited. Given point #1, I think this should be based on textual prefix matching of the URI, not just dropping the query (the scheme and authority sections should be treated specially, of course, it should not be allowed to have a fallback page in someone else's security domain). This will allow matching paths and also matching only specific kinds of queries (where the first parameter is set to some specific value, say). However, to make offline and online mode diverge as little as possible, I think perhaps such fallback pages should apply only when offline. When online, the UA should go to the real page. With the prefix-based fallback page solution, I'm not sure it will be necessary to also support individual client-generated pages. Thoughts? It sounds reasonable to me, a lot better than option 2). Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
I haven't had time to study Ian's proposal properly yet, sorry. But some easy comments: On 9/20/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: Upgrader: Create a hidden browsing context. Load the upgrader in it. I don't like this whole upgrader idea. Parsing HTML and CSS and executing JavaScript seems like an inefficient way to do an app update. I think it is reasonable to require a manifest file for multipage apps, and writing an HTML/CSS/JS upgrader that can cover all pages of a multipage app does not seem significantly easier than creating a manifest file. The implicit manifest idea seems handy as a quick way to handle one-page apps but it does not seem reasonable for the multipage case, and this would obviate the need for an upgrader. I totally agree with this. Just before onload, fire an 'upgrading' event to every instance of a top-level page using a cache with the same identifier. Whether or not there are upgraders though, I think events should dispatch when a manifest-based upgrade either completes or fails (and perhaps also when the upgrade starts). Agree... The event has a handle to the Window object of the hidden browsing context. After every 'upgrading' event has been fired, the 'load' event must be fired on the upgrader. After that happens, if any of the aforementioned instances are still using old versions of the cache, then the user agent may inform user they can reload to update. I think it would be preferable to let the apps upgrade themselves instead. They could choose to use location.reload() if they are not holding any interesting state, or they could offer to save the user's state before doing this, or they could make some alternate call that requests all new resource loads for this instance should come from the freshly upgraded cache, which would let it perform an upgrade manually preserving current state if feasible. Agree. The Upgrader can do such things as updating the database schema between versions, and when there are multiple instances running, it allows them to negotiate who will do that work instead of it happening several times. I would suggest instead that the instance that triggered the upgrade be given a special event so that it can do this and could optionally present in-page UI while doing so. This seems simpler than adding a hidden browsing context. Changing the schema may require pausing other instances, however, if there is no way to lock the database. Agree. Modal alerts (window.alert, .prompt, etc) in the background page can either raise an exception, be ignored, drop a message to a console, or possibly display a message over the top of the foreground app's browsing context. To avoid such complexities it would be better to avoid the idea of a hidden upgrader. And in-page UI could be more tasteful than prompts or alerts. Agree. I'm not sure if an API to introspect what is currently in the cache is needed. I can't think of a use case off hand. But both the Google Gears LocalServer API and the Mozilla offline API have this. Two different use cases: -- If you can programmatically force URIs into the offline cache, then you want to be able to enumerate the resources in the offline cache, otherwise there is no way to reliably remove unneeded resources (especially if there was an older, buggy version of the app that may have loaded resources from unexpected URIs). -- Several Web app authors have asked for the ability to test whether a resource is cached, for their online apps. For example, when you're zooming in and out of a map, the application could choose which tile(s) to use for the animation by scaling them up or down. This would also be convenient for offline use, where a resource might not necessarily be in the offline cache but you could use it if it happened to be available. A la the Google Gears API, I also think a feature is needed to do something useful with input type=file when offline, to save a resource for later upload to the server. Preferably this should not require round-tripping the data through an ECMAScript string or number array, or it will be too inefficient to work for large files. Actually we have an experimental API for this now. See here: http://mxr.mozilla.org/seamonkey/source/dom/public/idl/html/nsIDOMNSHTMLInputElement.idl#55 http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFileList.idl http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFile.idl The core is: readonly attribute DOMString fileName; readonly attribute unsigned long long fileSize; DOMString getAsText(in DOMString encoding); // raises(FileException) on retrieval DOMString getAsDataURL(); // raises(FileException) on retrieval DOMString getAsBinary(); // raises(FileException) on retrieval These should be self-explanatory. I guess this
Re: [whatwg] Offline Web Apps
On 9/21/07, Robert O'Callahan [EMAIL PROTECTED] wrote: Actually we have an experimental API for this now. See here: http://mxr.mozilla.org/seamonkey/source/dom/public/idl/html/nsIDOMNSHTMLInputElement.idl#55 http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFileList.idl http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFile.idl The core is: readonly attribute DOMString fileName; readonly attribute unsigned long long fileSize; DOMString getAsText(in DOMString encoding); // raises(FileException) on retrieval DOMString getAsDataURL(); // raises(FileException) on retrieval DOMString getAsBinary(); // raises(FileException) on retrieval These should be self-explanatory. I guess this isn't great for really huge files, but multi-megabyte files should be OK on most machines, and it avoids having to deal with a client-writable cache. It obviously has some interesting uses for online apps as well. It'd be possible to extend this to avoid bringing these files into memory. We'd just need globalStorage (or something like it) to accept/return nsIDOMFile objects, and a way for XHR to send them. -dave
Re: [whatwg] Offline Web Apps
On Sep 21, 2007, at 10:49 AM, Dave Camp wrote: On 9/21/07, Robert O'Callahan [EMAIL PROTECTED] wrote: Actually we have an experimental API for this now. See here: http://mxr.mozilla.org/seamonkey/source/dom/public/idl/html/nsIDOMNSHTMLInputElement.idl#55 http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFileList.idl http://mxr.mozilla.org/seamonkey/source/content/base/public/nsIDOMFile.idl The core is: readonly attribute DOMString fileName; readonly attribute unsigned long long fileSize; It would be nice if this was designed to handle the possibility of multiple file selection (which I think Web Forms 2 enables). DOMString getAsText(in DOMString encoding); // raises(FileException) on retrieval DOMString getAsDataURL(); // raises(FileException) on retrieval DOMString getAsBinary(); // raises(FileException) on retrieval These should be self-explanatory. Minor nitpicks: For the ones that don't take a parameter, I think a read-only attribute would be more appropriate than a get function. It's relatively rare for JS APIs to use no-arg getter instead of an attribute, and even when it does it's rare for the function name to start with get. DOMString getAsBinary() isn't actually self-explanatory to me. How do you encode binary data as a UTF-16 string? I can think of at least two vaguely obvious ways (each code point is a byte, or each code point is a 16-bit chunk of the data). Both seem awkward to work with. I think it would be more effective to use a dedicated type for binary data. This is already likely to happen for XHR 2 binary data access, with something based on the ES4-proposed ByteArray class. What do you guys think of that? I guess this isn't great for really huge files, but multi-megabyte files should be OK on most machines, and it avoids having to deal with a client-writable cache. It obviously has some interesting uses for online apps as well. It'd be possible to extend this to avoid bringing these files into memory. We'd just need globalStorage (or something like it) to accept/return nsIDOMFile objects, and a way for XHR to send them. Sounds reasonable. I'd love to see a rough cut at a spec for this. Regards, Maciej
Re: [whatwg] Offline Web Apps
On Sat, 22 Sep 2007 01:07:23 +0200, Maciej Stachowiak [EMAIL PROTECTED] wrote: It would be nice if this was designed to handle the possibility of multiple file selection (which I think Web Forms 2 enables). It does actually. There's a fileList attribute on HTMLInputElement that returns a FileList. It'd be possible to extend this to avoid bringing these files into memory. We'd just need globalStorage (or something like it) to accept/return nsIDOMFile objects, and a way for XHR to send them. Sounds reasonable. I'd love to see a rough cut at a spec for this. XMLHttpRequest level 2 supports sending ByteArray. So you could do something like the following maybe: xhr.send(file.bytes) Although if HTML5 gains a native File object I suppose support for that could be added as well if there's any benefit. -- Anne van Kesteren http://annevankesteren.nl/ http://www.opera.com/
Re: [whatwg] Offline Web Apps
On 9/21/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: DOMString getAsBinary() isn't actually self-explanatory to me. How do you encode binary data as a UTF-16 string? I can think of at least two vaguely obvious ways (each code point is a byte, or each code point is a 16-bit chunk of the data). Both seem awkward to work with. I think it would be more effective to use a dedicated type for binary data. This is already likely to happen for XHR 2 binary data access, with something based on the ES4-proposed ByteArray class. What do you guys think of that? We implemented this as a byte per code point. A ByteArray version would make sense. -dave
Re: [whatwg] Offline Web Apps
My commentary below. Overall, I think the basic model is fairly sound. But I do think some improvements could be made. On Sep 6, 2007, at 5:46 PM, Ian Hickson wrote: Ok, new proposal: There's a concept of an application cache. An application cache is a group of resources, the group being identified by a URI (which typically happens to resolve to a manifest). Resources in a cache are either top-level or not; top-level resources are those that are HTML or XML and when parsed with scripting disabled have html application=... with the value of the attribute pointing to the same URI as identifies the cache. When you visit a page you first check to see if you have that page in a cache as a known top-level page. Is there any need to treat top-level resources differently? If the user directly loads a PNG, JPG or for that matter PDF that's part of an offline manifest, I think it makes sense to serve it from the app cache. It seems like it would simplify the model a bit a bit for the offline cache to treat all items in the manifest as part of the application when visited directly. The only problem here is the potential inconsistency if an HTML or XML document doesn't have the html application=... declaration at the top, but is still cited in some other app's manifest. Then it would be treated as part of the application if an app page citing that manifest was visited first, but not if it wasn't. I think this is ok though and may even be a desirable behavior. For instance, you might not want a single flickr photo page to be an app by itself, but you'd still want it to be treated as part of the app domain for someone who had visited the main application page. If you do, skip the next two paragraphs; the 'new cache' flag is set to false. If you don't, you fetch the URL. If it has no application= attribute, then do whatever the normal thing to do is. Ignore the rest of this. The presence of the attribute indicates that it's expecting an application cache to apply. The presence is detected at parse time, and must be present on the first html start tag before any other start tags. Check that the attribute's value is same-origin safe. If it isn't, pretend the attribute wasn't there (and ignore the rest of this). Otherwise, check to see if you already have a cache for the given URI. If you don't, create a new cache identified by the given URI. In any case, save this resource to the identified cache, as a known top-level page for that cache. Then, act as if you had known about the cache when you started (next step), except with the 'new cache' flag set to true. Load the page from the cache and display it. I assume any resource that's not found in the cache can be loaded normally (it would have to be if this is a brand new cache). Actually, I'm not sure from the cache makes sense here given the next sentence. Any resources that the page tries to fetch using GETs that aren't XMLHttpRequest'ed must be taken from the cache, if available. Is it really the right thing for XMLHttpRequest to bypass reading from the cache? It makes sense to me that they wouldn't be implicitly stored in the cache, but I don't think the data you get for a URI should depend on whether you used XMLHttpRequest or loaded it in a frame. To be fair, I'm not sure why you'd want to do an XHR for a resource that then gets served from the offline cache. But I'm also not sure why you'd list an item in your manifest that you then wanted to load with XHR. So it seems simpler to omit this slight complication. When they aren't, the resources must be fetched then stored in the cache. If there is an explicit manifest, it seems wrong to store things in the cache that aren't in the manifest but are part of this page. That means you get the union of the manifest and things the page loads, which will make offline behavior hard to debug I think. It would be better to fetch the manifest (possibly getting it from the existing application cache, if any) before proceeding. Then you'd know which of the resources loaded as part of this page belong in the cache up front. That would affect the following steps. Once the UA is ready to do so the UA must go on to the next steps. UAs may do this immediately, or may wait for the original page load to complete, or may delay it up to a UA-defined minimum delay. If 'new cache' is true, and the cache identifier URI is the same as the URI that was just downloaded and put in the cache: Do nothing. If 'new cache' is true, and the cache identifier URI is different from the URI that was just downloaded: Fetch the resource identified by that URI. Store it in the cache. If it's a manifest and it parses correctly, download all the URIs given in that manifest and add them to the cache. If any are HTML files which, when parsed with scripting disabled, trigger the application= handling and have a
Re: [whatwg] Offline Web Apps with Open-Ended URI Spaces (was Re: Offline Web Apps)
On Sep 19, 2007 11:23 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: Given point #1, I think this should be based on textual prefix matching of the URI, not just dropping the query (the scheme and authority sections should be treated specially, of course, it should not be allowed to have a fallback page in someone else's security domain). This will allow matching paths and also matching only specific kinds of queries (where the first parameter is set to some specific value, say). However, to make offline and online mode diverge as little as possible, I think perhaps such fallback pages should apply only when offline. When online, the UA should go to the real page. With the prefix-based fallback page solution, I'm not sure it will be necessary to also support individual client-generated pages. I think having a feature where an offline page can be served for all URIs with a certain prefix makes sense. It is basically a superset of the current Gears ignoreQuery feature, and it is something that users of our API have asked about. - a
Re: [whatwg] Offline Web Apps
On Fri, 14 Sep 2007, Mark Baker wrote: +1 Just for the record, there's no need to actually post to this list just to say that you agree. I take into account all comments and base the edits to the spec only on the merit of the arguments, not their popularity. You can see which e-mails I've taken into account (which should include all of the e-mails sent to the WHATWG list that contain substantive evidence or arguments, as well as mail from other sources) here: http://www.whatwg.org/issues/ Cheers, -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On 9/13/07, Ian Hickson [EMAIL PROTECTED] wrote: By doing this we're basically saying that the query string never gets sent to the server anymore. That seems like a huge violation of the URI semantics. I think the problem here isn't necessarily just the query parameters though. The problem is more that the application has an open-ended URI space, and we want to capture the whole thing, without actually downloading a near-infinite amount of data per user. +1 Mark. -- Mark Baker. Ottawa, Ontario, CANADA. http://www.markbaker.ca Coactus; Web-inspired integration strategies http://www.coactus.com
Re: [whatwg] Offline Web Apps
On Sep 13, 2007 3:59 PM, Ian Hickson [EMAIL PROTECTED] wrote: We're talking about Bugzilla here. There's a LOT of data to send per bug. All the metadata, all the comments, the entire changelog, it adds up to probably not much less than the actual page as generated by the server. In order to offline-enable bugzilla, you would first need to turn it into an ajax-style application. Where you separate the UI template from the data. If you still want to keep the old URLs working, this basically means capturing (and periodically updating) every single possible entry point, even though they will all have the exact same content (the ui template). This doesn't seem reasonable to me. I think the problem here isn't necessarily just the query parameters though. The problem is more that the application has an open-ended URI space, and we want to capture the whole thing, without actually downloading a near-infinite amount of data per user. Another example would be flickr, where there are bazillions of images, each with their own permalink. Those, though, aren't query parameters. That's true, this was a compromise for Gears. Perhaps what's needed is the concept of aliases. The developer could specify every possible address that could be used to access this application. This would be more flexible than the current Gears solution, and I could see it being used for other things as well. I still would think you would want some sort of simple pattern matching so you don't have to list every single bug separately, but it would also work for the flickr case you describe. - a
Re: [whatwg] Offline Web Apps
On 9/11/07, Dimitri Glazkov [EMAIL PROTECTED] wrote: Since, AFAIK, the fragment identifier is not passed onto the server by the UA, I can't see how an application could be designed with proper noscript degradation and reliance frament ids for query communication. Besides, using query parameters is much more natural for HTML: forms with method=get are the way to build it. Those are good points that I should have thought of :-). Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Thu, 13 Sep 2007 11:22:59 +0200, Robert O'Callahan [EMAIL PROTECTED] wrote: On 9/11/07, Dimitri Glazkov [EMAIL PROTECTED] wrote: Since, AFAIK, the fragment identifier is not passed onto the server by the UA, I can't see how an application could be designed with proper noscript degradation and reliance frament ids for query communication. Besides, using query parameters is much more natural for HTML: forms with method=get are the way to build it. Those are good points that I should have thought of :-). I'm not sure I understand the query parameter use case. If you have a web page foo.cgi?page=x wouldn't that page also be simply the offline page? What exactly is the scenario in some more detail? (Note that with history.pushState() you can set the URI of the current page so the fragment identifier bookmarking argument is no longer very relevant.) -- Anne van Kesteren http://annevankesteren.nl/ http://www.opera.com/
Re: [whatwg] Offline Web Apps
On Sep 13, 2007 4:44 AM, Anne van Kesteren [EMAIL PROTECTED] wrote: I'm not sure I understand the query parameter use case. If you have a web page foo.cgi?page=x wouldn't that page also be simply the offline page? What exactly is the scenario in some more detail? I feel like me and the other querystringers are missing some critical detail that would make omitting querystring support work. So here is how I see it. Please tell me what is missing. The bugzilla scenario is a good one. Someone wants to offline-enable bugzilla. They could rewrite bugzilla to use fragment identifiers instead of querystrings, but then bug shortcuts on the web would not work with the offline-enabled application. They couldn't really cache all possible pages (there are lots of bugs, and that would be really inefficient). I suppose you could have each bug page be a separate application, and cache each one as it is viewed online, but this is really wasteful, and more importantly, bug shortcuts won't work offline unless you have previously visited them. - a
Re: [whatwg] Offline Web Apps
On Sep 13, 2007, at 8:30 AM, Aaron Boodman wrote: They could rewrite bugzilla to use fragment identifiers instead of querystrings, but then bug shortcuts on the web would not work with the offline-enabled application. If you're designing a new application, even one that works both online and offline, it's not so helpful. But if you're trying to transition an existing application, with lots of existing URLs, then you might wish for some way to handle these URLs with query strings. I could imagine a similar issue if YouTube was creating the offline mode for YouTube. However, it may be asking too much of the offline design to accommodate this. The query string is part of the URL, a part normally parsed only by the server, and it's hard to see how to fit this in. -- Darin
Re: [whatwg] Offline Web Apps
On 9/13/07, Aaron Boodman [EMAIL PROTECTED] wrote: The bugzilla scenario is a good one. Someone wants to offline-enable bugzilla. They could rewrite bugzilla to use fragment identifiers instead of querystrings, but then bug shortcuts on the web would not work with the offline-enabled application. They couldn't really cache all possible pages (there are lots of bugs, and that would be really inefficient). I suppose you could have each bug page be a separate application, and cache each one as it is viewed online, but this is really wasteful, and more importantly, bug shortcuts won't work offline unless you have previously visited them. - a This kind of puts us at crossroads as to whether keep treating a URL as an opaque identifier or attempt to break it down to determine whether a page belongs to a given set. Another, less cool path would be to use regular expressions or somesuch instead of explicit list. What if an application could be given an event when the link, clicked on a document that is part of the application leads to a page that is not present in cache? This way, the app could potentially manage the fallback. :DG
Re: [whatwg] Offline Web Apps
The following has a rant flavor to it, but I am hoping you'll find it helpful in the thought process. Distinct, server-reaching URLs (no fragment identifiers) for each page in an web application are a _good_thing_. Packing the whole application into one document and managing history with id hashes and other hacks is not. It's a necessary kludge that you have to do in order to avoid browser context re-initializing, re-parsing scripts, and re-requesting all accompanying graphical and stylistic overhead every time the user clicks on anything. I would've loved it if Google Reader had a distinct URL for each click I make on the page, and I am sure Google Reader devs would've loved it too. Except they also would've loved not having to worry about the browser/scripting context change. Instead, they have to essentially reinvent the way web works (http://www.tbray.org/ongoing/When/200x/2006/03/26/On-REST) by overloading fragment identifier with an entire URI management system. I applaud the effort and the result is awesome, but it doesn't make a good bedtime story. I guess the vision is that application context transcends and encompasses browser/scripting context somehow. :DG
Re: [whatwg] Offline Web Apps
On Thu, 13 Sep 2007 18:21:21 +0200, Dimitri Glazkov [EMAIL PROTECTED] wrote: I would've loved it if Google Reader had a distinct URL for each click I make on the page, and I am sure Google Reader devs would've loved it too. Except they also would've loved not having to worry about the browser/scripting context change. You seem to have missed what I pointed out earlier: http://www.whatwg.org/specs/web-apps/current-work/#pushstate This allows applications to make distinct URIs while keeping all the other benefits. -- Anne van Kesteren http://annevankesteren.nl/ http://www.opera.com/
Re: [whatwg] Offline Web Apps
On Sep 6, 2007 5:46 PM, Ian Hickson [EMAIL PROTECTED] wrote:
We provide an API that can add files to the cache, and that can be queried
to determine if we are in upgrader mode or not, and that can swap in a
new cache without reloading the page, during the 'upgrading' event.
Given this, and the hidden context that is used while upgrading (not
the 'upgrader context', just the one that the page you are viewing is
loaded into), isn't it possible to simplify by just doing something
like this?
addFileToCache(otherTopLevelPage.html);
addFileToCache(yetAnotherTopLevelPage.html);
addFileToCache(imageThatWasntReferenced.png);
Then you don't need the manifest anymore, do you?
Upgrader:
Create a hidden browsing context.
Load the upgrader in it.
Just before onload, fire an 'upgrading' event to every instance of a
top-level page using a cache with the same identifier.
The event has a handle to the Window object of the hidden browsing
context.
After every 'upgrading' event has been fired, the 'load' event must be
fired on the upgrader.
After that happens, if any of the aforementioned instances are still
using old versions of the cache, then the user agent may inform user
they can reload to update.
The Upgrader can do such things as updating the database schema between
versions, and when there are multiple instances running, it allows them to
negotiate who will do that work instead of it happening several times.
I've been thinking about this, and it seems like an interesting idea,
but to me it creates more complexity than it's worth. Here are some of
the questions around it:
* Is an update marked as successful before or after an update runs?
If before, the new version of the app must be prepared for the
scenario that the updater did not run completely, in which case you
may as well put the upgrade logic in the new version and be done with
it.
If after, how do you track success/failure of the update? Keep in
mind that an update may require asynchronous server access to retrieve
data. If you allow for this, you just add lots of API for a relatively
small case. If you don't allow for it, then in some cases upgrading
will have to be done by the new version of the app.
* I like the idea of app instances being able to talk to each other
during upgrade, but it seems to conflate several goals:
Prevent two apps from trying to upgrade at the same time
Prevent instances of an app that are old from accessing a newer local schema
Allow multiple instances of an app to talk to each other
Separate application logic from UI logic and share the application
logic between multiple views (Dimitri and Alex Russell's point)
For the first two problems in gears, we have been kicking around the
idea of versioning the database connections. So you could do:
db.open(mydb, 1);
db.execute(...);
db.execute(...);
..etc..
But as soon as somebody in another window did:
var db.open(mydb, 1);
db.setVersion(2);
The previous connection starts throwing errors. This assumes that the
database has transaction support and a few other things though. From
my point of view this is simpler, but I understand that it relies on
some bits that are not in whatwg's current database proposal.
For the long term, I agree with Dimitri that what is really needed is
a shared JS context between all instances of an application.
Otherwise, anybody that is doing significant synchronization, for
example, will have to somehow organize through all the instances of an
app which page is doing the work. It's silly. We need the ability to
separate an app into the ui bits and the application bit, and have a
one-to-many relationship between them.
If a future version of the spec had something like a gears worker, one could do:
// If the named worker already exists (namespaced to the current
origin I suppose), then a
// reference to it will be retrieved. Otherwise, one will be created.
var worker = getOrCreateWorker(app-logic, app.js);
worker.sendMessage(load_customers, {account_id: 42}, function(result) {
result.customers.forEach(function(customer) {
console.log(Got customer name: %s, id: %d, customer.name, customer.id);
});
});
- a
Re: [whatwg] Offline Web Apps
On Thu, 13 Sep 2007, Aaron Boodman wrote: The bugzilla scenario is a good one. Someone wants to offline-enable bugzilla. They could rewrite bugzilla to use fragment identifiers instead of querystrings, but then bug shortcuts on the web would not work with the offline-enabled application. They couldn't really cache all possible pages (there are lots of bugs, and that would be really inefficient). I suppose you could have each bug page be a separate application, and cache each one as it is viewed online, but this is really wasteful, and more importantly, bug shortcuts won't work offline unless you have previously visited them. Ok, but what are you proposing to _solve_ this? There's no difference between the following two models as I see it: * Download an HTML page for each bug * Download a single page to generate the bug pages plus one data page per bug ...except that the former will mean there's no difference between online and offline, and the latter will mean there _is_ a difference between online and offline, which will bring in its associated sets of bugs. I don't understand how you would expect the client-side server idea (the parsing of server-side URIs on the client) to work. It would be helpful to see actual sample code, maybe. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Sep 13, 2007 1:59 PM, Ian Hickson [EMAIL PROTECTED] wrote:
Ok, but what are you proposing to _solve_ this? There's no difference
between the following two models as I see it:
* Download an HTML page for each bug
* Download a single page to generate the bug pages plus one data
page per bug
By 'data page' you mean a row of data representing that page? I
presume that downloading a single row of data will be more efficient
than downloading an entire web page plus rechecking all its resources.
It also has the advantage of not making update take as long as
refetching a resource from the server for every possible bug.
...except that the former will mean there's no difference between online
and offline, and the latter will mean there _is_ a difference between
online and offline, which will bring in its associated sets of bugs.
I don't understand how you would expect the client-side server idea (the
parsing of server-side URIs on the client) to work. It would be helpful to
see actual sample code, maybe.
Most AJAX libraries already have code to parse querystrings. It comes
up a lot and it isn't that hard:
http://trac.mochikit.com/browser/mochikit/trunk/MochiKit/Base.js#L1140
Basically, what I'm saying is that an application like Bugzilla has
thousands of possible entry points. These URLs are spread all over the
web and they should continue to work with the offline-enabled version
of Bugzilla. I don't see any way to make this possible efficiently
other than having something like the ability to ignore querystrings.
Something to do more flexible matching would be better, but we thought
this was a good happy medium for gears.
Maybe you were asking how you could keep querystring-based urls in the
offline version of Bugzilla? One option would be to go ahead and keep
using them. In the offline version you could do:
function gotoBug(bugId) {
location.href = ?b= + bugId;
}
This would cause a refresh, but since the page is captured and ignores
querystrings, it is fast.
In your initialization you have something like:
window.onload = function() {
var bugData = db.execute(select * from bugs where id = ?,
parseQueryString()[b]);
}
- a
Re: [whatwg] Offline Web Apps
On Thu, 13 Sep 2007, Aaron Boodman wrote: Ok, but what are you proposing to _solve_ this? There's no difference between the following two models as I see it: * Download an HTML page for each bug * Download a single page to generate the bug pages plus one data page per bug By 'data page' you mean a row of data representing that page? I presume that downloading a single row of data will be more efficient than downloading an entire web page plus rechecking all its resources. We're talking about Bugzilla here. There's a LOT of data to send per bug. All the metadata, all the comments, the entire changelog, it adds up to probably not much less than the actual page as generated by the server. Maybe you were asking how you could keep querystring-based urls in the offline version of Bugzilla? There isn't an offline version. There's just one version, it just happens to support being online and offline. By doing this we're basically saying that the query string never gets sent to the server anymore. That seems like a huge violation of the URI semantics. I think the problem here isn't necessarily just the query parameters though. The problem is more that the application has an open-ended URI space, and we want to capture the whole thing, without actually downloading a near-infinite amount of data per user. Another example would be flickr, where there are bazillions of images, each with their own permalink. Those, though, aren't query parameters. The core problem though is that having an offline mode URI remapping script, as it were, only makes sense if there are separate online and offline apps. The model we're looking at here doesn't have separate online and offline apps. There's just one app, and it works as if it was offline all the time, with two differences: 1. It can atomically update all the infrastructure (markup, scripts, styles, database schema). 2. It can communicate with the server using an out-of-band communication channel while the client is online. To me this means that we either want apps to have a finite URI space, or we want apps to have a way of passing information to pages via URIs that doesn't screw with the existing URI infrastructure. Or both. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On 9/10/07, Ian Hickson [EMAIL PROTECTED] wrote: On Mon, 10 Sep 2007, Robert O'Callahan wrote: That's an option, but then you can't use the fragment identifier for its scroll-to behaviour when you use it to pass parameters. You can just pass the scroll position as one of the parameters. Hmm... Here's a real-life example: https://bugzilla.mozilla.org/show_bug.cgi?id=389437#c3 Now if we refit Bugzilla for offline support, this will become something like https://bugzilla.mozilla.org/show_bug.cgi#id=389437,c3 If we want the same URIs used both offline and online (which I strongly believe is required), then for things to work when script is disabled, the query parameters have to be stuffed into the name attributes. I guess that's OK although it's a bit annoying. Also, when retrofitting an existing app, this requires changing the sources of links as well as the destination pages. That seems like a minor issue, compared to the scale of the effort. I'd be interested in studying an actual example of what this would entail. Do people feel strongly about wanting this dichtomy between the online version and the offline version? Fundamentally, I feel quite strongly that we'd want them to use the exact same code all the time. I'm just saying these are extra things that would have to be changed when making an existing app offline-capable. This isn't a big deal to me, authors can use the fragment identifier, it's just ugly. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Mon, 10 Sep 2007, Robert O'Callahan wrote: Here's a real-life example: https://bugzilla.mozilla.org/show_bug.cgi?id=389437#c3 Now if we refit Bugzilla for offline support, this will become something like https://bugzilla.mozilla.org/show_bug.cgi#id=389437,c3 If we want the same URIs used both offline and online (which I strongly believe is required), then for things to work when script is disabled, the query parameters have to be stuffed into the name attributes. I guess that's OK although it's a bit annoying. Why wouldn't you just offline-cache the https://bugzilla.mozilla.org/show_bug.cgi?id=389437 ...file? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On 9/10/07, Ian Hickson [EMAIL PROTECTED] wrote: Why wouldn't you just offline-cache the https://bugzilla.mozilla.org/show_bug.cgi?id=389437 ...file? I might be storing its data in a local database so I can make changes to it locally while I'm offline. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Mon, 10 Sep 2007, Robert O'Callahan wrote: On 9/10/07, Ian Hickson [EMAIL PROTECTED] wrote: Why wouldn't you just offline-cache the https://bugzilla.mozilla.org/show_bug.cgi?id=389437 ...file? I might be storing its data in a local database so I can make changes to it locally while I'm offline. I still don't understand how you see this working using the same codebase both online and offline. The model I'm proposing basically relies on the app being an offline app, except that while you're online the offline app is talking to the server to keep its database updated and the server updated with the user's changes. What you're describing seems like it would require a different set of code for the offline case than the online case. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Sep 10, 2007 2:21 PM, Ian Hickson [EMAIL PROTECTED] wrote: I still don't understand how you see this working using the same codebase both online and offline. The model I'm proposing basically relies on the app being an offline app, except that while you're online the offline app is talking to the server to keep its database updated and the server updated with the user's changes. What you're describing seems like it would require a different set of code for the offline case than the online case. You can share the UI code for normal web apps with local-mode ones if you have a clean separation between the server and the UI. In fact, I think it will be common to want to do this. Bookmarks and URLs from the online app should work with the offline one and vice versa. - a
Re: [whatwg] Offline Web Apps
On Mon, 10 Sep 2007, Aaron Boodman wrote: On Sep 10, 2007 2:21 PM, Ian Hickson [EMAIL PROTECTED] wrote: I still don't understand how you see this working using the same codebase both online and offline. The model I'm proposing basically relies on the app being an offline app, except that while you're online the offline app is talking to the server to keep its database updated and the server updated with the user's changes. What you're describing seems like it would require a different set of code for the offline case than the online case. You can share the UI code for normal web apps with local-mode ones if you have a clean separation between the server and the UI. In fact, I think it will be common to want to do this. Bookmarks and URLs from the online app should work with the offline one and vice versa. I agree, I just don't see how this applies to the example Robert gave with Bugzilla. The idea of having some URIs be decomposed by the client-side and have them fetch different pages from the server seems really unwise, especially since in browsers that don't support this, they'll not be decomposed and will end up fetching different pages. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
Since, AFAIK, the fragment identifier is not passed onto the server by the UA, I can't see how an application could be designed with proper noscript degradation and reliance frament ids for query communication. Besides, using query parameters is much more natural for HTML: forms with method=get are the way to build it. On 9/10/07, Ian Hickson [EMAIL PROTECTED] wrote: On Mon, 10 Sep 2007, Aaron Boodman wrote: On Sep 10, 2007 2:21 PM, Ian Hickson [EMAIL PROTECTED] wrote: I still don't understand how you see this working using the same codebase both online and offline. The model I'm proposing basically relies on the app being an offline app, except that while you're online the offline app is talking to the server to keep its database updated and the server updated with the user's changes. What you're describing seems like it would require a different set of code for the offline case than the online case. You can share the UI code for normal web apps with local-mode ones if you have a clean separation between the server and the UI. In fact, I think it will be common to want to do this. Bookmarks and URLs from the online app should work with the offline one and vice versa. I agree, I just don't see how this applies to the example Robert gave with Bugzilla. The idea of having some URIs be decomposed by the client-side and have them fetch different pages from the server seems really unwise, especially since in browsers that don't support this, they'll not be decomposed and will end up fetching different pages. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
In fact, interrogating such a form should provide enough information to the UA on what query parameters are and even some of the values of these parameters. On 9/10/07, Dimitri Glazkov [EMAIL PROTECTED] wrote: Since, AFAIK, the fragment identifier is not passed onto the server by the UA, I can't see how an application could be designed with proper noscript degradation and reliance frament ids for query communication. Besides, using query parameters is much more natural for HTML: forms with method=get are the way to build it. On 9/10/07, Ian Hickson [EMAIL PROTECTED] wrote: On Mon, 10 Sep 2007, Aaron Boodman wrote: On Sep 10, 2007 2:21 PM, Ian Hickson [EMAIL PROTECTED] wrote: I still don't understand how you see this working using the same codebase both online and offline. The model I'm proposing basically relies on the app being an offline app, except that while you're online the offline app is talking to the server to keep its database updated and the server updated with the user's changes. What you're describing seems like it would require a different set of code for the offline case than the online case. You can share the UI code for normal web apps with local-mode ones if you have a clean separation between the server and the UI. In fact, I think it will be common to want to do this. Bookmarks and URLs from the online app should work with the offline one and vice versa. I agree, I just don't see how this applies to the example Robert gave with Bugzilla. The idea of having some URIs be decomposed by the client-side and have them fetch different pages from the server seems really unwise, especially since in browsers that don't support this, they'll not be decomposed and will end up fetching different pages. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On 9/8/07, Ian Hickson [EMAIL PROTECTED] wrote: Given that you'd have to radically rewrite the app anyway to use an offline database instead of just using HTTP, why would we reuse the URI query syntax feature? It seems like it'd be better (from a consistency with existing specs point of view) to use the fragment identifer. No? That's an option, but then you can't use the fragment identifier for its scroll-to behaviour when you use it to pass parameters. Also, when retrofitting an existing app, this requires changing the sources of links as well as the destination pages. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Mon, 10 Sep 2007, Robert O'Callahan wrote: That's an option, but then you can't use the fragment identifier for its scroll-to behaviour when you use it to pass parameters. You can just pass the scroll position as one of the parameters. Also, when retrofitting an existing app, this requires changing the sources of links as well as the destination pages. That seems like a minor issue, compared to the scale of the effort. I'd be interested in studying an actual example of what this would entail. Do people feel strongly about wanting this dichtomy between the online version and the offline version? Fundamentally, I feel quite strongly that we'd want them to use the exact same code all the time. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
Still reading through... am I right assuming that the shortest attribute value would be #, i.e. html application=#, pointing to itself? :DG
Re: [whatwg] Offline Web Apps
On Fri, 7 Sep 2007, Dimitri Glazkov wrote: Still reading through... am I right assuming that the shortest attribute value would be #, i.e. html application=#, pointing to itself? The shortest value would be , pointing to itself. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
Erm. Right. :DG
Re: [whatwg] Offline Web Apps
On 8/24/07, Ian Hickson [EMAIL PROTECTED] wrote: * Do we really need a way to ignore the query parameters when fetching and serving from cache when offline? (The idea below assumes not. I don't really understand the use case if the answer is yes.) Yes. Suppose Bugzilla had offline support. The Bugzilla UI has a bunch of links it with with different query parameters (e.g., bug numbers). Bugzilla running offline could take those parameters and use them to look up a local database and populate the DOM. Maybe if we force offline apps to all be single-page then this isn't an issue but I don't think that's a good idea. Rob -- Two men owed money to a certain moneylender. One owed him five hundred denarii, and the other fifty. Neither of them had the money to pay him back, so he canceled the debts of both. Now which of them will love him more? Simon replied, I suppose the one who had the bigger debt canceled. You have judged correctly, Jesus said. [Luke 7:41-43]
Re: [whatwg] Offline Web Apps
On Sat, 8 Sep 2007, Robert O'Callahan wrote: Yes. Suppose Bugzilla had offline support. The Bugzilla UI has a bunch of links it with with different query parameters (e.g., bug numbers). Bugzilla running offline could take those parameters and use them to look up a local database and populate the DOM. Maybe if we force offline apps to all be single-page then this isn't an issue but I don't think that's a good idea. The new proposal doesn't rely on just having one top-level page. However, the query parameter thing seems overly complex. It means you're not just using the same file online as offline, which I'm strongly trying to keep, to ensure simple back compat. Given that you'd have to radically rewrite the app anyway to use an offline database instead of just using HTTP, why would we reuse the URI query syntax feature? It seems like it'd be better (from a consistency with existing specs point of view) to use the fragment identifer. No? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Fri, 24 Aug 2007, Maciej Stachowiak wrote: I think it's easy to extend Ian's idea in a way that keeps it really simple for the simple case, but that works better for the multi-page case or other complex cases where pages load some resources dynamically. html application=manifest-file The manifest file would indicate all resources used by the web app, including other pages, and other resources that may be loaded by the current page but normally would not be at startup (another problem with Ian's proposal IMO). Multiple pages that refer to the same manifest are considered part of the same web app and share the same cache. If you give an empty value for the application attribute, then the implicit thing that Ian describes happens - the resources that the page actually loads are the ones cached. Ok, new proposal: There's a concept of an application cache. An application cache is a group of resources, the group being identified by a URI (which typically happens to resolve to a manifest). Resources in a cache are either top-level or not; top-level resources are those that are HTML or XML and when parsed with scripting disabled have html application=... with the value of the attribute pointing to the same URI as identifies the cache. When you visit a page you first check to see if you have that page in a cache as a known top-level page. If you do, skip the next two paragraphs; the 'new cache' flag is set to false. If you don't, you fetch the URL. If it has no application= attribute, then do whatever the normal thing to do is. Ignore the rest of this. The presence of the attribute indicates that it's expecting an application cache to apply. The presence is detected at parse time, and must be present on the first html start tag before any other start tags. Check that the attribute's value is same-origin safe. If it isn't, pretend the attribute wasn't there (and ignore the rest of this). Otherwise, check to see if you already have a cache for the given URI. If you don't, create a new cache identified by the given URI. In any case, save this resource to the identified cache, as a known top-level page for that cache. Then, act as if you had known about the cache when you started (next step), except with the 'new cache' flag set to true. Load the page from the cache and display it. Any resources that the page tries to fetch using GETs that aren't XMLHttpRequest'ed must be taken from the cache, if available. When they aren't, the resources must be fetched then stored in the cache. Once the UA is ready to do so the UA must go on to the next steps. UAs may do this immediately, or may wait for the original page load to complete, or may delay it up to a UA-defined minimum delay. If 'new cache' is true, and the cache identifier URI is the same as the URI that was just downloaded and put in the cache: Do nothing. If 'new cache' is true, and the cache identifier URI is different from the URI that was just downloaded: Fetch the resource identified by that URI. Store it in the cache. If it's a manifest and it parses correctly, download all the URIs given in that manifest and add them to the cache. If any are HTML files which, when parsed with scripting disabled, trigger the application= handling and have a value that points to the same URI as the one identifying this application cache, then mark them as known top-levels for this cache. If 'new cache' is false: Create a new cache. Fetch the resource with the URI of the cache identifier. If it's a manifest, and it has changed from what's in the last cache, and it parses correctly, download all the URIs in that manifest and add them to the new cache. If the manifest has an upgrader entry, use that as the upgrader as described below. Otherwise, if it's not a manifest but an HTML/XML file, and it has changed from what's in the last cache, use that as the upgrader as described below. If it's a manifest that misparsed, or if it's another kind of file, then act as if it the URI just pointed to the top level page being loaded (and use that as the upgrader as described below). If the newly updated cache doesn't contain the current top-level page, then fetch that too. When a file is fetched by the main page loading in a background browsing context, the loads are conditional loads, so that files that haven't changed since the previous update are directly copied from the old cache. If the newly update cache's copy of the top level page being shown is no longer categorised as a known top-level for this cache (e.g. because it doesn't have an html application attribute any more) then inform the user, e.g. an infobar saying something like This application may no longer be available. (( View new page in a new window )) (( Delete application from cache )) (( Keep application in cache and check for updates later )) [x]. The first of these buttons would just show the background browsing context in the
Re: [whatwg] Offline Web Apps
Ian Hickson wrote: Comments? As useful as this would be, I can't help thinking that: 1. It's a huge time sink that will delay the rest of the spec. 2. We simply don't know enough at this point in time to reliably standardize something like this. Keep in mind that most of what this group is doing is based on 5-10 years of practical, hard-earned experience. Offline web apps are based on what we'd like to do at some point in the future. I would suggest this be a separate, experimental effort. When we know more about the problem, we can come back to it in a couple of years. -- Elliotte Rusty Harold [EMAIL PROTECTED] Java I/O 2nd Edition Just Published! http://www.cafeaulait.org/books/javaio2/ http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/
Re: [whatwg] Offline Web Apps
- Original Message - From: Ian Hickson [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Thursday, August 23, 2007 6:42 PM Subject: Offline Web Apps (If you reply, please only include one of the mailing lists in your reply. Thanks.) So I read through all the offline Web app discussions: http://www.whatwg.org/issues/#filesystem http://code.google.com/apis/gears/api_localserver.html http://www.campd.org/stuff/Offline%20Cache.html ...and various others. USER INTERACTION SCENARIOS It seems like we are talking about the following kinds of scenarios: 1. User goes to a page, then, without closing the page, goes offline and uses it, then goes back online and continues to use it. The page and its subresources are always at their most up-to-date. Interactions with the page while offline are synced to the server when going online. 2. User goes to a page, then closes the browser. User restarts the browser while offline, and goes to the page. User restarts the browser again while online, and goes to the page. The page and its subresources are always at their most up-to-date. Interactions with the page while offline are synced to the server when going online. 3. Same as 1 or 2, except that the user is not online for long enough to fully download the updated resources. The application doesn't stop working, it is still usable the second time the user is offline. OTHER NEEDS We also seem to have the following requirements: * Some way to specify what pages are needed while offline, even if the page doesn't link to it. * Some way to handle two offline Web apps both using the same secondary resource, if we have some sort of versioning scheme. (So you can update the two apps independently without breaking either despite them using the same resource.) * Resilience to updates -- if the page is open when you go online and there's an update pending, or when you go online just as you're loading the page (common with wireless networks, since you're likely to take a few seconds to connect and your browser is often ready beforehand) and there's an update pending. * Resilience to broken updates. (More than the reload button?) * There needs to be a way for the application to talk to the server without talking to the offline cache. * The API should be simple, both to authors on the client side, and to authors on the server side, and to the end user, and ideally to the implementors (and to me, the spec author!). There are two distinct types of applications (in context of this topic) 1) Online web applications. 2) Occasionally connected web applications (OCWA). These two groups differ significantly in their design. So what exactly you are trying to make offline aware? 1) Online web applications - to make transparent features on UA level that allow this group of apps to work offline. 2) To extend UA feature set to support occasionally connected web applications - applications that work offline as a primary operational mode. These applications require client side storage and other forms of persistence. I don't think that making online web applications offline aware makes any or significant sense. Or even feasible as they use data flow (RPC, aka AJAX) that implies active server. I propose to clarify first what we are trying to solve. Andrew Fedoniouk. http://terrainformatica.com
Re: [whatwg] Offline Web Apps
On Fri, 24 Aug 2007, Andrew Fedoniouk wrote: There are two distinct types of applications (in context of this topic) 1) Online web applications. 2) Occasionally connected web applications (OCWA). These two groups differ significantly in their design. So what exactly you are trying to make offline aware? Both. Basically we're aiming at the same kind of stuff as the Google Gears offline mode APIs, but using a standards-based API. So things like Google Reader, Yahoo! Mail, or even MSN Live Map Search or whatever it's called this week. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Aug 23, 2007, at 8:56 PM, Aaron Boodman wrote: On Aug 23, 2007 8:18 PM, Ian Hickson [EMAIL PROTECTED] wrote: On Thu, 23 Aug 2007, Maciej Stachowiak wrote: I haven't read over the details but there seems to be an obvious showstopper problem: this won't work for web applications that consist of more than one page. Indeed, that was called out as a potential issue. But is that really a problem? It's easy to work around (e.g. with iframe)s if you really must have multiple top-level pages, but aren't web apps moving to a one-page model anyway? The problem gets significantly more complicated with multiple top- level pages all taking part in the same conceptual application. The single-page model has other nice advantages. For example, there's never any confusion about which cache should serve a resource. It's the one that's associated with the application which the resource is contained in. Could multi-page apps be addressed by letting applications specify that other applications should be cached (using a similar api to the one that lets applications programatically cache resources)? I don't think that works very well - you'd have to parse all the HTML, CSS and scripts associated with those other pages just to do the caching. That's a huge cost compared to just downloading the resources. Consider web apps like flickr and upcoming which consist of many many pages. Obviously these specific examples can't cache all of their pages offline but they may well want to cache a significant subset that is interesting to the user. I think it's easy to extend Ian's idea in a way that keeps it really simple for the simple case, but that works better for the multi-page case or other complex cases where pages load some resources dynamically. html application=manifest-file The manifest file would indicate all resources used by the web app, including other pages, and other resources that may be loaded by the current page but normally would not be at startup (another problem with Ian's proposal IMO). Multiple pages that refer to the same manifest are considered part of the same web app and share the same cache. If you give an empty value for the application attribute, then the implicit thing that Ian describes happens - the resources that the page actually loads are the ones cached. My thoughts on this aren't fully fleshed out, but Ian said he wants to start editing the spec right away so I wanted to give this minimal feedback before he writes up something in detail that I would strongly object to. Regards, Maciej
Re: [whatwg] Offline Web Apps
- Original Message - From: Ian Hickson [EMAIL PROTECTED] To: Andrew Fedoniouk [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Friday, August 24, 2007 11:39 AM Subject: Re: Offline Web Apps On Fri, 24 Aug 2007, Andrew Fedoniouk wrote: There are two distinct types of applications (in context of this topic) 1) Online web applications. 2) Occasionally connected web applications (OCWA). These two groups differ significantly in their design. So what exactly you are trying to make offline aware? Both. Basically we're aiming at the same kind of stuff as the Google Gears offline mode APIs, but using a standards-based API. So things like Google Reader, Yahoo! Mail, or even MSN Live Map Search or whatever it's called this week. Google Gears has three modules so far: 1) Application delivery module, a.k.a. LocalServer; 2) Local database module; 3) Async JS execution module, a.k.a. WorkerPool. Local database module is clearly a part of OCWA functionality. Client side DB cannot make any good if application is not designed to use it. Async execution has nothing too specific to online/offline - this is what JS will benefit in general. (I believe that 'yield' and friends in JS should allow to do this without any additional features - generator functions use their own stack - so are threads generally speaking) So we can discuss/focus on Application Delivery Module. Applications like you've mentioned (Google Reader, etc.) consist of so called mostly static pages/resources/scripts and use local DB requests and/or RPC (remote procedure calls, AJAX kind of thing) to get relatime data (JSON or HTML fragments) to compose final UI. In principle such applications (their client side part) can be packaged in something like zip file and delivered as a whole to the client machine. In this case distribution of them is not different from ordinary zip file downloading and caching. This is very close to what Java uses in years (ClassLoader, NetworkClassLoader and other already invented wheels). Application can consist of multiple distribution units (WADU - web application distribution unit - zip file for example). To be able to use resources from other DUs the page should have mechanism similar to include/use directives. This could be some separate manifest/configuration file or simply translation directives like: meta use=/module-a.zip as=/module-a/ Such a meta will allow to support old UAs (that do not support packaging) - web server can contain web app as packaged modules together with the old style tree of resources. Pages (HTML) of application can be mostly static - this means that they may have some dynamic islands. As an example: include src=/some-advertisement/fragment.htm /. That will include html content (and only, no scripts or CSS) from other locations. Update cycle is defined by http headers of refered html. include is purely client side thing - sort of lightweight iframe means is not a part of server side html processing. Therefore, I think, in context of the discussion it makes sense to take a look on better distribution schemas too. Andrew Fedoniouk. http://terrainformatica.com
Re: [whatwg] Offline Web Apps
Intuitively, I think I agree with Maciej. Manifest is not as elegant as participation by association approach, but it allows for better packaging an application. I am thinking about scripts/stylesheets that are typically a limited set of resources, reused throughout an application. I also don't yet understand why Ian wants to store multiple versions of resource representations, if same representation is used by multiple apps. What is the motivation here? ... and how would one make an app like Twitter or Facebook available offline? Perhaps a markup attribute is not a good idea in this case, where every profile page is potentially an application. I am thinking that only _your_ profile page is an offline app. Right? On 8/24/07, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Aug 23, 2007, at 8:56 PM, Aaron Boodman wrote: On Aug 23, 2007 8:18 PM, Ian Hickson [EMAIL PROTECTED] wrote: On Thu, 23 Aug 2007, Maciej Stachowiak wrote: I haven't read over the details but there seems to be an obvious showstopper problem: this won't work for web applications that consist of more than one page. Indeed, that was called out as a potential issue. But is that really a problem? It's easy to work around (e.g. with iframe)s if you really must have multiple top-level pages, but aren't web apps moving to a one-page model anyway? The problem gets significantly more complicated with multiple top- level pages all taking part in the same conceptual application. The single-page model has other nice advantages. For example, there's never any confusion about which cache should serve a resource. It's the one that's associated with the application which the resource is contained in. Could multi-page apps be addressed by letting applications specify that other applications should be cached (using a similar api to the one that lets applications programatically cache resources)? I don't think that works very well - you'd have to parse all the HTML, CSS and scripts associated with those other pages just to do the caching. That's a huge cost compared to just downloading the resources. Consider web apps like flickr and upcoming which consist of many many pages. Obviously these specific examples can't cache all of their pages offline but they may well want to cache a significant subset that is interesting to the user. I think it's easy to extend Ian's idea in a way that keeps it really simple for the simple case, but that works better for the multi-page case or other complex cases where pages load some resources dynamically. html application=manifest-file The manifest file would indicate all resources used by the web app, including other pages, and other resources that may be loaded by the current page but normally would not be at startup (another problem with Ian's proposal IMO). Multiple pages that refer to the same manifest are considered part of the same web app and share the same cache. If you give an empty value for the application attribute, then the implicit thing that Ian describes happens - the resources that the page actually loads are the ones cached. My thoughts on this aren't fully fleshed out, but Ian said he wants to start editing the spec right away so I wanted to give this minimal feedback before he writes up something in detail that I would strongly object to. Regards, Maciej
Re: [whatwg] Offline Web Apps
On Aug 23, 2007, at 6:42 PM, Ian Hickson wrote: IDEA Ok so here's my idea based on the existing ideas, the comments on those ideas, and so forth. One of my main goals was keeping everything as simple as possible. My proposal is that we add a new attribute to the html element, which flags that the page is a Web app that wants to be pinned for offline execution and that when you next fetch the file or one of its subresources while online, it should try to update all the subresources atomically. html application This could either trigger the different behaviour automatically, or only when requested by the user, or we could say this applies to any page, but that the user must enable it, or we could provide an API which triggers this for the page. I kind of like the explicit attribute, though. ... Comments? I haven't read over the details but there seems to be an obvious showstopper problem: this won't work for web applications that consist of more than one page. Regards, Maciej
Re: [whatwg] Offline Web Apps
On Thu, 23 Aug 2007, Maciej Stachowiak wrote: I haven't read over the details but there seems to be an obvious showstopper problem: this won't work for web applications that consist of more than one page. Indeed, that was called out as a potential issue. But is that really a problem? It's easy to work around (e.g. with iframe)s if you really must have multiple top-level pages, but aren't web apps moving to a one-page model anyway? The problem gets significantly more complicated with multiple top-level pages all taking part in the same conceptual application. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Offline Web Apps
On Aug 23, 2007 8:18 PM, Ian Hickson [EMAIL PROTECTED] wrote: On Thu, 23 Aug 2007, Maciej Stachowiak wrote: I haven't read over the details but there seems to be an obvious showstopper problem: this won't work for web applications that consist of more than one page. Indeed, that was called out as a potential issue. But is that really a problem? It's easy to work around (e.g. with iframe)s if you really must have multiple top-level pages, but aren't web apps moving to a one-page model anyway? The problem gets significantly more complicated with multiple top-level pages all taking part in the same conceptual application. The single-page model has other nice advantages. For example, there's never any confusion about which cache should serve a resource. It's the one that's associated with the application which the resource is contained in. Could multi-page apps be addressed by letting applications specify that other applications should be cached (using a similar api to the one that lets applications programatically cache resources)? - a
Re: [whatwg] Offline Web Apps
On Thu, 23 Aug 2007, Aaron Boodman wrote: The single-page model has other nice advantages. For example, there's never any confusion about which cache should serve a resource. It's the one that's associated with the application which the resource is contained in. Indeed. Could multi-page apps be addressed by letting applications specify that other applications should be cached (using a similar api to the one that lets applications programatically cache resources)? Ooh, that might work. Yeah. Maciej, what do you think? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
