Re: DataCache API - editor's draft available
This is very encouraging. It sounds like we agree in principal that the best place for the features represented in the DataCache proposal are extensions to the AppCache corner of the world. My preference would be to keep the official AppCache spec stable right now, but to begin designing those extensions sooner rather than later. This would help inform the architecture of AppCache implementations underway by browser vendors today, and help better inform application teams of what to expect as well. The DataCache proposal is a great starting point. In a sense, specify these extensions on a development branch of the spec. On Wed, Jul 22, 2009 at 10:56 PM, Ian Hickson i...@hixie.ch wrote: On Wed, 22 Jul 2009, Adrian Bateman wrote: My preference would be to see this functionality proposed as an evolution of AppCache. While I can't commit that we would implement it any time soon, it would be included in our considerations and at the very least if we implement AppCache we would try to ensure any architecture we select wouldn't preclude these extensions in the future. With work on the HTML5 spec trying to get it locked down towards a Last Call, adding new features to that document is clearly less desirable and I understand Ian's reluctance to incorporate extensions there. My preference in general would be for us to wait until we have stable, well-tested implementations before adding more features to appcache, but I certainly think that on the long term, appcache should be extended to support more than it does now. As it stands, the model is quite extensible, so I think it would be relatively easy for us to move in that direction in the future. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: DataCache API - editor's draft available
On Jul 22, 2009, at 10:56 PM, Ian Hickson wrote: On Wed, 22 Jul 2009, Adrian Bateman wrote: My preference would be to see this functionality proposed as an evolution of AppCache. While I can't commit that we would implement it any time soon, it would be included in our considerations and at the very least if we implement AppCache we would try to ensure any architecture we select wouldn't preclude these extensions in the future. With work on the HTML5 spec trying to get it locked down towards a Last Call, adding new features to that document is clearly less desirable and I understand Ian's reluctance to incorporate extensions there. My preference in general would be for us to wait until we have stable, well-tested implementations before adding more features to appcache, but I certainly think that on the long term, appcache should be extended to support more than it does now. As it stands, the model is quite extensible, so I think it would be relatively easy for us to move in that direction in the future. Don't we have AppCache implementations in Firefox and Safari? Are there parts of the spec that we don't yet have implementation experience or enough feedback about? I happen to think that work on DataCache should not wait for the long term to arrive. May be it is just me, but I don't see harm in getting to an FPWD on DataCache before HTML5 Last Call. I do agree that we should not create an impression that AppCache is solid, and so am open to suggestions about avoiding that. Nikunj http://o-micron.blogspot.com
Re: DataCache API - editor's draft available
On Thu, 23 Jul 2009, Nikunj R. Mehta wrote: Don't we have AppCache implementations in Firefox and Safari? Are there parts of the spec that we don't yet have implementation experience or enough feedback about? I happen to think that work on DataCache should not wait for the long term to arrive. May be it is just me, but I don't see harm in getting to an FPWD on DataCache before HTML5 Last Call. Moving fast is the main way to introduce bugs into the platform. The Web platform has enough bugs already, we don't need to make it worse. I would be much happier waiting a year even, getting multiple implementations shipped, have lots of sites using them, have the features really well understood by authors and implementors... We need to let enough time pass to make sure there are no latent fundamental design problems, security bugs, performance issues, etc. There's no rush, and there's plenty of other stuff being implemented as well. What we have today is two barely tested implementations with virtually no usage and comparatively little deployment. Let's not get ahead of ourselves. Having said that, it's certainly fine to work on proposals, to bang on ideas for future versions, and so on. I just don't think it's appropriate to work on formalising those ideas yet. Of course, I'm just one person, if the working group wants to go faster, I can't stop y'all. :-) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: DataCache API - editor's draft available
On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote: I don't think the problem is that we couldn't build yet another cache that is similar but different to the AppCache that others are already shipping so I don't think a reference implementation is the solution. I think the problem is motivation - are there any use cases that adding DataCache enables that couldn't otherwise be implemented with what we already have and are those compelling enough to complicate the platform with another cache mechanism. Further, would we end up with conflicts between the AppCache and the DataCache since they're currently not unified as far as I can tell. [...] What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This is pretty close to my own point of view. I think the functionality offered by DataCache is interesting, but I'd be much more interested in putting it into Safari and WebKit if it was exposed as a set of extra AppCache features, rather than a competing and unintegrated separate model. I believe AppCache can reasonably be extended to support the extra functionality of DataCache. For example, it should be possible extend the manifest format with a way to express that a certain URI should be served by script, and to give that script access to a static cached copy if it chooses to use it. This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. I agree with this as well. I think extending AppCache is likely to be a more successful approach than building a full alternative. I will go further and say that if some of the advanced DataCache features were expressed as extensions AppCache, we would likely be very interested in implementing them. Regards, Maciej
Re: DataCache API - editor's draft available
On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote: While it might not be the perfect solution (we know the web far from ideal and is a lot of compromise), this type of proposal would be a lot more compelling to me if I could say This is what we have to add, this is how, and here are the use cases that make it valuable with a roadmap for extending what people are already building instead of something brand new. Would you mind explaining the last point with a roadmap for extending what people are already building instead of something brand new. for me? I would definitely strive to make the proposal more compelling. What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. What kind of extensions/changes to AppCache would be acceptable at this point? In a previous exchange with Ian, he declined to consider DataCache like extensions to ApplicationCache for HTML5, which might be the reasonable thing to do. I can of course put together a proposal, but it would be good to know from browser vendors what their preferences are in this matter. I am open to the idea of incrementally evolving AppCache to be more supportive of DataCache requirements. Regards, Nikunj
RE: DataCache API - editor's draft available
On Tuesday, July 21, 2009 11:26 PM, Nikunj R. Mehta wrote: On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote: What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. What kind of extensions/changes to AppCache would be acceptable at this point? In a previous exchange with Ian, he declined to consider DataCache like extensions to ApplicationCache for HTML5, which might be the reasonable thing to do. I can of course put together a proposal, but it would be good to know from browser vendors what their preferences are in this matter. I am open to the idea of incrementally evolving AppCache to be more supportive of DataCache requirements. At this point we haven't made a commitment one way or the other about whether we will implement support for AppCache in the next release of IE (or at all). It is supported in at least two shipping browsers and so we're naturally considering the possibility (along with a huge list of other possible features). I am confident that we won't implement a competing and unintegrated separate model, as Maciej put it, such as DataCache. As I've said, I think that would be a mistake. My preference would be to see this functionality proposed as an evolution of AppCache. While I can't commit that we would implement it any time soon, it would be included in our considerations and at the very least if we implement AppCache we would try to ensure any architecture we select wouldn't preclude these extensions in the future. With work on the HTML5 spec trying to get it locked down towards a Last Call, adding new features to that document is clearly less desirable and I understand Ian's reluctance to incorporate extensions there. I think a separate spec written here that proposes just the extensions and explains how it relates to the AppCache analogous to your DataCache spec would be appropriate but that's just my personal preference. Cheers, Adrian.
Re: DataCache API - editor's draft available
On Jul 21, 2009, at 11:25 PM, Nikunj R. Mehta wrote: On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote: While it might not be the perfect solution (we know the web far from ideal and is a lot of compromise), this type of proposal would be a lot more compelling to me if I could say This is what we have to add, this is how, and here are the use cases that make it valuable with a roadmap for extending what people are already building instead of something brand new. Would you mind explaining the last point with a roadmap for extending what people are already building instead of something brand new. for me? I would definitely strive to make the proposal more compelling. What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. What kind of extensions/changes to AppCache would be acceptable at this point? In a previous exchange with Ian, he declined to consider DataCache like extensions to ApplicationCache for HTML5, which might be the reasonable thing to do. I can of course put together a proposal, but it would be good to know from browser vendors what their preferences are in this matter. I am open to the idea of incrementally evolving AppCache to be more supportive of DataCache requirements. We'd be willing to consider implementing AppCache extensions in WebKit even if they were initially proposed in a separate specification from HTML5, assuming there was rough consensus among browser vendors that the extensions are a good idea. I think the ability to have a resource that is served by a client-side script would be an example of a reasonable extensions. I'm not totally sure how to recast all of the DataCache ideas to fit with the AppCache model. I'd be glad to review and provide suggestions, if everyone thinks this is a good idea. Regards, Maciej
Re: DataCache API - editor's draft available
On Wed, Jul 22, 2009 at 4:00 PM, Maciej Stachowiak m...@apple.com wrote: On Jul 21, 2009, at 11:25 PM, Nikunj R. Mehta wrote: On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote: While it might not be the perfect solution (we know the web far from ideal and is a lot of compromise), this type of proposal would be a lot more compelling to me if I could say This is what we have to add, this is how, and here are the use cases that make it valuable with a roadmap for extending what people are already building instead of something brand new. Would you mind explaining the last point with a roadmap for extending what people are already building instead of something brand new. for me? I would definitely strive to make the proposal more compelling. What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. What kind of extensions/changes to AppCache would be acceptable at this point? In a previous exchange with Ian, he declined to consider DataCache like extensions to ApplicationCache for HTML5, which might be the reasonable thing to do. I can of course put together a proposal, but it would be good to know from browser vendors what their preferences are in this matter. I am open to the idea of incrementally evolving AppCache to be more supportive of DataCache requirements. We'd be willing to consider implementing AppCache extensions in WebKit even if they were initially proposed in a separate specification from HTML5, assuming there was rough consensus among browser vendors that the extensions are a good idea. I think the ability to have a resource that is served by a client-side script would be an example of a reasonable extensions. I'm not totally sure how to recast all of the DataCache ideas to fit with the AppCache model. I'd be glad to review and provide suggestions, if everyone thinks this is a good idea. I've talked to several people here at Google (including people who have worked on Gears) and I think we all agree there are some good ideas in DataCache, but we also agree that they should be framed into the AppCache model. Unfortunately, I don't think anyone has any precise ideas on how to do this yet. I think it's safe to say that we'd also be glad to review and provide suggestions/experience.
Re: DataCache API - editor's draft available
On Wed, Jul 22, 2009 at 4:00 PM, Maciej Stachowiak m...@apple.com wrote: On Jul 21, 2009, at 11:25 PM, Nikunj R. Mehta wrote: On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote: While it might not be the perfect solution (we know the web far from ideal and is a lot of compromise), this type of proposal would be a lot more compelling to me if I could say This is what we have to add, this is how, and here are the use cases that make it valuable with a roadmap for extending what people are already building instead of something brand new. Would you mind explaining the last point with a roadmap for extending what people are already building instead of something brand new. for me? I would definitely strive to make the proposal more compelling. What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. What kind of extensions/changes to AppCache would be acceptable at this point? In a previous exchange with Ian, he declined to consider DataCache like extensions to ApplicationCache for HTML5, which might be the reasonable thing to do. I can of course put together a proposal, but it would be good to know from browser vendors what their preferences are in this matter. I am open to the idea of incrementally evolving AppCache to be more supportive of DataCache requirements. We'd be willing to consider implementing AppCache extensions in WebKit even if they were initially proposed in a separate specification from HTML5, assuming there was rough consensus among browser vendors that the extensions are a good idea. I think the ability to have a resource that is served by a client-side script would be an example of a reasonable extensions. I'm not totally sure how to recast all of the DataCache ideas to fit with the AppCache model. I'd be glad to review and provide suggestions, if everyone thinks this is a good idea. Thank you for putting this proposal together. The Chrome team is willing as well to consider adding extensions along these lines to Chrome. And several app teams here at Google would be willing to put them to good use. Regards, Maciej
RE: DataCache API - editor's draft available
On Wed, 22 Jul 2009, Adrian Bateman wrote: My preference would be to see this functionality proposed as an evolution of AppCache. While I can't commit that we would implement it any time soon, it would be included in our considerations and at the very least if we implement AppCache we would try to ensure any architecture we select wouldn't preclude these extensions in the future. With work on the HTML5 spec trying to get it locked down towards a Last Call, adding new features to that document is clearly less desirable and I understand Ian's reluctance to incorporate extensions there. My preference in general would be for us to wait until we have stable, well-tested implementations before adding more features to appcache, but I certainly think that on the long term, appcache should be extended to support more than it does now. As it stands, the model is quite extensible, so I think it would be relatively easy for us to move in that direction in the future. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
RE: DataCache API - editor's draft available
On Monday, July 20, 2009 1:43 PM, Nikunj R. Mehta wrote: On Jul 17, 2009, at 9:31 AM, Adrian Bateman wrote: On Thursday, July 16, 2009 4:46 PM, Nikunj R. Mehta wrote: On Jul 16, 2009, at 3:31 PM, Adrian Bateman wrote: ... Does the situation change if I am building an application from scratch that wants to deal with network disconnections? In that case, you should be able to take advantage of the presence of interceptors and write your code to move to the client when client- side execution is desirable. If this application is used in environments that do not support interception, then it will work just fine with server-side execution. Of course, the application won't do the right recovery when it is disconnected if interception is not supported. Thanks for this and the preceding answers that I snipped. I'm not sure I find this model more compelling than the imperative equivalent, but okay. We can agree to disagree on which is more natural to web developers. I understand that your DataCache proposal provides different functionality to AppCache. I wasn't suggesting that they were equivalent. I am glad to see that you see a different set of functions offered in DataCache. Several commenters before you have taken the exact opposite position. I didn't interpret the other responses I read in that way but I may have mis-read them. I thought they were making a similar point to my next one. However, the AppCache as specified is getting implemented today (it is in Safari and Firefox with other implementations on the way). The problem I have is that I can't go implement AppCache and then add on some new methods and alternative interception-path to get DataCache- like features - I have to write a completely independent mechanism. If I propose that we build AppCache and then DataCache, someone will say, Hang on, didn't we just build this - why do we need two? OK. I get that. I also consider this to be an important challenge for everyone concerned. I thought I would write a concrete proposal. Oracle could help out with building a reference implementation for multiple browsers and make its source code available. What else can I do to help? I don't think the problem is that we couldn't build yet another cache that is similar but different to the AppCache that others are already shipping so I don't think a reference implementation is the solution. I think the problem is motivation - are there any use cases that adding DataCache enables that couldn't otherwise be implemented with what we already have and are those compelling enough to complicate the platform with another cache mechanism. Further, would we end up with conflicts between the AppCache and the DataCache since they're currently not unified as far as I can tell. While it might not be the perfect solution (we know the web far from ideal and is a lot of compromise), this type of proposal would be a lot more compelling to me if I could say This is what we have to add, this is how, and here are the use cases that make it valuable with a roadmap for extending what people are already building instead of something brand new. Would you mind explaining the last point with a roadmap for extending what people are already building instead of something brand new. for me? I would definitely strive to make the proposal more compelling. What I'm asking for is a more unified proposal that says If you have already implemented AppCache, here's what you add to make the same cache provide the additional functionality needed to enable these additional use cases. This will inevitably be a compromise from what a pure implementation looks like (your current DataCache spec, say) but lots of the web is necessarily a compromise because it builds on prior art that might not have been ideal but has been specified, built and deployed (and not always in that order). This would allow people to form a judgement about whether the additional use cases are worth the additional effort instead of whether the additional use cases are worth yet another cache. I think the ship has already sailed on AppCache and we can't undo that even if we wanted to and I don't think a competing solution is the right approach. Cheers, Adrian.
Re: DataCache API - editor's draft available
Hi Mark, I am happy to see your feedback on DataCache. Forgive me for the delay in responding. On Jul 17, 2009, at 4:50 PM, Mark Nottingham wrote: I think this work is in an interesting space but, unfortunately, it's doing it without reference to the existing HTTP caching model, resulting in a lot of duplicated work, potential conflicts and ambiguities, as well as opportunity cost. I don't understand this fully, can you please explain? From what I know, the Gears implementation can be easily extended to support DataCache. Of course, one doesn't need all of Gears - only LocalServer and browser integration is required. I don't see that as a lot of duplicated work. Furthermore, it's specifying an API as the primary method of controlling caches. While that's understandable if you look at the world as a collection of APIs, it's also quite limiting; it precludes reuse of information, unintended uses, and caching by anything except the browser. FWIW, DataCache is not the first attempt at obtaining an API to control a browser's HTTP cache. That was already the case with ApplicationCache in HTML5. I don't quite understand what problems you foresee with DataCache's approach. It does not ask the implementor to violate any HTTP caching semantics. If anything, it suggests that the implementation can offer an off-line response should an on-line response be infeasible. This is based on my reading of the following pieces of text from RFC2616. From §13, [[ Requirements for performance, availability, and disconnected operation require us to be able to relax the goal of semantic transparency. ... Protocol features that allow a cache to attach warnings to responses that do not preserve the requested approximation of semantic transparency. ]] From §13.1.6 [[ A client MAY also specify that it will accept stale responses, up to some maximum amount of staleness. This loosens the constraints on the caches, and so might violate the origin server's specified constraints on semantic transparency, but might be necessary to support disconnected operation, or high availability in the face of poor connectivity. ]] Can you please correct me if I have misinterpreted or misapplied these provisions of HTTP? Alternatively, can you point me to a valid interpretation of these portions in the context of an open implementation/application?c A much better solution would be to declaratively define what URIs delineate an application (e.g., in response headers and/or a separate representation), and then allow clients to request an entire application to be fetched before they go offline (for example). I'm aware that there are other use cases and capabilities here, this is just one example. Am I correct in understanding that you find pre-fetching the entire application to be better than pre-fetching parts of it. In any case, are you also suggesting a data format for specifying a collection of such URIs that the user agent should pin down in cache? How does a data format form a better solution as opposed to an API? Additionally, it is not always possible to statically define the collection of URIs that are of interest to an application. Let me take an example - *Sales force automation* My sales reps work in parts of the world where assuming a reliable network connection is not a good assumption to make. Still I would like to deploy order entry applications that work reliably in the face of poor network connection on a small mobile computer with a Web browser. Today I am going on a round of my customers in Fallujah and I need to have information about customers in that area, including their names, addresses, and order history (and status). This information changes regularly and my sales reps benefit from up-to-the-minute order history information if I can connect to the server at the time I am at the customer's office. If I don't have network access, I at least have up-to-the-date information. Finally, I want to enable the sales rep to take orders when they are out in the field and provided they don't lose the device, I want to assure them that their orders will make it to the company's servers. If connectivity is available at that instant, then the order will be confirmed immediately and processing would begin. If not, it would be kept pending. Developers until now have developed and deployed such off-line applications outside the context of the Web architecture - i.e., no URIs, no uniform methods, etc. They will continue to do the same with SQL databases inside Web browsers - still no URIs, a single method - POST - and an off-line only solution (meaning it cannot take opportunistic advantage of available networks). Is this a more desirable approach than to provide an API to a subset of the browser's HTTP cache? Doubtless there's still a need for some new APIs here, but I think they should
Re: DataCache API - editor's draft available
On Jul 17, 2009, at 1:40 PM, Jonas Sicking wrote:
On Thu, Jul 16, 2009 at 4:17 PM, Nikunj R. Mehtanikunj.me...@oracle.com
wrote:
On Jul 16, 2009, at 3:54 PM, Jonas Sicking wrote:
I do understand how Interceptor/DataCache works. And understand that
it's seamless and can (based on a decision made by the browser)
seamlessly intercept both XHR requests and img src=.. requests.
What is not entirely clear to me is how you envision that authors
will use
it.
Authors will use it in all three kinds of use cases I explained
earlier -
XHR, form, and hyperlinked data. DataCache is designed to support
all three.
The intention is to accommodate any use case arising from the
issuance of a
network request when the device is off-line.
I.e. are you expecting authors to want to intercept XHR requests?
For data driven applications, I expect this usage scenario.
Or
img src=... requests? Or script src=... requests?
script src=... should be adequately supported by HTML5
(ApplicationCache),
unless the src value is dynamic in nature. For the static case, I
don't
expect to see much usage of DataCache.
Ok that answers my question.
See, what I don't fully understand is what the use cases are. For
static resources like static images, scripts, plugins etc, it seems to
me that the HTML5 ApplicationCache should work, as you point out.
Not entirely. Static resources like stylesheets are adequately covered
by ApplicationCache. Scripts (to the extent they are used for JSON or
JSONP) are not adequately covered by ApplicationCache.
Moreover, images (when used for more than laying out the page) are
also inadequately covered by ApplicationCache. For example, how does
one store a personal Flickr image set using ApplicationCache? I mean,
if I want to keep a copy of my image sets locally, and keep that set
synchronized with Flickr, does ApplicationCache (with its requirement
to version manifests, and require the entire manifest to be statically
identified and served in a special data format) appear like the right
thing to use?
Same goes with other resources that require helper applications to use
for viewing. For example, document attachments also don't appear to be
well supported by ApplicationCache.
As I see it, the ApplicationCache is primarily for keeping all the
nuts and bolts that hold together the application pinned in the
browser's cache. It does not serve a useful purpose for any other kind
of resources even if the resources themselves are relatively static.
So the static vs. dynamic resource differentiation needs to be seen in
a different light. Probably as to whether the resource is user-
specific or not and whether the resource need is time-varying.
For dynamic images, I think canvas is what you'd want to use anyway
since generating encoded png images and serving it through the http
Interceptor is a whole lot more complicated.
I am not so interested in dynamic images. If anything, I am looking at
XML documents, JSONP documents etc. as the dynamic content more.
For loading data using XHR, or submitting data using a form, it seems
like if the web application is abstracted at a higher level, then http
interception isn't strictly needed. If you currently have code like:
function readData(callback) {
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (xhr.readyState == 4 xhr.status == 200)
callback(xhr.responseText);
}
xhr.open(...);
xhr.send();
}
Then in order to support offline you could do:
function readData(callback) {
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (xhr.readyState == 4 xhr.status == 200)
callback(xhr.responseText);
}
xhr.onerror = function() {
text = readDataFromLocalStore();
callback(text);
}
xhr.open(...);
xhr.send();
}
The problem here is three fold:
1. You can't do this with things other than XHR - think about form
submissions, link navigations, and JSONP script references.
2. The current storage APIs are inadequate for storing resource
representations (headers and entities - text and binary).
3. Lastly, but most importantly, I lose continuity of the user's view
over the data, unless I do a lot of extra bookkeeping. Let me explain
further by continuing with the code you use in the second example above:
Let's assume that the xhr was for a POST or PUT request. Now the state
of the resource has changed and so no longer is the value of the uri
used with xhr the same as what it was before the xhr. When
readDataFromLocalStore (written to handle update requests, and not
just read requests) responds with the results of this request, I need
to ensure that subsequent XHRs benefit from that result as opposed to
a previously stored data, which would be inconsistent with the result.
This is something interception and dynamic caching allows me to do
that cannot be done with existing APIs.
So I'm not sure what new use cases we are
Re: DataCache API - editor's draft available
On Jul 17, 2009, at 9:31 AM, Adrian Bateman wrote: On Thursday, July 16, 2009 4:46 PM, Nikunj R. Mehta wrote: On Jul 16, 2009, at 3:31 PM, Adrian Bateman wrote: I agree with Jonas and I'd like to understand the expected use cases better too. I think I get the point that making the network access seamless regardless of whether there is network connectivity or not simplifies the network request code but seems to me to require more complex interception code. There is a tradeoff when adding the complexity of unreliable networks - either more complex client application code, or more complex client interception code. In our experience, we have found the latter to be actually more manageable for cases where the interception and off-line serving is done in a relatively application-independent manner. This isn't a pattern that I can readily map to customer requests we've received nor is it a familiar pattern to me personally. DataCache's interception pattern has existed for a while. A good survey of mobile data management and transaction support has covered this pattern [1]. The discussion there is not limited to Web browsers, though. I scanned through the paper you cite but it isn't clear to me which part of that is equivalent to your proposal. Is it possible to point out which section I should read more closely? The paper talks about flexible client-server architecture in section 2.1: [[ The roles of clients and servers as well as the application functionalities can be dynamically relocated. The distinction between clients and servers may be temporarily blurred for performance and availability purposes. ]] Further, the paper identifies one of the mobile file systems as Coda in Section 3.4 [1]. If you were to read the Coda paper, you would see the following text in its section 3.2. By the way, Coda was first conceived in early 1987 at CMU by Prof. M. Satyanarayanan's group. [[ Individual replicas are not normally visible to users or application programs. All they see is an ordinary file system mounted at /coda. This illusion is created by the Venus cache manager on the client. Venus intercepts all file references to /coda, discovers which servers hold replicas of a volume and fetches file and directory data as necessary from those servers, and manages cache copies of this data using files in the local file system as containers. ]] The pattern that isn't familiar to me is one where it appears that application logic is provided by interception. Given the generic nature of the Mobile Transactions paper you focus on application independence, are you proposing that the DataCache is part of a solution with no application specific code? Interception provides for the ability to introduce application- specific code that in emulating the server during disconnection. In other words is there some platform component, whether that be a JavaScript library or otherwise server-provided infrastructure, that operates generally at the interception point. An application can choose to use a library via which application- specific code is tied to the interception, but this is not necessary. Alternatively, is it application specific code that will be hand written for my use case every time? Generally speaking, yes. However a certain class of applications may choose to use a protocol such as Atom which generalizes the server behavior in terms of a specific set of semantics. In such a case, it may be possible to share a single library across applications that may do the interception. When Jonas asks the question about how do you expect people to use this, what comes to mind for me is a question about a scenario. I'm a web developer and I've built an application that makes XHR calls to exchange data with my server. I'd like to provide access to this application in situations where I am occasionally disconnected because of intermittent network connectivity. This is precisely where the application programmer needs to make a call - do I want to offer this functionality to a user when they are unable to access the network. If not, then skip the rest and return with just an error message. Do I now sit down and try to figure out what network requests I need to intercept and write JavaScript code for handling all of those requests? For those application behaviors, that require HTTP requests, that are to be supported off-line, I need to figure out what the off-line response should be. Then I have to translate this in to JavaScript code and attach it to interceptors. Certain application developers will be smart about out how to move code from the server environment to a client environment. Some developers will figure out an automated process for doing so, whereas others will be able to do so with manual support and computer guidance. In some cases, neither may be possible. Or is there a library I can just plug in
RE: DataCache API - editor's draft available
On Thursday, July 16, 2009 4:46 PM, Nikunj R. Mehta wrote: On Jul 16, 2009, at 3:31 PM, Adrian Bateman wrote: I agree with Jonas and I'd like to understand the expected use cases better too. I think I get the point that making the network access seamless regardless of whether there is network connectivity or not simplifies the network request code but seems to me to require more complex interception code. There is a tradeoff when adding the complexity of unreliable networks - either more complex client application code, or more complex client interception code. In our experience, we have found the latter to be actually more manageable for cases where the interception and off-line serving is done in a relatively application-independent manner. This isn't a pattern that I can readily map to customer requests we've received nor is it a familiar pattern to me personally. DataCache's interception pattern has existed for a while. A good survey of mobile data management and transaction support has covered this pattern [1]. The discussion there is not limited to Web browsers, though. I scanned through the paper you cite but it isn't clear to me which part of that is equivalent to your proposal. Is it possible to point out which section I should read more closely? The pattern that isn't familiar to me is one where it appears that application logic is provided by interception. Given the generic nature of the Mobile Transactions paper you focus on application independence, are you proposing that the DataCache is part of a solution with no application specific code? In other words is there some platform component, whether that be a JavaScript library or otherwise server-provided infrastructure, that operates generally at the interception point. Alternatively, is it application specific code that will be hand written for my use case every time? When Jonas asks the question about how do you expect people to use this, what comes to mind for me is a question about a scenario. I'm a web developer and I've built an application that makes XHR calls to exchange data with my server. I'd like to provide access to this application in situations where I am occasionally disconnected because of intermittent network connectivity. Do I now sit down and try to figure out what network requests I need to intercept and write JavaScript code for handling all of those requests? Or is there a library I can just plug in that, with some configuration, does that for me? Or is this something my server infrastructure should already support and if it does I just turn it on but otherwise I'm out of luck? Does the situation change if I am building an application from scratch that wants to deal with network disconnections? On the other hand, it appears to me that it proposes an entirely new programming model - did you consider an incremental approach that modifies window.applicationCache? Did you have particular reasons for rejecting that? Yes. I certainly considered window.applicationCache. I have previously proposed including additional things in HTML5, but the feedback was that the application cache could not be broken down in to any further primitives due to bootstrapping requirements [3]. I certainly feel that applicationCache can be composed with the primitives of DataCache + versioning. On the other hand, the converse is not possible due to ApplicationCache's: a. strictly static manifest serving, b. lack of support for version-free resources, and c. missing authorization checks. [1] http://portal.acm.org/citation.cfm?id=992378 [2] http://o-micron.blogspot.com/2009/04/how-is-bitsy-different-from- html-dojo.html [3] http://lists.w3.org/Archives/Public/public-html/2008Nov/0224.html I understand that your DataCache proposal provides different functionality to AppCache. I wasn't suggesting that they were equivalent. However, the AppCache as specified is getting implemented today (it is in Safari and Firefox with other implementations on the way). The problem I have is that I can't go implement AppCache and then add on some new methods and alternative interception-path to get DataCache-like features - I have to write a completely independent mechanism. If I propose that we build AppCache and then DataCache, someone will say, Hang on, didn't we just build this - why do we need two? While it might not be the perfect solution (we know the web far from ideal and is a lot of compromise), this type of proposal would be a lot more compelling to me if I could say This is what we have to add, this is how, and here are the use cases that make it valuable with a roadmap for extending what people are already building instead of something brand new. Cheers, Adrian.
Re: DataCache API - editor's draft available
On Thu, Jul 16, 2009 at 4:17 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote:
On Jul 16, 2009, at 3:54 PM, Jonas Sicking wrote:
I do understand how Interceptor/DataCache works. And understand that
it's seamless and can (based on a decision made by the browser)
seamlessly intercept both XHR requests and img src=.. requests.
What is not entirely clear to me is how you envision that authors will use
it.
Authors will use it in all three kinds of use cases I explained earlier -
XHR, form, and hyperlinked data. DataCache is designed to support all three.
The intention is to accommodate any use case arising from the issuance of a
network request when the device is off-line.
I.e. are you expecting authors to want to intercept XHR requests?
For data driven applications, I expect this usage scenario.
Or
img src=... requests? Or script src=... requests?
script src=... should be adequately supported by HTML5 (ApplicationCache),
unless the src value is dynamic in nature. For the static case, I don't
expect to see much usage of DataCache.
Ok that answers my question.
See, what I don't fully understand is what the use cases are. For
static resources like static images, scripts, plugins etc, it seems to
me that the HTML5 ApplicationCache should work, as you point out.
For dynamic images, I think canvas is what you'd want to use anyway
since generating encoded png images and serving it through the http
Interceptor is a whole lot more complicated.
For loading data using XHR, or submitting data using a form, it seems
like if the web application is abstracted at a higher level, then http
interception isn't strictly needed. If you currently have code like:
function readData(callback) {
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (xhr.readyState == 4 xhr.status == 200)
callback(xhr.responseText);
}
xhr.open(...);
xhr.send();
}
Then in order to support offline you could do:
function readData(callback) {
xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (xhr.readyState == 4 xhr.status == 200)
callback(xhr.responseText);
}
xhr.onerror = function() {
text = readDataFromLocalStore();
callback(text);
}
xhr.open(...);
xhr.send();
}
So I'm not sure what new use cases we are solving.
What I do agree this new API does is that it allows a *different* way
to deal with offline. One that could be argued to be simpler (I
haven't looked into it enough to have an opinion on if it's simpler or
not).
So rather than solving new use cases, it seems like this is providing
another way to solve them.
Is that correct?
/ Jonas
Re: DataCache API - editor's draft available
Hi Nikunj, So one of the things I've never fully understood with your proposal is what usage patterns people are going to want to use this new API with. Is the idea that people that use XMLHttpRequest to load data from the server will in offline mode want to intercept those XHR requests and respond with data stored in for example localStorage or some other type of clientside storage? Without having to change the code that uses the XHR object? Or is the idea that for things like img src=... or script src=... people will in offline mode want to intercept these requests and serve replies based on data stored in localStorage/elsewhere? Or both? Or something else? / Jonas On Thu, Jul 16, 2009 at 1:10 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: I have published the first draft of the DataCache API, which is based on Oracle's BITSY proposal [1]. Here's a link to the draft: http://dev.w3.org/2006/webapi/DataCache/ This document defines APIs for dynamically and statically serving off-line representations of HTTP resources. As the Chairs have concluded this to be within the scope of our current charter, I would request feedback from you so we can incorporate it before publication as a working draft. Nikunj http://o-micron.blogspot.com [1] http://www.oracle.com/technology/tech/feeds/spec/bitsy.html [2] http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/0098.html
Re: DataCache API - editor's draft available
On Jul 16, 2009, at 2:43 PM, Jonas Sicking wrote: Hi Nikunj, So one of the things I've never fully understood with your proposal is what usage patterns people are going to want to use this new API with. Thanks for asking. Please ask me again if this response does not adequately address your needs. Is the idea that people that use XMLHttpRequest to load data from the server will in offline mode want to intercept those XHR requests and respond with data stored in for example localStorage or some other type of clientside storage? Without having to change the code that uses the XHR object? The client would continue to make the same calls during periods of network disruptions - this is why DataCache is transparent. However, instead of the response coming from the server, it would come from the DataCache. Whether to switch to using DataCache, or not, would be the browser's call - this is why DataCache is seamless. The browser can choose to switch depending on a variety of environmental or system conditions. In case where the client access data via XHR, the DataCache can produce either a static or a dynamic response. The static response does not involve any client side storage API - it is generated from the internal storage of DataCache as was populated when a resource was captured. The dynamic response is produced by some JavaScript code identified as the interceptor and it uses any state information accessible to the interceptor, including localStorage (or B-tree storage) and/or DataCache static responses. Or is the idea that for things like img src=... or script src=... people will in offline mode want to intercept these requests and serve replies based on data stored in localStorage/elsewhere? In the case where client access data via resource hyperlinks, e.g., a href=..., img src=... , object data=... or video src=..., the data could once again be serviced using static DataCache responses. Dynamic responses are also possible but less likely. In the case where client accesses data via form posts, through things such as form target=... either programmatically (i.e., form.submit) or not, the data could be servied using dynamic DataCache responses produced by an interceptor using the DataCache static responses and/or localStorage (or B-tree storage). All three cases would be equally well supported by DataCache API. The above processing approaches are the reason why DataCache and HTTP Interceptor are a single API and not two separate ones. Or both? Or something else? / Jonas On Thu, Jul 16, 2009 at 1:10 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: I have published the first draft of the DataCache API, which is based on Oracle's BITSY proposal [1]. Here's a link to the draft: http://dev.w3.org/2006/webapi/DataCache/ This document defines APIs for dynamically and statically serving off-line representations of HTTP resources. As the Chairs have concluded this to be within the scope of our current charter, I would request feedback from you so we can incorporate it before publication as a working draft. Nikunj http://o-micron.blogspot.com [1] http://www.oracle.com/technology/tech/feeds/spec/bitsy.html [2] http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/0098.html
RE: DataCache API - editor's draft available
On Thursday, July 16, 2009 2:43 PM, Jonas Sicking wrote: Hi Nikunj, So one of the things I've never fully understood with your proposal is what usage patterns people are going to want to use this new API with. [snip] / Jonas On Thu, Jul 16, 2009 at 1:10 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: I have published the first draft of the DataCache API, which is based on Oracle's BITSY proposal [1]. Here's a link to the draft: http://dev.w3.org/2006/webapi/DataCache/ Nikunj, thanks for putting together the document and publishing it. I agree with Jonas and I'd like to understand the expected use cases better too. I think I get the point that making the network access seamless regardless of whether there is network connectivity or not simplifies the network request code but seems to me to require more complex interception code. This isn't a pattern that I can readily map to customer requests we've received nor is it a familiar pattern to me personally. The other concern that I have is that this seems like an extension to the AppCache support in HTML 5. The new part is the ability to denote certain end-points that respond dynamically via script including supporting verbs other than GET. On the other hand, it appears to me that it proposes an entirely new programming model - did you consider an incremental approach that modifies window.applicationCache? Did you have particular reasons for rejecting that? Cheers, Adrian.
Re: DataCache API - editor's draft available
On Thu, Jul 16, 2009 at 3:13 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: On Jul 16, 2009, at 2:43 PM, Jonas Sicking wrote: Hi Nikunj, So one of the things I've never fully understood with your proposal is what usage patterns people are going to want to use this new API with. Thanks for asking. Please ask me again if this response does not adequately address your needs. Hi Nikunj, Starting over since I think you misunderstood my question. I do understand how Interceptor/DataCache works. And understand that it's seamless and can (based on a decision made by the browser) seamlessly intercept both XHR requests and img src=.. requests. What is not entirely clear to me is how you envision that authors will use it. I.e. are you expecting authors to want to intercept XHR requests? Or img src=... requests? Or script src=... requests? I understand that all of this is possible, but my question is what you expect people to do. I hope that makes sense? / Jonas
Re: DataCache API - editor's draft available
On Jul 16, 2009, at 3:54 PM, Jonas Sicking wrote: On Thu, Jul 16, 2009 at 3:13 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: On Jul 16, 2009, at 2:43 PM, Jonas Sicking wrote: Hi Nikunj, So one of the things I've never fully understood with your proposal is what usage patterns people are going to want to use this new API with. Thanks for asking. Please ask me again if this response does not adequately address your needs. Hi Nikunj, Starting over since I think you misunderstood my question. I do understand how Interceptor/DataCache works. And understand that it's seamless and can (based on a decision made by the browser) seamlessly intercept both XHR requests and img src=.. requests. What is not entirely clear to me is how you envision that authors will use it. Authors will use it in all three kinds of use cases I explained earlier - XHR, form, and hyperlinked data. DataCache is designed to support all three. The intention is to accommodate any use case arising from the issuance of a network request when the device is off- line. I.e. are you expecting authors to want to intercept XHR requests? For data driven applications, I expect this usage scenario. Or img src=... requests? Or script src=... requests? script src=... should be adequately supported by HTML5 (ApplicationCache), unless the src value is dynamic in nature. For the static case, I don't expect to see much usage of DataCache. I understand that all of this is possible, but my question is what you expect people to do. I hope that makes sense? / Jonas
Re: DataCache API - editor's draft available
Hi Adrian, I am glad to explain the use cases further as needed. I addressed Jonas' questions in separate messages, so I will focus here solely on your questions. Please see responses in-line. Nikunj http://o-micron.blogspot.com On Jul 16, 2009, at 3:31 PM, Adrian Bateman wrote: On Thursday, July 16, 2009 2:43 PM, Jonas Sicking wrote: Hi Nikunj, So one of the things I've never fully understood with your proposal is what usage patterns people are going to want to use this new API with. [snip] I agree with Jonas and I'd like to understand the expected use cases better too. I think I get the point that making the network access seamless regardless of whether there is network connectivity or not simplifies the network request code but seems to me to require more complex interception code. There is a tradeoff when adding the complexity of unreliable networks - either more complex client application code, or more complex client interception code. In our experience, we have found the latter to be actually more manageable for cases where the interception and off-line serving is done in a relatively application-independent manner. This isn't a pattern that I can readily map to customer requests we've received nor is it a familiar pattern to me personally. DataCache's interception pattern has existed for a while. A good survey of mobile data management and transaction support has covered this pattern [1]. The discussion there is not limited to Web browsers, though. Due to poor network coverage in the past, this pattern was not very valuable. However, as networks improve yet remain unpredictable, this pattern becomes more relevant. For example, if the network is 95% unavailable for 90% of the time (e.g., when one is out-of-office), then it makes sense to have a fully disconnected architecture. However, if the network is only 25% unavailable for 25% of the time, then it is better to have an architecture that fully utilizes server resources whenever possible and downshifts to an off-line scenario when needed. This is the pattern used in DataCache. The other concern that I have is that this seems like an extension to the AppCache support in HTML 5. The new part is the ability to denote certain end-points that respond dynamically via script including supporting verbs other than GET. Besides the verb difference, there are others that I have previously stated on this ML and blogged about [2]. I will summarize for your benefit. * ApplicationCache does not allow programmatic inclusion of items (dynamic entries were removed some time ago). all data capture in DataCache is through an API, i.e., as a dynamic entry. • ApplicationCache does not secure one user's private resources from another; DataCache requires the presence of a specified cookie • ApplicationCache uses its own data format for identifying items for local storage and exludes any other formats such as JSON and Atom; DataCache does not have any format limitations * ApplicationCache operates per its own refresh protocol and that excludes a different protocol, especially one that does not require all or nothing semantics for data versioning; DataCache has no protocol limitations. On the other hand, it appears to me that it proposes an entirely new programming model - did you consider an incremental approach that modifies window.applicationCache? Did you have particular reasons for rejecting that? Yes. I certainly considered window.applicationCache. I have previously proposed including additional things in HTML5, but the feedback was that the application cache could not be broken down in to any further primitives due to bootstrapping requirements [3]. I certainly feel that applicationCache can be composed with the primitives of DataCache + versioning. On the other hand, the converse is not possible due to ApplicationCache's: a. strictly static manifest serving, b. lack of support for version-free resources, and c. missing authorization checks. Cheers, Adrian. [1] http://portal.acm.org/citation.cfm?id=992378 [2] http://o-micron.blogspot.com/2009/04/how-is-bitsy-different-from-html-dojo.html [3] http://lists.w3.org/Archives/Public/public-html/2008Nov/0224.html
