Re: SearchBox API
I'm not sure there's any particular coupling between the syntax of the API (IDL v. postMessage protocol) and how the API is implemented by the user agent. We could implement either syntax using either native or JavaScript code. For consistency with the rest of the platform, IDL seems like a better choice at the moment. Adam On Thu, Mar 31, 2011 at 7:39 AM, Sean Eagan wrote: > The reason I suggested web messaging is that more and more browser UI > is being built on top of the web platform with things like chromeless > [1] and chrome's WebUI [2], and this will likely include search boxes > at some point. This would give the search box a natural endpoint > browsing context for web messaging based communication with web apps. > > Now that I think about it, few if any changes would be needed to the > Search Box specification to support this, as the SearchBox interface > could be implemented as a javascript library that uses web messaging > behind the scenes. The only way to observe the difference between > this and a host object implementation would be by overriding > window.postMessage which doesn't seem to be a problem. > > My main observation would be that javascript implementations using web > messaging should be considered as potential alternatives to host > objects in specifications involving communication between browser UI > and web apps. > > [1] https://mozillalabs.com/chromeless/ > [2] http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/webui/ > > On Mon, Mar 28, 2011 at 1:59 PM, Adam Barth wrote: >> Sure, that syntax would work, but we'd still need to specify the >> postMessage protocol, just like we need to specify the IDL interface. >> It's not clear what moving to postMessage would buy us. >> >> Adam >> >> >> On Mon, Mar 28, 2011 at 11:20 AM, Sean Eagan wrote: >>> Why not just build this on top of web messaging[1] by having a >>> browsing context associated with the search box (or entire browser >>> chrome) that can communicate with a SERP or any page that wants to >>> accept input from a search box or otherwise communicate directly with >>> a user agent? >>> >>> [1] http://dev.w3.org/html5/postmsg/ >>> >>> On Mon, Mar 21, 2011 at 1:38 PM, Adam Barth wrote: On Mon, Mar 21, 2011 at 11:16 AM, Edward Lee wrote: >> enables "instant" style interaction between the user agent's search > Assuming the user agent automatically loads a url that is triggered by > a user key stroke, e.g., typing "g" results in > "http://www.google.com/";, the instant-style interaction is almost > there already for certain urls. These instant-style urls would include > what the user typed -- perhaps as a query parameter. > > For example, a user agent might request these pages as the user types: > http://www.google.com/search?q=g > http://www.google.com/search?q=go > http://www.google.com/search?q=goo > > Here, the results page shows the new query and updated results on > every user keystroke. > > These instant-style urls can also avoid refetching and rerendering the > whole page if the user's input shows up in the #fragment and the page > detects onHashChange. That's true, but you can only transmit one event that way. In this design, you've chosen to map the "change" event to hashchange. How should the user agent communicate that the user is done typing (i.e., the "submit" event, which triggers when presses the enter key)? Similarly, the communication in that approach is unidirectional, which means the page can't suggest search completions. Adam >>> >>> >>> >>> -- >>> Sean Eagan >>> >> > > > > -- > Sean Eagan >
[Bug 12310] We should add IDBCursor.advance(number of items to move forward)
http://www.w3.org/Bugs/Public/show_bug.cgi?id=12310 Jeremy Orlow changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED -- Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug.
[Bug 12233] We should add a method to compare two IndxedDB keys
http://www.w3.org/Bugs/Public/show_bug.cgi?id=12233 Jeremy Orlow changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED -- Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
> Currently there are no APIs in JavaScript to compare strings using
specific collations
We dont actually need this, just a mapping from UTF-16 string to a
sort-score (binary blob).
Its true that downloading the collation tables might take time, so we could
just provide:
var blob = string_to_score('utf-16 string', 'en-US');
as a built in function to make this efficient.
I agree with the other points though.
Cheers,
Keean.
On 31 March 2011 22:38, Pablo Castro wrote:
>
> From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy
> Orlow
> Sent: Thursday, March 31, 2011 11:36 AM
>
> >> I can find a lot of stuff on collation, but not a lot about why it could
> not be done in a library. Could you summerise the reasons why this needs to
> be core functionality for me?
> >>
> >> Sorry, but that stuff is paged out of my brain. Pablo, can you?
> >>
> >> A library could chose to use an object store as meta-data to store the
> collation orders that it is using for various indexes for example.
>
> - Currently there are no APIs in JavaScript to compare strings using
> specific collations. There are folks that are looking into this, but it will
> need time.
> - I'm far from an expert in the topic, but from talking to folks that
> understand this well it seems that to actually implement this entirely in
> JavaScript it would mean you have to download collation tables and apply
> them as needed in callbacks. Not only this means a hit in download size/time
> for the app but also that callbacks have to either download stuff or inline
> collation rules/tables in the callback itself.
> - In pure practical terms, I suspect the 80% scenario can be covered by
> implementing this natively, having it be fast and simple to use for common
> cases. Not pushing back on the callback stuff, just saying that I find it
> valuable to have users simply say "en-US" and get what they wanted.
> - Also from the practical perspective, simple cases that don't require the
> flexibility and can avoid having to take care of making the callbacks
> perfectly consistent even as you roll out updates that may hit only some of
> the pages, use components written by someone else, etc.
> - By default we would still do binary collation (there was a question in
> the thread, I forget exactly where).
>
> Thanks
> -pablo
>
>
[Bug 11269] Evaluating keyPaths needs to be better specified
http://www.w3.org/Bugs/Public/show_bug.cgi?id=11269 Jeremy Orlow changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED -- Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug.
RE: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow Sent: Thursday, March 31, 2011 11:36 AM >> I can find a lot of stuff on collation, but not a lot about why it could not >> be done in a library. Could you summerise the reasons why this needs to be >> core functionality for me? >> >> Sorry, but that stuff is paged out of my brain. Pablo, can you? >> >> A library could chose to use an object store as meta-data to store the >> collation orders that it is using for various indexes for example. - Currently there are no APIs in JavaScript to compare strings using specific collations. There are folks that are looking into this, but it will need time. - I'm far from an expert in the topic, but from talking to folks that understand this well it seems that to actually implement this entirely in JavaScript it would mean you have to download collation tables and apply them as needed in callbacks. Not only this means a hit in download size/time for the app but also that callbacks have to either download stuff or inline collation rules/tables in the callback itself. - In pure practical terms, I suspect the 80% scenario can be covered by implementing this natively, having it be fast and simple to use for common cases. Not pushing back on the callback stuff, just saying that I find it valuable to have users simply say "en-US" and get what they wanted. - Also from the practical perspective, simple cases that don't require the flexibility and can avoid having to take care of making the callbacks perfectly consistent even as you roll out updates that may hit only some of the pages, use components written by someone else, etc. - By default we would still do binary collation (there was a question in the thread, I forget exactly where). Thanks -pablo
Re: Offline Web Applications status
Hi again, I have in mind several extensions to the ApplicationCache that I think could address some of the additional desirements from the web developement community. I'll post them here because people seem to be more willing to have a discussion on the topic here than over in whatwg. 1. Allow cross-origin HTTPS resources to be included in manifest files. This much is actually done already in chromium impl as described on the whatwg list. 2. Allow a syntax to associate a page with an application cache, but does not add that page to the cache. A common feature request also mentioned on the whatwg list, but it's not getting any engagement from other browser vendors or the spec writer (which is kind of frustrating). The premise is to allow pages vended from a server to take advantage of the resources in an application cache when loading subresources. A perfectly reasonable request, . 3. Introduce a new manifest file section to INTERCEPT requests into a prefix matched url namespace and satisfy them with a cached resource. The resulting page would be free to interpret the location url and act accordingly based on the path and query elements beyond the prefix matched url string. This section would be similar to the FALLBACK section in that prefix matching is involved, but different in that instead of being used only in the case of a network/server error, the cached INTERCEPT resource would be used immediately w/o first going to the server. INTERCEPT: urlprefix redirect newlocationurl urlprefix return cachedresourceurl Here's where the INTERCEPT namespace could fit into the changes to the network model. if (url is EXPLICITLY_CACHED) // exact match return cached_response; if (url is in NETWORK namespace) // prefix match return network_response_as_usual; if (url is in INTERCEPT namespace) // prefix match < this is the new section return handle_intercepted_request_accordingly if (url is in FALLBACK namespace) // prefix match return repsonse_as_usual_unless_fallback_conditions_are_met_by_that_response; otherwise return synthesized_error_response; 4. Allow an INTERCEPT cached resources to be "executable". Instead of simply returning the cached resource in response to the request, load it into a background worker context (if not already loaded) and invoke a function in that context to asynchronously compute response headers and body based on the request headers (including cookie) and body. The background worker would have access to various local storage facilities (fileSystem, indexed/sqlDBs) as well as the ability to make network requests via XHR. INTERCEPT: urlprefix execute cachedexecutableresourceurl 5. Create a syntax to allow FALLBACK resources to be similarly executable in a background worker context. As an author of gears and chromium's appcache, I've been interested in pursuing something along these lines for a while. Chrome is finally getting built up to the point where this could be done, the file system and better support for many worker threads are prerequisite. With the outline above in mind, this is why earlier in the thread i questioned the need for an explicit add(url) and remove(url) sort of api. That capability can be composed out of the file system and an executable intercept handler. And there is also window.createObjectUrl() and FileEntry.toUrl() which could cover many use cases that are behind the feature request for add() and remove(). -Michael On Sat, Mar 26, 2011 at 8:32 PM, Jack Coulter wrote: > IndexedDB would be more suited to what you're doing Nathan, I've always > seen > ApplicationCache as something to only use on the core HTML/JS/CSS and > perhaps > small images, like icons (none of this would change often, and would > generally > be rather small) whereas IndexedDB sounds more like what you did with > gears. > > Or, there's always the File System API, but I don't think it's widely > implemented yet. > > Excerpts from Nathan Kitchen's message of Sun Mar 27 08:46:35 +1100 2011: > > A couple of other app cache observations from a hobbyist who's played > around > > with Google's Gears... > > > > I built an offline web application based on Gears, with the intention to > > migrate to something a bit more standardized as it became available. That > > was a good two years ago now, but the existing and proposed > implementations > > still don't offer the capability that I can get from Gears. > > > > If you're in Chrome, point your browser at: http://my.offlinebible.com/. > My > > observations are going to be related to this app. I apologise if they're > > quite specific, but they present a snapshot of some real-world issues > > encountered while developing an offline app. > > > >- *App Cache* > >The first task that the app undertakes is downloading a large amount > of > >offline data. This data includes the usual web page resources, plus a > >massive amount of data destined for the Gears/WebS
Re: [FileAPI] Result of calling MultipleReads on FileReader
On Thu, Mar 31, 2011 at 2:55 PM, Adrian Bateman wrote: > On Thursday, March 31, 2011 10:19 AM, Arun Ranganathan wrote: >> On 3/30/11 2:01 PM, Eric Uhrhane wrote: >> > On Mon, Mar 28, 2011 at 5:37 PM, Adrian Bateman >> wrote: >> >> Is there a reason for the current spec text? >> > I don't know the original rationale, but in the absence of any strong >> > technical constraint, I'd much prefer that subsequent read calls just >> > throw an exception immediately. They seem likely to be indicative of >> > bad code anyway, and it's so much easier to debug when you find out >> > right away. >> > >> >> The original rationale was to do what XHR does w.r.t. open()! >> Essentially, the goal was: >> >> 1. To abort previous reads in favor of the last one, like how XHR does. >> 2. The last read goes through. >> >> So: Adrian/Eric -- do you object to keeping this like XHR in terms of >> aborting previous reads? What I should *definitely* do is make spec. >> text more robust to reflect this; in general I want to make asynchronous >> parts of this spec. more like HTML5 here. I think it's cleaner and simpler just to throw. FileReader and XHR are already different enough that a bit more, as long as it's a usability improvement, isn't a big deal. The efficiency improvement is just a bonus. > As long as the spec is clear I don't mind whether a subsequent read throws > or includes an implicit abort() call. That's not the way I read the current > spec text. We need to know things like whether we should queue any events > for the abort or just silently move on to the new read, etc. It would be > good if we can be explicit about when you're allowed to call readAs > (this way it sounds like always) and if anything is different between the > LOADING and DONE states. Agreed on the need for this to be very explicit. But I think we should skip all the extra queued events by just making it illegal. > Thanks, > > Adrian. >
RE: [FileAPI] Result of calling MultipleReads on FileReader
On Thursday, March 31, 2011 10:19 AM, Arun Ranganathan wrote: > On 3/30/11 2:01 PM, Eric Uhrhane wrote: > > On Mon, Mar 28, 2011 at 5:37 PM, Adrian Bateman > wrote: > >> Is there a reason for the current spec text? > > I don't know the original rationale, but in the absence of any strong > > technical constraint, I'd much prefer that subsequent read calls just > > throw an exception immediately. They seem likely to be indicative of > > bad code anyway, and it's so much easier to debug when you find out > > right away. > > > > The original rationale was to do what XHR does w.r.t. open()! > Essentially, the goal was: > > 1. To abort previous reads in favor of the last one, like how XHR does. > 2. The last read goes through. > > So: Adrian/Eric -- do you object to keeping this like XHR in terms of > aborting previous reads? What I should *definitely* do is make spec. > text more robust to reflect this; in general I want to make asynchronous > parts of this spec. more like HTML5 here. As long as the spec is clear I don't mind whether a subsequent read throws or includes an implicit abort() call. That's not the way I read the current spec text. We need to know things like whether we should queue any events for the abort or just silently move on to the new read, etc. It would be good if we can be explicit about when you're allowed to call readAs (this way it sounds like always) and if anything is different between the LOADING and DONE states. Thanks, Adrian.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 10:07 PM, Shawn Wilsher wrote: > On 3/31/2011 11:47 AM, Joran Greef wrote: > Let those who introduced these design flaws be among the first to take > responsibility and fix them. > You aren't being constructive, and that's a surefire way to be ignored. You > have yet to convince the working group that these are "design flaws" in the > first place. > > /sdwilsh Agreed. I am actively using the API with real-world data and I am providing feedback. You are welcome to use it or not. It is not for me to convince anyone. As I said, if people think there is a problem, let those who introduced it fix it. Joran Greef
Re: Mail List Etiquette [Was: WebSQL] Any future plans, or has IndexedDB replaced WebSQL?]
Thank you Art. To clarify, I have heard from a contributor to the specification in question who referred to LocalStorage himself as "little more than a toy", expressing his frustrations at the specification. It is well known that most LocalStorage implementations do not support more than 10mb, some load the entire contents into memory synchronously on first access, and there were some issues around locking that were not addressed as far as I recall. LocalStorage does not work as advertised. Many developers, including myself, got excited, spent hours with it, only to see these issues left unresolved. It would be true to say that most LocalStorage implementations are "crippled" in this sense. No one need be offended since specification and implementation are two separate things. I do wish however, that the specification would have addressed large quota support, and encouraged certain implementation practices, and in this sense I feel that not enough was done. The same with WebSQL. And recently I learned that IDB prevents applications from managing indices? These things are disappointing to us developers. I think we have a right to be critical on these issues where criticism is due. If the specification is inadequate, or burdened by politics, we should be free to say so (respectfully and professionally of course, but also honestly and directly and with the right measure of urgnency), without fear of offending anyone or being policed for it. Joran Greef On 31 Mar 2011, at 9:37 PM, Arthur Barstow wrote: >> This is painful to read. WebSQL development died because SQLite, the most >> widely-deployed database software in the world, was too good? That sounds >> like a catastrophic failure of the W3C process. >> >> -- >> Glenn Maynard > Hear. > > I am starting to think that Mozilla will step up and provide an embedding of > SQLite, even if it has to only think of it as such. It will have to. > > People would rather use a working database than something crippled albeit > "specced" (see LocalStorage or IndexedDB). > > It was things like XHR in all their unspecced glory that brought the web to > where it is today. Joran - as one of the moderators of public-webapps, I find your comments above offensive to those that work on the specs you mention. All - this is a reminder that all e-mails on this list are expected to be respectful and professional. Please see the following for more information about the etiquette and usage of this list: http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/1216.html -Regards, Art Barstow
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 3/31/2011 11:47 AM, Joran Greef wrote: Let those who introduced these design flaws be among the first to take responsibility and fix them. You aren't being constructive, and that's a surefire way to be ignored. You have yet to convince the working group that these are "design flaws" in the first place. /sdwilsh smime.p7s Description: S/MIME Cryptographic Signature
Mail List Etiquette [Was: WebSQL] Any future plans, or has IndexedDB replaced WebSQL?]
This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. -- Glenn Maynard Hear. I am starting to think that Mozilla will step up and provide an embedding of SQLite, even if it has to only think of it as such. It will have to. People would rather use a working database than something crippled albeit "specced" (see LocalStorage or IndexedDB). It was things like XHR in all their unspecced glory that brought the web to where it is today. Joran - as one of the moderators of public-webapps, I find your comments above offensive to those that work on the specs you mention. All - this is a reminder that all e-mails on this list are expected to be respectful and professional. Please see the following for more information about the etiquette and usage of this list: http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/1216.html -Regards, Art Barstow
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 31 March 2011 19:08, Joran Greef wrote: > > This is painful to read. WebSQL development died because SQLite, the > most widely-deployed database software in the world, was too good? That > sounds like a catastrophic failure of the W3C process. > > > > -- > > Glenn Maynard > > Hear. > > I am starting to think that Mozilla will step up and provide an embedding > of SQLite, even if it has to only think of it as such. It will have to. > > People would rather use a working database than something crippled albeit > "specced" (see LocalStorage or IndexedDB). > > It was things like XHR in all their unspecced glory that brought the web to > where it is today. > Do you want to take a look at my RelationalDB library - it could form the basis of a replacement for WebSQL, and as it is based on relational algebra not SQL, it has not user visible dependencies on the particular SQL implementation? Have a look at: https://github.com/keean/RelationalDB/blob/master/examples/candy.html For a usage example. This should run in chrome right now (using WebSQL as a backend). I would appreciate any thoughts, comments etc. Cheers, Keean.
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
> This is painful to read. WebSQL development died because SQLite, the most > widely-deployed database software in the world, was too good? That sounds > like a catastrophic failure of the W3C process. > > -- > Glenn Maynard Hear. I am starting to think that Mozilla will step up and provide an embedding of SQLite, even if it has to only think of it as such. It will have to. People would rather use a working database than something crippled albeit "specced" (see LocalStorage or IndexedDB). It was things like XHR in all their unspecced glory that brought the web to where it is today.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 March 2011 18:36, Jeremy Orlow wrote: > On Thu, Mar 31, 2011 at 11:24 AM, Keean Schupke wrote: > >> On 31 March 2011 18:17, Jeremy Orlow wrote: >> >>> On Thu, Mar 31, 2011 at 11:09 AM, Keean Schupke wrote: >>> On 31 March 2011 17:41, Jonas Sicking wrote: > On Thu, Mar 31, 2011 at 1:32 AM, Joran Greef > wrote: > > On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: > > > >> I previously have asked for a detailed proposal, but so far you have > >> not supplied one but instead keep referring to other unnamed > database > >> APIs. > > > > I have already provided an adequate interface proposal for putObject > and deleteObject. > > That is hardly a comprehensive proposal, but rather just one small part > of it. > I wanted to make a few comments about these points :- > > I do really think the idea of not having the implementation keep track > of the set of indexes for a objectStore is a really interesting one. > As is the idea of not even having a set set of objectStores. However, > there are several problems that needs to be solved. In particular how > do you deal with collations? > no indexes, no object stores... well I for one prefer the "validate_object_store", "validate_index" approach, in that it can hide statefullness if necessary (like I do with RelationalDB) whilst presenting a stateless API. It also keeps the size of the put statements down. > > I.e. we have concluded that there are important use cases which > require using different collations for different indexes and > objectStores. Even for different indexes attached to the same > objectStore. > > Additionally, if we're getting rid of setVersion, how do we expect > pages dealing with the (application managed) schema changing while the > page has a connection open to the database? > 1 - there is no schema 2 - dont allow it to change whilst the database is open In reality a schema is implicitly tied to a code version. In other words the source code of the application assumes a certain schema. If the assumed schema and the schema in the DB do not match things are going to go very wrong very quickly. Schema changes _always_ accompany code changes (otherwise they are not schema changes just data changes). As such they never happen when a DB is open. The way I handle this in RelationalDB, by validating the actual schema against the source-code schema in the db-open (actually the method is called validate), is probably the best way to handle this. If the database does not exist we create it according to the schema. If it exists we check it matches the schema. If there is a difference we see if we can 'upgrade' the database automatically (certain changes like adding a new column with a default value can be done automaticall), if we cannot automaticall upgrade, we exit with an error - as allowing the program to run will result in corruption of the data already in the database. At this point it is up to the application to figure out how to upgrade the database (by opening one database with an old schema and another with a new schema)... There is not point in ever allowing a database to be opened with the wrong schema. > So pretty please, with sugar on top, please come up with a proposal > for the full API rather than bits and pieces. > > And I should mention that I have as an absolute requirement that you > should be able to specify collation by simply saying that you want to > use "en-US" or "sv-SV" sorting. Using callbacks or other means is ok > *in addition to this*, but callback mechanisms tend to be a lot more > complex since they have to deal with the callback doing all sorts of > evil things such as returning inconsistent results (think "return > Math.random()"), or simply do evil things like navigate the current > page, deleting the database, or modifying the record that is in the > process of being stored. > The core API only needs to deal with sorting binary-blob sort orders. A library wrapper could provide all the collation ordering goodness that people want. For example RelationalDB will have to deal with sorting orders, it does not need the browser to provide that functionality. In fact browser provided functionality may limit what can be done in libraries on top. >>> >>> This is difficult if not impossible to do. See previous threads on the >>> matter. >>> >>> J >>> >> >> I can find a lot of stuff on collation, but not a lot about why it could >> not be done in a library. Could you summerise the reasons why this needs to >> be core functionality for me? >> > > Sorry, but that stuff is paged out of my brain.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 7:41 PM, Jonas Sicking wrote: > So pretty please, with sugar on top, please come up with a proposal > for the full API rather than bits and pieces. Let those who introduced these design flaws be among the first to take responsibility and fix them.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 7:27 PM, Jeremy Orlow wrote: >> 1. Provide the application with a first-class means to manage indexes at >> time of putting/deleting objects. > > I'm OK with doing this for v1 if the others are. It doesn't seem like that > big of an addition and it would give a decent amount of additional > flexibility. Thanks Jeremy that would be great. >> (reduces serialization/deserialization overhead where application already >> has the object as a string) > > I'm not sure why you think this would reduce overhead. How long would it take an iPad to JSON deserialize/serialize 500 / 5,000 / 50,000 / 500,000 / 5,000,000 2KB objects? That's a reasonable device and those are reasonable workloads. In it's present state, IndexedDB needs to do this every time setVersion is called with a createIndex in there... you see the problem is there's no way for the application to control this. The application would arguably be able to find better ways of migrating indexes than using key paths which necessitate deserialization/serialization to be performed on the client. For instance, you could use batch jobs on the server to do this on behalf of clients, and this would make sense especially where many clients/devices share the same objects. With IndexedDB this is not possible. With pure storage primitives it would have been possible. This is just one use-case, and for every one of these there will be plenty more. > Like I said above, although I think we should make it possible to operate > more statelessly, I don't see a reason we need to remove stuff like this. > Some users will find it more convenient to work this way. Agreed on both counts. It is clearly too late to remove it now. But it may be a good idea in future to keep the focus on providing low-level primitives rather than convenience features, since the latter often get in the way of the former.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 11:24 AM, Keean Schupke wrote: > On 31 March 2011 18:17, Jeremy Orlow wrote: > >> On Thu, Mar 31, 2011 at 11:09 AM, Keean Schupke wrote: >> >>> On 31 March 2011 17:41, Jonas Sicking wrote: >>> On Thu, Mar 31, 2011 at 1:32 AM, Joran Greef wrote: > On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: > >> I previously have asked for a detailed proposal, but so far you have >> not supplied one but instead keep referring to other unnamed database >> APIs. > > I have already provided an adequate interface proposal for putObject and deleteObject. That is hardly a comprehensive proposal, but rather just one small part of it. >>> >>> I wanted to make a few comments about these points :- >>> >>> I do really think the idea of not having the implementation keep track of the set of indexes for a objectStore is a really interesting one. As is the idea of not even having a set set of objectStores. However, there are several problems that needs to be solved. In particular how do you deal with collations? >>> >>> no indexes, no object stores... well I for one prefer the >>> "validate_object_store", "validate_index" approach, in that it can hide >>> statefullness if necessary (like I do with RelationalDB) whilst presenting a >>> stateless API. It also keeps the size of the put statements down. >>> >>> I.e. we have concluded that there are important use cases which require using different collations for different indexes and objectStores. Even for different indexes attached to the same objectStore. Additionally, if we're getting rid of setVersion, how do we expect pages dealing with the (application managed) schema changing while the page has a connection open to the database? >>> >>> 1 - there is no schema >>> 2 - dont allow it to change whilst the database is open >>> >>> In reality a schema is implicitly tied to a code version. In other words >>> the source code of the application assumes a certain schema. If the assumed >>> schema and the schema in the DB do not match things are going to go very >>> wrong very quickly. Schema changes _always_ accompany code changes >>> (otherwise they are not schema changes just data changes). As such they >>> never happen when a DB is open. The way I handle this in RelationalDB, by >>> validating the actual schema against the source-code schema in the db-open >>> (actually the method is called validate), is probably the best way to handle >>> this. If the database does not exist we create it according to the schema. >>> If it exists we check it matches the schema. If there is a difference we see >>> if we can 'upgrade' the database automatically (certain changes like adding >>> a new column with a default value can be done automaticall), if we cannot >>> automaticall upgrade, we exit with an error - as allowing the program to run >>> will result in corruption of the data already in the database. At this point >>> it is up to the application to figure out how to upgrade the database (by >>> opening one database with an old schema and another with a new schema)... >>> There is not point in ever allowing a database to be opened with the wrong >>> schema. >>> >>> So pretty please, with sugar on top, please come up with a proposal for the full API rather than bits and pieces. And I should mention that I have as an absolute requirement that you should be able to specify collation by simply saying that you want to use "en-US" or "sv-SV" sorting. Using callbacks or other means is ok *in addition to this*, but callback mechanisms tend to be a lot more complex since they have to deal with the callback doing all sorts of evil things such as returning inconsistent results (think "return Math.random()"), or simply do evil things like navigate the current page, deleting the database, or modifying the record that is in the process of being stored. >>> >>> The core API only needs to deal with sorting binary-blob sort orders. A >>> library wrapper could provide all the collation ordering goodness that >>> people want. For example RelationalDB will have to deal with sorting orders, >>> it does not need the browser to provide that functionality. In fact browser >>> provided functionality may limit what can be done in libraries on top. >>> >> >> This is difficult if not impossible to do. See previous threads on the >> matter. >> >> J >> > > I can find a lot of stuff on collation, but not a lot about why it could > not be done in a library. Could you summerise the reasons why this needs to > be core functionality for me? > Sorry, but that stuff is paged out of my brain. Pablo, can you? > A library could chose to use an object store as meta-data to store the > collation orders that it is using for various indexes for example. > > > Cheers, > Keean. > >
[Bug 9832] keyPath is underspecified
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9832 Jeremy Orlow changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution||DUPLICATE --- Comment #5 from Jeremy Orlow 2011-03-31 18:27:21 UTC --- *** This bug has been marked as a duplicate of bug 11269 *** -- Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 March 2011 18:17, Jeremy Orlow wrote: > On Thu, Mar 31, 2011 at 11:09 AM, Keean Schupke wrote: > >> On 31 March 2011 17:41, Jonas Sicking wrote: >> >>> On Thu, Mar 31, 2011 at 1:32 AM, Joran Greef wrote: >>> > On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: >>> > >>> >> I previously have asked for a detailed proposal, but so far you have >>> >> not supplied one but instead keep referring to other unnamed database >>> >> APIs. >>> > >>> > I have already provided an adequate interface proposal for putObject >>> and deleteObject. >>> >>> That is hardly a comprehensive proposal, but rather just one small part >>> of it. >>> >> >> I wanted to make a few comments about these points :- >> >> >>> >>> I do really think the idea of not having the implementation keep track >>> of the set of indexes for a objectStore is a really interesting one. >>> As is the idea of not even having a set set of objectStores. However, >>> there are several problems that needs to be solved. In particular how >>> do you deal with collations? >>> >> >> no indexes, no object stores... well I for one prefer the >> "validate_object_store", "validate_index" approach, in that it can hide >> statefullness if necessary (like I do with RelationalDB) whilst presenting a >> stateless API. It also keeps the size of the put statements down. >> >> >>> >>> I.e. we have concluded that there are important use cases which >>> require using different collations for different indexes and >>> objectStores. Even for different indexes attached to the same >>> objectStore. >>> >>> Additionally, if we're getting rid of setVersion, how do we expect >>> pages dealing with the (application managed) schema changing while the >>> page has a connection open to the database? >>> >> >> 1 - there is no schema >> 2 - dont allow it to change whilst the database is open >> >> In reality a schema is implicitly tied to a code version. In other words >> the source code of the application assumes a certain schema. If the assumed >> schema and the schema in the DB do not match things are going to go very >> wrong very quickly. Schema changes _always_ accompany code changes >> (otherwise they are not schema changes just data changes). As such they >> never happen when a DB is open. The way I handle this in RelationalDB, by >> validating the actual schema against the source-code schema in the db-open >> (actually the method is called validate), is probably the best way to handle >> this. If the database does not exist we create it according to the schema. >> If it exists we check it matches the schema. If there is a difference we see >> if we can 'upgrade' the database automatically (certain changes like adding >> a new column with a default value can be done automaticall), if we cannot >> automaticall upgrade, we exit with an error - as allowing the program to run >> will result in corruption of the data already in the database. At this point >> it is up to the application to figure out how to upgrade the database (by >> opening one database with an old schema and another with a new schema)... >> There is not point in ever allowing a database to be opened with the wrong >> schema. >> >> >>> So pretty please, with sugar on top, please come up with a proposal >>> for the full API rather than bits and pieces. >>> >>> And I should mention that I have as an absolute requirement that you >>> should be able to specify collation by simply saying that you want to >>> use "en-US" or "sv-SV" sorting. Using callbacks or other means is ok >>> *in addition to this*, but callback mechanisms tend to be a lot more >>> complex since they have to deal with the callback doing all sorts of >>> evil things such as returning inconsistent results (think "return >>> Math.random()"), or simply do evil things like navigate the current >>> page, deleting the database, or modifying the record that is in the >>> process of being stored. >>> >> >> The core API only needs to deal with sorting binary-blob sort orders. A >> library wrapper could provide all the collation ordering goodness that >> people want. For example RelationalDB will have to deal with sorting orders, >> it does not need the browser to provide that functionality. In fact browser >> provided functionality may limit what can be done in libraries on top. >> > > This is difficult if not impossible to do. See previous threads on the > matter. > > J > I can find a lot of stuff on collation, but not a lot about why it could not be done in a library. Could you summerise the reasons why this needs to be core functionality for me? A library could chose to use an object store as meta-data to store the collation orders that it is using for various indexes for example. Cheers, Keean.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 11:09 AM, Keean Schupke wrote: > On 31 March 2011 17:41, Jonas Sicking wrote: > >> On Thu, Mar 31, 2011 at 1:32 AM, Joran Greef wrote: >> > On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: >> > >> >> I previously have asked for a detailed proposal, but so far you have >> >> not supplied one but instead keep referring to other unnamed database >> >> APIs. >> > >> > I have already provided an adequate interface proposal for putObject and >> deleteObject. >> >> That is hardly a comprehensive proposal, but rather just one small part of >> it. >> > > I wanted to make a few comments about these points :- > > >> >> I do really think the idea of not having the implementation keep track >> of the set of indexes for a objectStore is a really interesting one. >> As is the idea of not even having a set set of objectStores. However, >> there are several problems that needs to be solved. In particular how >> do you deal with collations? >> > > no indexes, no object stores... well I for one prefer the > "validate_object_store", "validate_index" approach, in that it can hide > statefullness if necessary (like I do with RelationalDB) whilst presenting a > stateless API. It also keeps the size of the put statements down. > > >> >> I.e. we have concluded that there are important use cases which >> require using different collations for different indexes and >> objectStores. Even for different indexes attached to the same >> objectStore. >> >> Additionally, if we're getting rid of setVersion, how do we expect >> pages dealing with the (application managed) schema changing while the >> page has a connection open to the database? >> > > 1 - there is no schema > 2 - dont allow it to change whilst the database is open > > In reality a schema is implicitly tied to a code version. In other words > the source code of the application assumes a certain schema. If the assumed > schema and the schema in the DB do not match things are going to go very > wrong very quickly. Schema changes _always_ accompany code changes > (otherwise they are not schema changes just data changes). As such they > never happen when a DB is open. The way I handle this in RelationalDB, by > validating the actual schema against the source-code schema in the db-open > (actually the method is called validate), is probably the best way to handle > this. If the database does not exist we create it according to the schema. > If it exists we check it matches the schema. If there is a difference we see > if we can 'upgrade' the database automatically (certain changes like adding > a new column with a default value can be done automaticall), if we cannot > automaticall upgrade, we exit with an error - as allowing the program to run > will result in corruption of the data already in the database. At this point > it is up to the application to figure out how to upgrade the database (by > opening one database with an old schema and another with a new schema)... > There is not point in ever allowing a database to be opened with the wrong > schema. > > >> So pretty please, with sugar on top, please come up with a proposal >> for the full API rather than bits and pieces. >> >> And I should mention that I have as an absolute requirement that you >> should be able to specify collation by simply saying that you want to >> use "en-US" or "sv-SV" sorting. Using callbacks or other means is ok >> *in addition to this*, but callback mechanisms tend to be a lot more >> complex since they have to deal with the callback doing all sorts of >> evil things such as returning inconsistent results (think "return >> Math.random()"), or simply do evil things like navigate the current >> page, deleting the database, or modifying the record that is in the >> process of being stored. >> > > The core API only needs to deal with sorting binary-blob sort orders. A > library wrapper could provide all the collation ordering goodness that > people want. For example RelationalDB will have to deal with sorting orders, > it does not need the browser to provide that functionality. In fact browser > provided functionality may limit what can be done in libraries on top. > This is difficult if not impossible to do. See previous threads on the matter. J > > >> >> / Jonas >> >> > > Cheers, > Keean. > >
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 March 2011 17:41, Jonas Sicking wrote: > On Thu, Mar 31, 2011 at 1:32 AM, Joran Greef wrote: > > On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: > > > >> I previously have asked for a detailed proposal, but so far you have > >> not supplied one but instead keep referring to other unnamed database > >> APIs. > > > > I have already provided an adequate interface proposal for putObject and > deleteObject. > > That is hardly a comprehensive proposal, but rather just one small part of > it. > I wanted to make a few comments about these points :- > > I do really think the idea of not having the implementation keep track > of the set of indexes for a objectStore is a really interesting one. > As is the idea of not even having a set set of objectStores. However, > there are several problems that needs to be solved. In particular how > do you deal with collations? > no indexes, no object stores... well I for one prefer the "validate_object_store", "validate_index" approach, in that it can hide statefullness if necessary (like I do with RelationalDB) whilst presenting a stateless API. It also keeps the size of the put statements down. > > I.e. we have concluded that there are important use cases which > require using different collations for different indexes and > objectStores. Even for different indexes attached to the same > objectStore. > > Additionally, if we're getting rid of setVersion, how do we expect > pages dealing with the (application managed) schema changing while the > page has a connection open to the database? > 1 - there is no schema 2 - dont allow it to change whilst the database is open In reality a schema is implicitly tied to a code version. In other words the source code of the application assumes a certain schema. If the assumed schema and the schema in the DB do not match things are going to go very wrong very quickly. Schema changes _always_ accompany code changes (otherwise they are not schema changes just data changes). As such they never happen when a DB is open. The way I handle this in RelationalDB, by validating the actual schema against the source-code schema in the db-open (actually the method is called validate), is probably the best way to handle this. If the database does not exist we create it according to the schema. If it exists we check it matches the schema. If there is a difference we see if we can 'upgrade' the database automatically (certain changes like adding a new column with a default value can be done automaticall), if we cannot automaticall upgrade, we exit with an error - as allowing the program to run will result in corruption of the data already in the database. At this point it is up to the application to figure out how to upgrade the database (by opening one database with an old schema and another with a new schema)... There is not point in ever allowing a database to be opened with the wrong schema. > So pretty please, with sugar on top, please come up with a proposal > for the full API rather than bits and pieces. > > And I should mention that I have as an absolute requirement that you > should be able to specify collation by simply saying that you want to > use "en-US" or "sv-SV" sorting. Using callbacks or other means is ok > *in addition to this*, but callback mechanisms tend to be a lot more > complex since they have to deal with the callback doing all sorts of > evil things such as returning inconsistent results (think "return > Math.random()"), or simply do evil things like navigate the current > page, deleting the database, or modifying the record that is in the > process of being stored. > The core API only needs to deal with sorting binary-blob sort orders. A library wrapper could provide all the collation ordering goodness that people want. For example RelationalDB will have to deal with sorting orders, it does not need the browser to provide that functionality. In fact browser provided functionality may limit what can be done in libraries on top. > > / Jonas > > Cheers, Keean.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 1:32 AM, Joran Greef wrote: > On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: > >> I previously have asked for a detailed proposal, but so far you have >> not supplied one but instead keep referring to other unnamed database >> APIs. > > I have already provided an adequate interface proposal for putObject and > deleteObject. That is hardly a comprehensive proposal, but rather just one small part of it. I do really think the idea of not having the implementation keep track of the set of indexes for a objectStore is a really interesting one. As is the idea of not even having a set set of objectStores. However, there are several problems that needs to be solved. In particular how do you deal with collations? I.e. we have concluded that there are important use cases which require using different collations for different indexes and objectStores. Even for different indexes attached to the same objectStore. Additionally, if we're getting rid of setVersion, how do we expect pages dealing with the (application managed) schema changing while the page has a connection open to the database? So pretty please, with sugar on top, please come up with a proposal for the full API rather than bits and pieces. And I should mention that I have as an absolute requirement that you should be able to specify collation by simply saying that you want to use "en-US" or "sv-SV" sorting. Using callbacks or other means is ok *in addition to this*, but callback mechanisms tend to be a lot more complex since they have to deal with the callback doing all sorts of evil things such as returning inconsistent results (think "return Math.random()"), or simply do evil things like navigate the current page, deleting the database, or modifying the record that is in the process of being stored. / Jonas
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 March 2011 17:27, Jeremy Orlow wrote: > On Thu, Mar 31, 2011 at 5:41 AM, Joran Greef wrote: > >> On 31 Mar 2011, at 12:52 PM, Keean Schupke wrote: >> >> > I totally agree with everything so far... >> > >> >> 3. This requires an adjustment to the putObject and deleteObject >> interfaces (see previous threads). >> > >> > I disagree that a simple API change is the answer. The problem is >> architectural, not just a superficial API issue. >> >> Yes, for IndexedDB to be stateless with respect to application schema, one >> would need to: >> >> 1. Provide the application with a first-class means to manage indexes at >> time of putting/deleting objects. >> > > I'm OK with doing this for v1 if the others are. It doesn't seem like that > big of an addition and it would give a decent amount of additional > flexibility. > > >> 2. Treat objects as opaque (remove key path, > > > Key paths are quite useful. I agree that making it possible to use > statelessly is good, but I don't see any reason why making it 100% stateless > should be a goal. > > >> structured clone mechanisms > > > For sure, not going to happen. > > >> , application must provide an id and JSON value to put/delete calls, >> reduces serialization/deserialization overhead where application already has >> the object as a string). >> > > I'm not sure why you think this would reduce overhead. > > >> 3. Remove setVersion (redundant, application migrates objects and indexes >> using transactions as it needs to). >> 4. Remove createIndex. >> > > Like I said above, although I think we should make it possible to operate > more statelessly, I don't see a reason we need to remove stuff like this. > Some users will find it more convenient to work this way. > > J > Surely intent should be only to do what is necessary in the browser, and to do the rest in a library. So for me convenience functionality should be provided by libraries. Only that which has to be done in the browser should be done in the browser? Cheers, Keean.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 5:41 AM, Joran Greef wrote: > On 31 Mar 2011, at 12:52 PM, Keean Schupke wrote: > > > I totally agree with everything so far... > > > >> 3. This requires an adjustment to the putObject and deleteObject > interfaces (see previous threads). > > > > I disagree that a simple API change is the answer. The problem is > architectural, not just a superficial API issue. > > Yes, for IndexedDB to be stateless with respect to application schema, one > would need to: > > 1. Provide the application with a first-class means to manage indexes at > time of putting/deleting objects. > I'm OK with doing this for v1 if the others are. It doesn't seem like that big of an addition and it would give a decent amount of additional flexibility. > 2. Treat objects as opaque (remove key path, Key paths are quite useful. I agree that making it possible to use statelessly is good, but I don't see any reason why making it 100% stateless should be a goal. > structured clone mechanisms For sure, not going to happen. > , application must provide an id and JSON value to put/delete calls, > reduces serialization/deserialization overhead where application already has > the object as a string). > I'm not sure why you think this would reduce overhead. > 3. Remove setVersion (redundant, application migrates objects and indexes > using transactions as it needs to). > 4. Remove createIndex. > Like I said above, although I think we should make it possible to operate more statelessly, I don't see a reason we need to remove stuff like this. Some users will find it more convenient to work this way. J
Re: [FileAPI] Result of calling MultipleReads on FileReader
Eric, Adrian, On 3/30/11 2:01 PM, Eric Uhrhane wrote: On Mon, Mar 28, 2011 at 5:37 PM, Adrian Bateman wrote: As we continue to experiment with the File API, I'm trying to understand the rationale for the Multiple Reads section: http://dev.w3.org/2006/webapi/FileAPI/#MultipleReads The spec says: If multiple read methods are called on the same FileReader object, user agents MUST only process the last call to a read method, which is the call that occurs last in a script block that has the "already started" flag set [HTML5]. I'm trying to understand the rationale for respecting the LAST call - is it common for people to call read lots of times and want the last one to be respected. Since the read happens asynchronously, we'd rather kick off the read operation as soon as the first read is called and give an error to subsequent read calls. I'm not sure what the use case is for wanting the last one (you can always call abort() and start again). Is there a reason for the current spec text? I don't know the original rationale, but in the absence of any strong technical constraint, I'd much prefer that subsequent read calls just throw an exception immediately. They seem likely to be indicative of bad code anyway, and it's so much easier to debug when you find out right away. The original rationale was to do what XHR does w.r.t. open()! Essentially, the goal was: 1. To abort previous reads in favor of the last one, like how XHR does. 2. The last read goes through. So: Adrian/Eric -- do you object to keeping this like XHR in terms of aborting previous reads? What I should *definitely* do is make spec. text more robust to reflect this; in general I want to make asynchronous parts of this spec. more like HTML5 here. -- A*
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 1:38 AM, Joran Greef wrote: > On 31 Mar 2011, at 9:34 AM, Jeremy Orlow wrote: > > > We have made an effort to understand other "contributions to the field". > > > > I'm not convinced that these are "essential database concepts" and having > personally spent quite some time working with the API in JS and implementing > it, I feel pretty confident that what we have for v1 is pretty solid. There > are definitely some things I wouldn't mind re-visiting or looking at closer, > possibly even for v1, but they all seem reasonable to study further for v2 > as well. > > > > We've spent a lot of time over the last year and a half talking about > IndexedDB. But now it's shipping in Firefox 4 and soon Chrome 11. So > realistically v1 is not going to change much unless we are convinced that > what's there is fundamentally broken. > > > > We intentionally limited the scope of v1, which is why we know there'll > be a v2. We can't solve all the problems at once, and the difficulty of > speccing something is typically exponential to the size of the API. > > > > Maybe a constructive way to discuss this would be to look at what use > cases will be difficult or impossible to achieve with the current design? > > Application-managed indices for starters That's not a use case. > I would consider that to be essential when designing indexed key/value > stores, and I would consider that to be the contribution made by almost > every other indexed key/value store to date. If we have to use IDB the way > FriendFeed used MySQL to achieve application-managed indices then I would > argue that the API is in fact "fundamentally broken" and we would be better > off with an embedding of SQLite by Mozilla. > > Regarding "the difficulty of speccing something is typically exponential to > the size of the API", if people want to build a Rube Goldberg device then > they must deal with the spec issues of that. > > If we were provided with the primitives for an indexed key/value store with > application-managed indices (as Nikunj suggested at the time), we would have > been well out of the starting blocks by now, and issues such as "computed > indexes", "indexing array values" etc. would have been non-issues. > > Summary: > > 1. There's a problem. > 2. It can still be fixed with a minimum of fuss. > 3. This requires an adjustment to the putObject and deleteObject interfaces > (see previous threads).
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
What they mean is that defining a generic subset of SQL with one implementation was considered too difficult. With more than one implementation a common subset becomes easier for the standard writers to extract. With RelationalDB I have taken a completely different approach. We started from relational algebra on relation objects as a starting point. The SQL backend then conforms to the requirements of the relational algebra - thus we define relationally-complete functionality with no reference to any SQL specification. Cheers, Keean. On 31 March 2011 16:06, Glenn Maynard wrote: > On Thu, Mar 31, 2011 at 11:43 AM, Benjamin Poulain < > benjamin.poul...@nokia.com> wrote: > > WebSQL in its current form is pretty dead, see > > http://www.w3.org/TR/webdatabase/ : > > Beware. This specification is no longer in active maintenance and > the > > Web Applications Working Group does not intend to maintain it > > further. > > > > And: > > This document was on the W3C Recommendation track but > specification > > work has stopped. The specification reached an impasse: all interested > > implementors have used the same SQL backend (Sqlite), but we need > multiple > > independent implementations to proceed along a standardisation > path. > > This is painful to read. WebSQL development died because SQLite, the most > widely-deployed database software in the world, was too good? That sounds > like a catastrophic failure of the W3C process. > > -- > Glenn Maynard >
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
No real reason - just trying to implement a minimal framework. Date objects
would be a definite must have going forward.
I was interested in trying to get something like this standardised, as I
believe it has none of the issues that stopped WebSQL, as it defines a
complete relational API independent of the implementation of SQL behind it.
The key thing is to get the browser implementors interested in implementing
it. If even one of the main browser implementors is not interested in
implementing it, then it will suffer the same fate as WebSQL.
Independent of standardisation (which I would like) I intend to try and
implement the same API on top of WebSQL and IndexedDB as a library. So
people are free to use the backend with the best performance without
changing their code. It aims to be as stateless as possible, and to
implement relational algebra on relations.
Cheers,
Keean.
On 31 March 2011 15:54, Nathan Kitchen wrote:
> That's nice, pretty much what I was thinking but somewhat more complete : )
> Is there not a w3 group progressing something like this? And if not, who
> would need to be lobbied to get one started?!
>
> As an aside, I note you didn't implement "date" as a supported data type.
> Was that a conscious decision, and if so what was the reasoning behind it?
>
> N
>
>
> On 31 March 2011 16:33, Keean Schupke wrote:
>
>> Have a look at my RelationalDB API
>>
>> https://github.com/keean/RelationalDB
>>
>> In particular examples/candy.html
>>
>> A lot of work went into the underlying concepts - Its work originally
>> published by myself and others at the 2004 Haskell Workshop, and follows on
>> from HaskellDB which was the original inspiration behind C#s Linq
>> functionality).
>>
>> It implements the relational-algebra operators as methods that operate on
>> relation objects.
>>
>> Let me know what you think.
>>
>>
>> Cheers,
>> Keean.
>>
>>
>> On 31 March 2011 15:19, Nathan Kitchen wrote:
>>
>>> Hi.
>>>
>>> I've been watching discussions on IndexedDB for a while now, and wondered
>>> if anyone would mind spending a few moments to explain how IndexedDB is
>>> related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality
>>> originally offered by WebSQL? If not, are there any plans to make a
>>> cross-platform variant of Web SQL?
>>>
>>> If (?) most web developers know SQL, is there a case to be made for
>>> abstracting SQL into JSON/JavaScript rather than moving to IndexedDB
>>> document storage? Reasons for asking this:
>>>
>>>- Many of the posts appearing to come from "the dev community" rather
>>>than W3C seem to expect more SQL-esque functionality from IndexedDB. If
>>> the
>>>enthusiasts who get involved enough to post to the board are expecting
>>>SQL/query type experience, maybe there is a driver for a native database
>>> API
>>>supporting this.
>>>- Several people have noted that third-party frameworks could
>>>implement this functionality. This might be a daft question, but isn't it
>>>easier to implement an "IndexedDB"-like framework on top of "WebSQL",
>>> than a
>>>"WebSQL"-like framework on top of "IndexedDB" (overuse of quotes to
>>> indicate
>>>the general concept).
>>>
>>> I had a ponder on how I'd like to see such a framework implemented (in
>>> both Access & SQLite :p ), and came up with a stack of pseudo-code below in
>>> my lunch break. Might make an interesting discussion point. It's not really
>>> IndexedDB, it's WebSQL v2. Or maybe WebJSQL or something. I'd be really
>>> interested to understand what advantages IndexedDB has over an
>>> implementation like the one below though.
>>>
>>> // DATABASE
>>> // First, open a database with the specified name. The number at the
>>> // end denotes the version of the specification that the application
>>> // plans to use. This allows forward-compatibility with vNext.
>>> var db = window.openDatabase("shoppinglist", 1.0);
>>>
>>> // MIGRATIONS
>>> // Next, create some migrations. These are predefined structures which
>>> // are validated by the browser database engine. A migration consists
>>> // of two actions: one "up", one "down". Each action specifies some
>>> // operations and parameters. It's up to the browser database to read
>>> // these and perform the appropriate action, as defined in the spec.
>>>
>>> // Other actions may include a "batch add" for "static" data. It could
>>> // also be valid to have key and index creation and removal as separate
>>> // actions.
>>>
>>> // SHOPPING TRIP
>>> var createTripTableAction = {
>>> action: "create-table",
>>> params: {
>>> name: "trip",
>>> columns: [
>>> { name: "id", type: "whole-number", primaryKey: true },
>>> { name: "name", type: "string", length: 32, regex:
>>> "[A-Z]{1,32}" } // regex: wouldn't that be nice...
>>> ],
>>> indexes: [
>>> {
>>> columns: [
>>> { name: "name", type: "full-text" }
>>> ]
>>>
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 3/31/11 12:06 PM, Glenn Maynard wrote: This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. No, it actually sounds like a success; it prevented a "specification" being created which would have been tied to a particular implementation, no matter how "widely-deployed". For comparison, IE6 was very widely deployed circa 2002. And yet specifying CSS, say, by saying "just do exactly what IE6 does" would not have been a good idea. Neither is defining WebSQL to do exactly what some particular version of SQLite does, and no one stepped up to define it better. -Boris
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On Thu, Mar 31, 2011 at 11:43 AM, Benjamin Poulain < benjamin.poul...@nokia.com> wrote: > WebSQL in its current form is pretty dead, see > http://www.w3.org/TR/webdatabase/ : > Beware. This specification is no longer in active maintenance and the > Web Applications Working Group does not intend to maintain it > further. > > And: > This document was on the W3C Recommendation track but specification > work has stopped. The specification reached an impasse: all interested > implementors have used the same SQL backend (Sqlite), but we need multiple > independent implementations to proceed along a standardisation path. This is painful to read. WebSQL development died because SQLite, the most widely-deployed database software in the world, was too good? That sounds like a catastrophic failure of the W3C process. -- Glenn Maynard
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
That's nice, pretty much what I was thinking but somewhat more complete : )
Is there not a w3 group progressing something like this? And if not, who
would need to be lobbied to get one started?!
As an aside, I note you didn't implement "date" as a supported data type.
Was that a conscious decision, and if so what was the reasoning behind it?
N
On 31 March 2011 16:33, Keean Schupke wrote:
> Have a look at my RelationalDB API
>
> https://github.com/keean/RelationalDB
>
> In particular examples/candy.html
>
> A lot of work went into the underlying concepts - Its work originally
> published by myself and others at the 2004 Haskell Workshop, and follows on
> from HaskellDB which was the original inspiration behind C#s Linq
> functionality).
>
> It implements the relational-algebra operators as methods that operate on
> relation objects.
>
> Let me know what you think.
>
>
> Cheers,
> Keean.
>
>
> On 31 March 2011 15:19, Nathan Kitchen wrote:
>
>> Hi.
>>
>> I've been watching discussions on IndexedDB for a while now, and wondered
>> if anyone would mind spending a few moments to explain how IndexedDB is
>> related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality
>> originally offered by WebSQL? If not, are there any plans to make a
>> cross-platform variant of Web SQL?
>>
>> If (?) most web developers know SQL, is there a case to be made for
>> abstracting SQL into JSON/JavaScript rather than moving to IndexedDB
>> document storage? Reasons for asking this:
>>
>>- Many of the posts appearing to come from "the dev community" rather
>>than W3C seem to expect more SQL-esque functionality from IndexedDB. If
>> the
>>enthusiasts who get involved enough to post to the board are expecting
>>SQL/query type experience, maybe there is a driver for a native database
>> API
>>supporting this.
>>- Several people have noted that third-party frameworks could
>>implement this functionality. This might be a daft question, but isn't it
>>easier to implement an "IndexedDB"-like framework on top of "WebSQL",
>> than a
>>"WebSQL"-like framework on top of "IndexedDB" (overuse of quotes to
>> indicate
>>the general concept).
>>
>> I had a ponder on how I'd like to see such a framework implemented (in
>> both Access & SQLite :p ), and came up with a stack of pseudo-code below in
>> my lunch break. Might make an interesting discussion point. It's not really
>> IndexedDB, it's WebSQL v2. Or maybe WebJSQL or something. I'd be really
>> interested to understand what advantages IndexedDB has over an
>> implementation like the one below though.
>>
>> // DATABASE
>> // First, open a database with the specified name. The number at the
>> // end denotes the version of the specification that the application
>> // plans to use. This allows forward-compatibility with vNext.
>> var db = window.openDatabase("shoppinglist", 1.0);
>>
>> // MIGRATIONS
>> // Next, create some migrations. These are predefined structures which
>> // are validated by the browser database engine. A migration consists
>> // of two actions: one "up", one "down". Each action specifies some
>> // operations and parameters. It's up to the browser database to read
>> // these and perform the appropriate action, as defined in the spec.
>>
>> // Other actions may include a "batch add" for "static" data. It could
>> // also be valid to have key and index creation and removal as separate
>> // actions.
>>
>> // SHOPPING TRIP
>> var createTripTableAction = {
>> action: "create-table",
>> params: {
>> name: "trip",
>> columns: [
>> { name: "id", type: "whole-number", primaryKey: true },
>> { name: "name", type: "string", length: 32, regex: "[A-Z]{1,32}"
>> } // regex: wouldn't that be nice...
>> ],
>> indexes: [
>> {
>> columns: [
>> { name: "name", type: "full-text" }
>> ]
>> } // More indexes here if required
>> ]
>> }
>> };
>>
>> var removeTripTableAction = {
>> action: "remove-table",
>> params: {
>> name: "shopping",
>> cleardata: true
>> }
>> };
>>
>>
>> // SHOPPING
>> var createShoppingTableAction = {
>> action: "create-table",
>> params: {
>> name: "shopping",
>> columns: [
>> { name: "id", type: "whole-number", primaryKey: true },
>> { name: "tripid", type: "whole-number" },
>> { name: "name", type: "string", length: 128, nillable: false },
>> { name: "cost", type: "decimal-number" }
>> ],
>> keys: [
>> { local: "tripid", foreign: { table: "trips", column: "id" },
>> cascade-delete: true } }
>> ],
>> indexes: [
>> {
>> columns: [
>> { name: "tripid", type: "asc" },
>> { name: "name", type: "full-text" }
>> ]
>> }
>> ]
>> }
>> };
>>
>> var removeShoppi
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 03/31/2011 05:19 PM, ext Nathan Kitchen wrote: I've been watching discussions on IndexedDB for a while now, and wondered if anyone would mind spending a few moments to explain how IndexedDB is related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality originally offered by WebSQL? If not, are there any plans to make a cross-platform variant of Web SQL? WebSQL in its current form is pretty dead, see http://www.w3.org/TR/webdatabase/ : Beware. This specification is no longer in active maintenance and the Web Applications Working Group does not intend to maintain it further. And: This document was on the W3C Recommendation track but specification work has stopped. The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path. I suggest you to look at the mailing list archive if you want all the details. cheers, Benjamin
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Have a look at my RelationalDB API
https://github.com/keean/RelationalDB
In particular examples/candy.html
A lot of work went into the underlying concepts - Its work originally
published by myself and others at the 2004 Haskell Workshop, and follows on
from HaskellDB which was the original inspiration behind C#s Linq
functionality).
It implements the relational-algebra operators as methods that operate on
relation objects.
Let me know what you think.
Cheers,
Keean.
On 31 March 2011 15:19, Nathan Kitchen wrote:
> Hi.
>
> I've been watching discussions on IndexedDB for a while now, and wondered
> if anyone would mind spending a few moments to explain how IndexedDB is
> related (or not) to WebSQL. Is IndexedDB seen as replacing the functionality
> originally offered by WebSQL? If not, are there any plans to make a
> cross-platform variant of Web SQL?
>
> If (?) most web developers know SQL, is there a case to be made for
> abstracting SQL into JSON/JavaScript rather than moving to IndexedDB
> document storage? Reasons for asking this:
>
>- Many of the posts appearing to come from "the dev community" rather
>than W3C seem to expect more SQL-esque functionality from IndexedDB. If the
>enthusiasts who get involved enough to post to the board are expecting
>SQL/query type experience, maybe there is a driver for a native database
> API
>supporting this.
>- Several people have noted that third-party frameworks could implement
>this functionality. This might be a daft question, but isn't it easier to
>implement an "IndexedDB"-like framework on top of "WebSQL", than a
>"WebSQL"-like framework on top of "IndexedDB" (overuse of quotes to
> indicate
>the general concept).
>
> I had a ponder on how I'd like to see such a framework implemented (in both
> Access & SQLite :p ), and came up with a stack of pseudo-code below in my
> lunch break. Might make an interesting discussion point. It's not really
> IndexedDB, it's WebSQL v2. Or maybe WebJSQL or something. I'd be really
> interested to understand what advantages IndexedDB has over an
> implementation like the one below though.
>
> // DATABASE
> // First, open a database with the specified name. The number at the
> // end denotes the version of the specification that the application
> // plans to use. This allows forward-compatibility with vNext.
> var db = window.openDatabase("shoppinglist", 1.0);
>
> // MIGRATIONS
> // Next, create some migrations. These are predefined structures which
> // are validated by the browser database engine. A migration consists
> // of two actions: one "up", one "down". Each action specifies some
> // operations and parameters. It's up to the browser database to read
> // these and perform the appropriate action, as defined in the spec.
>
> // Other actions may include a "batch add" for "static" data. It could
> // also be valid to have key and index creation and removal as separate
> // actions.
>
> // SHOPPING TRIP
> var createTripTableAction = {
> action: "create-table",
> params: {
> name: "trip",
> columns: [
> { name: "id", type: "whole-number", primaryKey: true },
> { name: "name", type: "string", length: 32, regex: "[A-Z]{1,32}"
> } // regex: wouldn't that be nice...
> ],
> indexes: [
> {
> columns: [
> { name: "name", type: "full-text" }
> ]
> } // More indexes here if required
> ]
> }
> };
>
> var removeTripTableAction = {
> action: "remove-table",
> params: {
> name: "shopping",
> cleardata: true
> }
> };
>
>
> // SHOPPING
> var createShoppingTableAction = {
> action: "create-table",
> params: {
> name: "shopping",
> columns: [
> { name: "id", type: "whole-number", primaryKey: true },
> { name: "tripid", type: "whole-number" },
> { name: "name", type: "string", length: 128, nillable: false },
> { name: "cost", type: "decimal-number" }
> ],
> keys: [
> { local: "tripid", foreign: { table: "trips", column: "id" },
> cascade-delete: true } }
> ],
> indexes: [
> {
> columns: [
> { name: "tripid", type: "asc" },
> { name: "name", type: "full-text" }
> ]
> }
> ]
> }
> };
>
> var removeShoppingTableAction = {
> action: "remove-table",
> params: {
> name: "shopping",
> cleardata: true
> }
> };
>
> var migTrip = { up: createTripTableAction, down: removeTripTableAction };
> var migShopping = { up: createShoppingTableAction, down:
> removeShoppingTableAction };
>
> // Example calls which might be appropriate here, where the word
> // "callback" is placeholder for async operations:
>
> db.getVersion();// Returns 0
> db.migrations.add(migTrip); // Persists the up/do
[WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
Hi.
I've been watching discussions on IndexedDB for a while now, and wondered if
anyone would mind spending a few moments to explain how IndexedDB is related
(or not) to WebSQL. Is IndexedDB seen as replacing the functionality
originally offered by WebSQL? If not, are there any plans to make a
cross-platform variant of Web SQL?
If (?) most web developers know SQL, is there a case to be made for
abstracting SQL into JSON/JavaScript rather than moving to IndexedDB
document storage? Reasons for asking this:
- Many of the posts appearing to come from "the dev community" rather
than W3C seem to expect more SQL-esque functionality from IndexedDB. If the
enthusiasts who get involved enough to post to the board are expecting
SQL/query type experience, maybe there is a driver for a native database API
supporting this.
- Several people have noted that third-party frameworks could implement
this functionality. This might be a daft question, but isn't it easier to
implement an "IndexedDB"-like framework on top of "WebSQL", than a
"WebSQL"-like framework on top of "IndexedDB" (overuse of quotes to indicate
the general concept).
I had a ponder on how I'd like to see such a framework implemented (in both
Access & SQLite :p ), and came up with a stack of pseudo-code below in my
lunch break. Might make an interesting discussion point. It's not really
IndexedDB, it's WebSQL v2. Or maybe WebJSQL or something. I'd be really
interested to understand what advantages IndexedDB has over an
implementation like the one below though.
// DATABASE
// First, open a database with the specified name. The number at the
// end denotes the version of the specification that the application
// plans to use. This allows forward-compatibility with vNext.
var db = window.openDatabase("shoppinglist", 1.0);
// MIGRATIONS
// Next, create some migrations. These are predefined structures which
// are validated by the browser database engine. A migration consists
// of two actions: one "up", one "down". Each action specifies some
// operations and parameters. It's up to the browser database to read
// these and perform the appropriate action, as defined in the spec.
// Other actions may include a "batch add" for "static" data. It could
// also be valid to have key and index creation and removal as separate
// actions.
// SHOPPING TRIP
var createTripTableAction = {
action: "create-table",
params: {
name: "trip",
columns: [
{ name: "id", type: "whole-number", primaryKey: true },
{ name: "name", type: "string", length: 32, regex: "[A-Z]{1,32}" }
// regex: wouldn't that be nice...
],
indexes: [
{
columns: [
{ name: "name", type: "full-text" }
]
} // More indexes here if required
]
}
};
var removeTripTableAction = {
action: "remove-table",
params: {
name: "shopping",
cleardata: true
}
};
// SHOPPING
var createShoppingTableAction = {
action: "create-table",
params: {
name: "shopping",
columns: [
{ name: "id", type: "whole-number", primaryKey: true },
{ name: "tripid", type: "whole-number" },
{ name: "name", type: "string", length: 128, nillable: false },
{ name: "cost", type: "decimal-number" }
],
keys: [
{ local: "tripid", foreign: { table: "trips", column: "id" },
cascade-delete: true } }
],
indexes: [
{
columns: [
{ name: "tripid", type: "asc" },
{ name: "name", type: "full-text" }
]
}
]
}
};
var removeShoppingTableAction = {
action: "remove-table",
params: {
name: "shopping",
cleardata: true
}
};
var migTrip = { up: createTripTableAction, down: removeTripTableAction };
var migShopping = { up: createShoppingTableAction, down:
removeShoppingTableAction };
// Example calls which might be appropriate here, where the word
// "callback" is placeholder for async operations:
db.getVersion();// Returns 0
db.migrations.add(migTrip); // Persists the up/down actions as a
list somewhere
db.migrations.add(migShopping); // Persists the up/down actions as a
list somewhere
db.migrateUp(2, callback); // Runs the two migrations
db.getVersion();// Returns 2
db.migrateDown(0, callback);// Runs all "down" migrations to
schema version 0.
db.getVersion();// Returns 0
db.destroy(true); // Get rid of the database,
optionally running "down" migrations
// * Note that during migration operations the database should be locked: no
other threads should
// * be able to interact/access the database while this is taking place.
// QUERIES
// Invent a JSON structure for creating queries. There could be simple
// versio
Re: SearchBox API
The reason I suggested web messaging is that more and more browser UI is being built on top of the web platform with things like chromeless [1] and chrome's WebUI [2], and this will likely include search boxes at some point. This would give the search box a natural endpoint browsing context for web messaging based communication with web apps. Now that I think about it, few if any changes would be needed to the Search Box specification to support this, as the SearchBox interface could be implemented as a javascript library that uses web messaging behind the scenes. The only way to observe the difference between this and a host object implementation would be by overriding window.postMessage which doesn't seem to be a problem. My main observation would be that javascript implementations using web messaging should be considered as potential alternatives to host objects in specifications involving communication between browser UI and web apps. [1] https://mozillalabs.com/chromeless/ [2] http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/webui/ On Mon, Mar 28, 2011 at 1:59 PM, Adam Barth wrote: > Sure, that syntax would work, but we'd still need to specify the > postMessage protocol, just like we need to specify the IDL interface. > It's not clear what moving to postMessage would buy us. > > Adam > > > On Mon, Mar 28, 2011 at 11:20 AM, Sean Eagan wrote: >> Why not just build this on top of web messaging[1] by having a >> browsing context associated with the search box (or entire browser >> chrome) that can communicate with a SERP or any page that wants to >> accept input from a search box or otherwise communicate directly with >> a user agent? >> >> [1] http://dev.w3.org/html5/postmsg/ >> >> On Mon, Mar 21, 2011 at 1:38 PM, Adam Barth wrote: >>> On Mon, Mar 21, 2011 at 11:16 AM, Edward Lee wrote: > enables "instant" style interaction between the user agent's search Assuming the user agent automatically loads a url that is triggered by a user key stroke, e.g., typing "g" results in "http://www.google.com/";, the instant-style interaction is almost there already for certain urls. These instant-style urls would include what the user typed -- perhaps as a query parameter. For example, a user agent might request these pages as the user types: http://www.google.com/search?q=g http://www.google.com/search?q=go http://www.google.com/search?q=goo Here, the results page shows the new query and updated results on every user keystroke. These instant-style urls can also avoid refetching and rerendering the whole page if the user's input shows up in the #fragment and the page detects onHashChange. >>> >>> That's true, but you can only transmit one event that way. In this >>> design, you've chosen to map the "change" event to hashchange. How >>> should the user agent communicate that the user is done typing (i.e., >>> the "submit" event, which triggers when presses the enter key)? >>> Similarly, the communication in that approach is unidirectional, which >>> means the page can't suggest search completions. >>> >>> Adam >>> >>> >> >> >> >> -- >> Sean Eagan >> > -- Sean Eagan
Re: RfC: WebApps Testing Process
On Mar/31/2011 10:04 AM, ext Robin Berjon wrote: On Mar 31, 2011, at 14:04 , Arthur Barstow wrote: 1. What is the level of uptake of testharness.js within the HTML WG and other WGs? If any of these groups provide "usage" information, what are the URIs? Do any WGs make testharness.js's use Mandatory? Currently, its usage in the above documents makes its usage a "Should". DAP hasn't yet started writing tests in earnest, but as some of our documents are nearing LC we're getting ready for it. We've resolved to use testharness.js as well. We haven't captured it as a MUST, but given that we have a bunch of specs and that we expect work to be distributed to many people we expect to use it throughout — so pretty much a tacit MUST. Yes, I think that makes sense for DAP (which, AFAIU, generally will not have legacy test suites to leverage). In the case of WebApps, I would expect some vendors to have already created test suites for specs they have deployed (e.g. Web Storage, Server-sent Events). In those cases, I do not think we want a testharness.js dependency to block them from contributing those test suites to the W3C. -Art Barstow
Re: RfC: WebApps Testing Process
On Mar 31, 2011, at 14:04 , Arthur Barstow wrote: > 1. What is the level of uptake of testharness.js within the HTML WG and other > WGs? If any of these groups provide "usage" information, what are the URIs? > Do any WGs make testharness.js's use Mandatory? Currently, its usage in the > above documents makes its usage a "Should". DAP hasn't yet started writing tests in earnest, but as some of our documents are nearing LC we're getting ready for it. We've resolved to use testharness.js as well. We haven't captured it as a MUST, but given that we have a bunch of specs and that we expect work to be distributed to many people we expect to use it throughout — so pretty much a tacit MUST. > 2. What, if any, requirements/guidelines are there regarding copyrights in > test files? For example, does every file in a test suite need a W3C > copyright? Are external contributors allowed to include their own > copyright(s)? My understanding is that the W3C Test License (under which tests have to be released) essentially is a copyright license, which I'm guessing precludes other copyrights (as well as derivative works in many situations). I could be wrong though — it's probably worth asking Rigo. There's precedent for test suites being done externally, notably the XML test suite was largely created by OASIS. A quick look through it couldn't find any licensing or copyright information though, so it might just have been built on warm fuzzy feelings (as most good things are). http://www.w3.org/Consortium/Legal/2008/04-testsuite-license.html -- Robin Berjon - http://berjon.com/
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 March 2011 12:41, Joran Greef wrote: > On 31 Mar 2011, at 12:52 PM, Keean Schupke wrote: > > > I totally agree with everything so far... > > > >> 3. This requires an adjustment to the putObject and deleteObject > interfaces (see previous threads). > > > > I disagree that a simple API change is the answer. The problem is > architectural, not just a superficial API issue. > > Yes, for IndexedDB to be stateless with respect to application schema, one > would need to: > > 1. Provide the application with a first-class means to manage indexes at > time of putting/deleting objects. > 2. Treat objects as opaque (remove key path, structured clone mechanisms, > application must provide an id and JSON value to put/delete calls, reduces > serialization/deserialization overhead where application already has the > object as a string). > 3. Remove setVersion (redundant, application migrates objects and indexes > using transactions as it needs to). > 4. Remove createIndex. > > This would rip so much from the spec as to reduce it to a bunch of tatters, > defining nothing more than an interface for index/key/value primitives in > terms of well-established interfaces. > > Essentially, we need LocalStorage with asynchronous IO (based on Node's > callback style), large quota support, and a BTree API. Failing that, a > decent FileSystem API on which to build these. Stateless indexes can be provided differently to how you suggest. You can have a 'validate_index' call that checks the index exists and creates it if it does not. It is stateless in the sense that you call that to open existing index or create one, you dont care if the database has one already or not. Infact you can make SQL stateless by providing a "validate_schema" call that succeeds if the schema of the database matches the passed schema, can be modified with no data loss to be the same, or needs to be created. The RelationalDB wrapper for WebSQL provides this kind of stateless approach for SQL... you can check it out on github if you like (its a work in progress though): https://github.com/keean/RelationalDB Cheers, Keean.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 12:52 PM, Keean Schupke wrote: > I totally agree with everything so far... > >> 3. This requires an adjustment to the putObject and deleteObject interfaces >> (see previous threads). > > I disagree that a simple API change is the answer. The problem is > architectural, not just a superficial API issue. Yes, for IndexedDB to be stateless with respect to application schema, one would need to: 1. Provide the application with a first-class means to manage indexes at time of putting/deleting objects. 2. Treat objects as opaque (remove key path, structured clone mechanisms, application must provide an id and JSON value to put/delete calls, reduces serialization/deserialization overhead where application already has the object as a string). 3. Remove setVersion (redundant, application migrates objects and indexes using transactions as it needs to). 4. Remove createIndex. This would rip so much from the spec as to reduce it to a bunch of tatters, defining nothing more than an interface for index/key/value primitives in terms of well-established interfaces. Essentially, we need LocalStorage with asynchronous IO (based on Node's callback style), large quota support, and a BTree API. Failing that, a decent FileSystem API on which to build these.
RfC: WebApps Testing Process
Hi All, During the 2011 TPAC meeting, I agreed to an action (action-611) to work with Chaals and WebApps' Team Contacts to define the group's testing processes. To that end, I created the following documents: 1. http://www.w3.org/2008/webapps/wiki/Testing - some high level goals, and links to other related resources. 2. http://www.w3.org/2008/webapps/wiki/Submission - licensing info, test repository structure and how to submit tests. Notes: a) since WebApps will have a test suite per spec, the proposed structure is different than the structure used by HTML WG; b) if the proposed structure is accepted, some of WebApps existing dirs in Hg will need to be moved. 3. http://www.w3.org/2008/webapps/wiki/Approval - how to start a test case review, approval process, how to update an approved test case 4. http://www.w3.org/2008/webapps/wiki/Harness - short intro to the harness which is a reuse of the one used by the HTML WG (created and maintained by James Graham) Comments on any of the above are welcome, especially regarding the various "@@@ TBD: ..." tags that are sprinkled throughout the above documents. A couple of questions too ... 1. What is the level of uptake of testharness.js within the HTML WG and other WGs? If any of these groups provide "usage" information, what are the URIs? Do any WGs make testharness.js's use Mandatory? Currently, its usage in the above documents makes its usage a "Should". 2. What, if any, requirements/guidelines are there regarding copyrights in test files? For example, does every file in a test suite need a W3C copyright? Are external contributors allowed to include their own copyright(s)? 3. Where can I find some information about "RefTests"? In particular, where is the RefTest usage information HTML contributors use? Regarding an "inventory" of WebApps' test suites - currently, if a spec has a test suite, a link to the suite is included on the PubStatus page (Remarks column). I could add a new column to PubStatus like "Testing" to make testing standout a bit more; or I could create a new document for the test suite inventory. Comments on this are also welcome: http://www.w3.org/2008/webapps/wiki/PubStatus -Art Barstow
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 March 2011 08:38, Joran Greef wrote: > On 31 Mar 2011, at 9:34 AM, Jeremy Orlow wrote: > > > We have made an effort to understand other "contributions to the field". > > > > I'm not convinced that these are "essential database concepts" and having > personally spent quite some time working with the API in JS and implementing > it, I feel pretty confident that what we have for v1 is pretty solid. There > are definitely some things I wouldn't mind re-visiting or looking at closer, > possibly even for v1, but they all seem reasonable to study further for v2 > as well. > > > > We've spent a lot of time over the last year and a half talking about > IndexedDB. But now it's shipping in Firefox 4 and soon Chrome 11. So > realistically v1 is not going to change much unless we are convinced that > what's there is fundamentally broken. > > > > We intentionally limited the scope of v1, which is why we know there'll > be a v2. We can't solve all the problems at once, and the difficulty of > speccing something is typically exponential to the size of the API. > > > > Maybe a constructive way to discuss this would be to look at what use > cases will be difficult or impossible to achieve with the current design? > > Application-managed indices for starters. I would consider that to be > essential when designing indexed key/value stores, and I would consider that > to be the contribution made by almost every other indexed key/value store to > date. If we have to use IDB the way FriendFeed used MySQL to achieve > application-managed indices then I would argue that the API is in fact > "fundamentally broken" and we would be better off with an embedding of > SQLite by Mozilla. > > Regarding "the difficulty of speccing something is typically exponential to > the size of the API", if people want to build a Rube Goldberg device then > they must deal with the spec issues of that. > > If we were provided with the primitives for an indexed key/value store with > application-managed indices (as Nikunj suggested at the time), we would have > been well out of the starting blocks by now, and issues such as "computed > indexes", "indexing array values" etc. would have been non-issues. > > Summary: > > 1. There's a problem. > 2. It can still be fixed with a minimum of fuss. > I totally agree with everything so far... > 3. This requires an adjustment to the putObject and deleteObject interfaces > (see previous threads). > I disagree that a simple API change is the answer. The problem is architectural, not just a superficial API issue. Cheers, Keean.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
I was the one that asked for callbacks. > but what do we do if those callbacks don't > return consistent results? Or even do evil things like modify the > stores where data is being inserted? If the callback maps all values to a sort-order of '1' there could only ever be one entry in the index... its not hard, the callback is passed an immutable copy of the object and returns a sort-order as a binary-blob. If you capture the object store in the closure you of course you could do evil things as side-effects. But that is true in any non-purely-functional language, you can always do evil things with side-effects. > In short, I don't think we'll get much further here without a concrete proposal. Which basically means nobody working on the current implementations understands the issues, or thinks the issues are unimportant? Cheers, Keean.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 9:34 AM, Jeremy Orlow wrote: > We have made an effort to understand other "contributions to the field". > > I'm not convinced that these are "essential database concepts" and having > personally spent quite some time working with the API in JS and implementing > it, I feel pretty confident that what we have for v1 is pretty solid. There > are definitely some things I wouldn't mind re-visiting or looking at closer, > possibly even for v1, but they all seem reasonable to study further for v2 as > well. > > We've spent a lot of time over the last year and a half talking about > IndexedDB. But now it's shipping in Firefox 4 and soon Chrome 11. So > realistically v1 is not going to change much unless we are convinced that > what's there is fundamentally broken. > > We intentionally limited the scope of v1, which is why we know there'll be a > v2. We can't solve all the problems at once, and the difficulty of speccing > something is typically exponential to the size of the API. > > Maybe a constructive way to discuss this would be to look at what use cases > will be difficult or impossible to achieve with the current design? Application-managed indices for starters. I would consider that to be essential when designing indexed key/value stores, and I would consider that to be the contribution made by almost every other indexed key/value store to date. If we have to use IDB the way FriendFeed used MySQL to achieve application-managed indices then I would argue that the API is in fact "fundamentally broken" and we would be better off with an embedding of SQLite by Mozilla. Regarding "the difficulty of speccing something is typically exponential to the size of the API", if people want to build a Rube Goldberg device then they must deal with the spec issues of that. If we were provided with the primitives for an indexed key/value store with application-managed indices (as Nikunj suggested at the time), we would have been well out of the starting blocks by now, and issues such as "computed indexes", "indexing array values" etc. would have been non-issues. Summary: 1. There's a problem. 2. It can still be fixed with a minimum of fuss. 3. This requires an adjustment to the putObject and deleteObject interfaces (see previous threads).
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 9:53 AM, Jonas Sicking wrote: > I previously have asked for a detailed proposal, but so far you have > not supplied one but instead keep referring to other unnamed database > APIs. I have already provided an adequate interface proposal for putObject and deleteObject. I have already referenced at least Redis and Tokyo Cabinet as examples of "stateless" database interfaces, on numerous occasions. > For example, you've asked for callbacks to > implement collations, but what do we do if those callbacks don't > return consistent results? I have not once asked for callbacks, let alone callbacks to implement collations. You have jumped to this conclusion from my previous post, and missed the point of it entirely.
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 12:16 AM, Joran Greef wrote: > On 31 Mar 2011, at 1:01 AM, Jonas Sicking wrote: > >> Anyhow, I do think that the idea of passing in index values at the >> same time as a entry is created/modified is an interesting idea. And I >> have said so in the past on this list. It's definitely something we >> should consider for v2. > >> Oh, and if we did this, I wouldn't really know how to support things >> like collations. Neither if you did collations using built in sets of >> locales (like in Pablo's recent proposal), nor if you used some sort >> of callback to do collation. >> >> / Jonas > > That's fine. You don't need to figure it out. Just look at how stateless > databases have done it (or not done it) and do likewise. > > I submit to you that there is inadequate understanding of the concerns > raised, hence the lack of urgency in trying to address them. That there is > even a need for a "V2" is symptomatic of this. > > It may be a good idea to start looking at these things not as "interesting > ideas" but as essential database concepts. > > If someone were trying to build some kind of transactional indexed key value > store for the web, and they wanted to do a truly great job of it, they would > certainly want to learn everything they could from databases that have made > contributions to the field. I previously have asked for a detailed proposal, but so far you have not supplied one but instead keep referring to other unnamed database APIs. It has also been pointed out that there are unique constraints on APIs in browsers. For example due to the fact that several "applications", i.e. pages, will be interacting with a given database. At the same time, and in some browsers from different processes. Additionally we're aiming to make an API which is easier to use, and where we can't place as much trust in the web page as you'd normally put in the user of a database API. For example, you've asked for callbacks to implement collations, but what do we do if those callbacks don't return consistent results? Or even do evil things like modify the stores where data is being inserted? In short, I don't think we'll get much further here without a concrete proposal. Especially not for IndexedDB v1. / Jonas
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On Thu, Mar 31, 2011 at 12:16 AM, Joran Greef wrote: > On 31 Mar 2011, at 1:01 AM, Jonas Sicking wrote: > > > Anyhow, I do think that the idea of passing in index values at the > > same time as a entry is created/modified is an interesting idea. And I > > have said so in the past on this list. It's definitely something we > > should consider for v2. > > > Oh, and if we did this, I wouldn't really know how to support things > > like collations. Neither if you did collations using built in sets of > > locales (like in Pablo's recent proposal), nor if you used some sort > > of callback to do collation. > > > > / Jonas > > That's fine. You don't need to figure it out. Just look at how stateless > databases have done it (or not done it) and do likewise. > > I submit to you that there is inadequate understanding of the concerns > raised, hence the lack of urgency in trying to address them. That there is > even a need for a "V2" is symptomatic of this. > > It may be a good idea to start looking at these things not as "interesting > ideas" but as essential database concepts. > > If someone were trying to build some kind of transactional indexed key > value store for the web, and they wanted to do a truly great job of it, they > would certainly want to learn everything they could from databases that have > made contributions to the field. > We have made an effort to understand other "contributions to the field". I'm not convinced that these are "essential database concepts" and having personally spent quite some time working with the API in JS and implementing it, I feel pretty confident that what we have for v1 is pretty solid. There are definitely some things I wouldn't mind re-visiting or looking at closer, possibly even for v1, but they all seem reasonable to study further for v2 as well. We've spent a lot of time over the last year and a half talking about IndexedDB. But now it's shipping in Firefox 4 and soon Chrome 11. So realistically v1 is not going to change much unless we are convinced that what's there is fundamentally broken. We intentionally limited the scope of v1, which is why we know there'll be a v2. We can't solve all the problems at once, and the difficulty of speccing something is typically exponential to the size of the API. Maybe a constructive way to discuss this would be to look at what use cases will be difficult or impossible to achieve with the current design? J
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 31 Mar 2011, at 1:01 AM, Jonas Sicking wrote: > Anyhow, I do think that the idea of passing in index values at the > same time as a entry is created/modified is an interesting idea. And I > have said so in the past on this list. It's definitely something we > should consider for v2. > Oh, and if we did this, I wouldn't really know how to support things > like collations. Neither if you did collations using built in sets of > locales (like in Pablo's recent proposal), nor if you used some sort > of callback to do collation. > > / Jonas That's fine. You don't need to figure it out. Just look at how stateless databases have done it (or not done it) and do likewise. I submit to you that there is inadequate understanding of the concerns raised, hence the lack of urgency in trying to address them. That there is even a need for a "V2" is symptomatic of this. It may be a good idea to start looking at these things not as "interesting ideas" but as essential database concepts. If someone were trying to build some kind of transactional indexed key value store for the web, and they wanted to do a truly great job of it, they would certainly want to learn everything they could from databases that have made contributions to the field.
