Re: IndexedDB: Thoughts on implementing IndexedDB
On Tue, Jul 30, 2013 at 3:13 PM, Joshua Bell jsb...@google.com wrote:
And now replying to the non-nits:
On Tue, Jul 30, 2013 at 1:30 AM, Austin William Wright a...@bzfx.netwrote:
I've been meaning to implement IndexedDB in some fashion for a while.
Earlier this month, shortly after the call for implementations, I realized
I should be getting on that. I've been working on an in-memory ECMAScript
implementation with fast data structures and the like. I also intend to
experiment with new features like new types of indexes (hash tables that
can't be iterated, and index values calculated by expression/function,
which appears to have been discussed elsewhere).
I've had a few thoughts, mostly about language:
(1) Is there no way to specify an arbitrary nested path? I want to do
something like ['menus', x] where `x` is some token which may be anything,
like an empty string or a string with a period in it. This is especially
important if there are structures like {http://example.com/URI:
value} in documents, which is especially common in JSON-LD. From what I
can tell, IndexedDB essentially makes it impossible to index JSON-LD
documents.
It appears the current behavior instead allows you to index by multiple
keys, but it's not immediately obvious this is the rationale.
How *would* one include a property whose key includes a period? This
seems to be asking for security problems, if authors need to implement an
escaping scheme for their keys, either when constructing a key path or when
constructing objects. Database names can be anything, why not key names?
The key path mechanism (and by definition, the index mechanism) definitely
doesn't support every use case. It is focused on the simple case where
the structure of the data being stored is under the control of the
developer authoring code against the IDB API. Slap a library in the middle
that's exposing a radically different storage API to authors and that
library is going to need to compute index keys on its own and produce
wrapper objects, or some such.
One of the ideas that's been talked about for v2 is extensible indexing,
allowing the index key to be computed by a script function.
Computing index keys would be a fantastic step for any database, I think.
If one could also define a custom comparison operator, this would perhaps
one of the more powerful features in a database that I've seen.
(3) I had trouble deciphering the exact behavior of multiple open
transactions on one another. I eventually realized the definition of
IDBTransactionMode describes the behavior.
Still, however, this document appears to talk in terms of what is
written to the database. But this isn't well defined. If something is
written to the database, wouldn't it affect what is read in a readonly
transaction? (No.)
And the language seems inconsistent. The language for `abort` says that
changes to the database must be rolled back (as if every operation writes
to storage), but the language for `Steps for committing a transaction`
specifies it is at that time the data is written (as if all write
operations up to this point are kept in memory). There's not strictly a
contradiction here, but perhaps more neutral language could be used.
Agreed, this could be improved. (Practically speaking, I expect that would
happen if we end up with implementation differences that require refining
the language in a future iteration.)
(5) I found the language for iterating and creating a Cursor hard to
understand being nested in multiple layers of algorithms. Specifically,
where an IDBCursor instance was actually exposed to the user. But now it
makes sense, and I don't really see how it might be improved. An
(informative) example on iterating a cursor may be helpful.
I recently added one towards the start of the spec (The following example
looks up all books in the database by author using an index and a cursor)
- is that what you were thinking? Is it just a matter of spec organization?
I think at some point in the spec history the examples were more integrated
into the text.
I recall eventually finding that example, I think that works.
(6) The document refers to the HTML5 Structured Clone Algorithm. It's a
bit concerning that it has to refer to ECMAScript algorithms defined in a
specification that defines a markup language. I don't think referring to a
markup language should be necessary (I don't intend on using my
implementation in an (X)HTML environment, just straight XML if anything at
all), though perhaps this is just a modularity problem with the HTML5 draft
(or rather, lack thereof).
Agreed that it seems like an odd place for it in the abstract, but the
HTML spec defines much of the behavior of the browser environment beyond
the markup language. Hixie and Anne are doing some spec refactoring work;
perhaps some day it will be more modular. Indexed DB is very much designed
to be an API for scripts running
IndexedDB: Thoughts on implementing IndexedDB
I've been meaning to implement IndexedDB in some fashion for a while.
Earlier this month, shortly after the call for implementations, I realized
I should be getting on that. I've been working on an in-memory ECMAScript
implementation with fast data structures and the like. I also intend to
experiment with new features like new types of indexes (hash tables that
can't be iterated, and index values calculated by expression/function,
which appears to have been discussed elsewhere).
I've had a few thoughts, mostly about language:
(1) Is there no way to specify an arbitrary nested path? I want to do
something like ['menus', x] where `x` is some token which may be anything,
like an empty string or a string with a period in it. This is especially
important if there are structures like {http://example.com/URI: value}
in documents, which is especially common in JSON-LD. From what I can tell,
IndexedDB essentially makes it impossible to index JSON-LD documents.
It appears the current behavior instead allows you to index by multiple
keys, but it's not immediately obvious this is the rationale.
How *would* one include a property whose key includes a period? This seems
to be asking for security problems, if authors need to implement an
escaping scheme for their keys, either when constructing a key path or when
constructing objects. Database names can be anything, why not key names?
(2) There are still a few references to *Sync terms from the old
synchronous API. Specifically, IDBDatabaseSync and IDBCursorWithValueSync.
(3) I had trouble deciphering the exact behavior of multiple open
transactions on one another. I eventually realized the definition of
IDBTransactionMode describes the behavior.
Still, however, this document appears to talk in terms of what is written
to the database. But this isn't well defined. If something is written to
the database, wouldn't it affect what is read in a readonly transaction?
(No.)
And the language seems inconsistent. The language for `abort` says that
changes to the database must be rolled back (as if every operation writes
to storage), but the language for `Steps for committing a transaction`
specifies it is at that time the data is written (as if all write
operations up to this point are kept in memory). There's not strictly a
contradiction here, but perhaps more neutral language could be used.
(4) The section Steps for asynchronously executing a request says Set
/transaction/ to the `transaction` associated with `source`.
Perhaps it should instead say Let /transaction/ be the `transaction` which
is associated with `source`
Because /transaction/ is previously undefined.
(5) I found the language for iterating and creating a Cursor hard to
understand being nested in multiple layers of algorithms. Specifically,
where an IDBCursor instance was actually exposed to the user. But now it
makes sense, and I don't really see how it might be improved. An
(informative) example on iterating a cursor may be helpful.
(6) The document refers to the HTML5 Structured Clone Algorithm. It's a bit
concerning that it has to refer to ECMAScript algorithms defined in a
specification that defines a markup language. I don't think referring to a
markup language should be necessary (I don't intend on using my
implementation in an (X)HTML environment, just straight XML if anything at
all), though perhaps this is just a modularity problem with the HTML5 draft
(or rather, lack thereof).
Finally, is there a good test suite? I can't seem to find anything in the
way of regression tests. I'll perhaps publish my own, if not.
Austin Wright.
Re: IndexedDB: Thoughts on implementing IndexedDB
Finally, is there a good test suite? I can't seem to find anything in the way of regression tests. I'll perhaps publish my own, if not. +1, I've developed my own In-Memory implementation as a polyfill to ignore a bug on Chrome regarding to files storage and I'm also thinking about develop one for Node.js, and this would be really useful. I believe there are some tests on W3C GitHub account, but didn't have time to check them myself. -- Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix. – Linus Tordvals, creador del sistema operativo Linux
Re: IndexedDB: Thoughts on implementing IndexedDB
On 7/30/13 4:37 AM, ext pira...@gmail.com wrote: Finally, is there a good test suite? I can't seem to find anything in the way of regression tests. I'll perhaps publish my own, if not. +1, I've developed my own In-Memory implementation as a polyfill to ignore a bug on Chrome regarding to files storage and I'm also thinking about develop one for Node.js, and this would be really useful. I believe there are some tests on W3C GitHub account, but didn't have time to check them myself. Some IDB tests have been submitted https://github.com/w3c/web-platform-tests/tree/master/IndexedDB/submissions and [1] contains some information about the review status of these submissions. ([2] contains general information about WebApps' testing effort.) -AB [1] http://lists.w3.org/Archives/Public/public-webapps-testsuite/2013Jul/.html [2] http://www.w3.org/wiki/Webapps/Testing
Re: IndexedDB: Thoughts on implementing IndexedDB
That were just the tests I was talking about, thanks :-) Unluckily it seems they are not prepared to just give them to your IndexedDB implementation and run them all at once, but it's a good starting point. 2013/7/30 Arthur Barstow art.bars...@nokia.com: On 7/30/13 4:37 AM, ext pira...@gmail.com wrote: Finally, is there a good test suite? I can't seem to find anything in the way of regression tests. I'll perhaps publish my own, if not. +1, I've developed my own In-Memory implementation as a polyfill to ignore a bug on Chrome regarding to files storage and I'm also thinking about develop one for Node.js, and this would be really useful. I believe there are some tests on W3C GitHub account, but didn't have time to check them myself. Some IDB tests have been submitted https://github.com/w3c/web-platform-tests/tree/master/IndexedDB/submissions and [1] contains some information about the review status of these submissions. ([2] contains general information about WebApps' testing effort.) -AB [1] http://lists.w3.org/Archives/Public/public-webapps-testsuite/2013Jul/.html [2] http://www.w3.org/wiki/Webapps/Testing -- Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix. – Linus Tordvals, creador del sistema operativo Linux
