Re: How to standardize new Offline Web app features? [Was Re: Offline Web Applications status]

2011-04-04 Thread Ian Hickson
On Sat, 2 Apr 2011, Arthur Barstow wrote:
 
 Perhaps subscribers to both lists (Mike Smith, Maciej, Hixie) could 
 provide some guidance on which list to use for Offline Web applications 
 (again, I'm OK with public-webapps) and which Bugzilla product/component 
 to use to file feature requests for Offline Web apps.

If one wants to send e-mail and be guaranteed a response, then the 
wha...@whatwg.org list will be most effective.

If one wants to discuss the issue within the W3C context, then 
public-h...@w3.org is the list most appropriate for the current spec text.

If one wants to file a bug and be guaranteed a response, then open the 
WHATWG HTML Standard, click on the relevant section, and type the comment 
in the feedback form at the bottom right. It'll automatically pick the 
right component and so forth. I don't remember what the component is 
called off-hand, probably HTML spec or some such. You can also file a 
bug using the form at the top of the W3C draft that the feature is defined 
in, which is currently the HTML5 draft; that should also pick the right 
component but doesn't give me as much context. You can also file the bug 
directly in Bugzilla if you can work out which component to use.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'



Re: How to standardize new Offline Web app features? [Was Re: Offline Web Applications status]

2011-04-04 Thread Charles Pritchard

On 4/3/2011 11:20 PM, Ian Hickson wrote:

On Sat, 2 Apr 2011, Arthur Barstow wrote:

Perhaps subscribers to both lists (Mike Smith, Maciej, Hixie) could
provide some guidance on which list to use for Offline Web applications
(again, I'm OK with public-webapps) and which Bugzilla product/component
to use to file feature requests for Offline Web apps.

If one wants to send e-mail and be guaranteed a response, then the
wha...@whatwg.org list will be most effective.

If one wants to discuss the issue within the W3C context, then
public-h...@w3.org is the list most appropriate for the current spec text.


I'd prefer to see discussion continue on public-webapps . I'm not 
against cross posting.


Offline apps are the first and only instance of an 'installable' 
applications implementation across vendors.


At some point, I hope to see some discussion about an installation manifest,
for apps which require privileges across origins and/or other enhanced 
access.


This seems to be the direction things are going (from Google's Chrome):
link rel=chrome-application-definition href=arbitrary-manifest.json
chrome.app.install()

That scheme allows the author to provide a user with a choice, prior
to prompting them for permissions.

For an application vendor, that's mighty helpful, as some users reject:
Do you want to grant this website access to 
EVERYTHING--or-it-will-not-work,

but accept: Do you want to grant this website access to this one domain.

Google Chrome
http://code.google.com/intl/en-US/chrome/apps/docs/no_crx.html
chrome.app.install()

Mozilla Firefox
https://mozillalabs.com/blog/2011/03/first-developer-release-of-web-apps-project/
https://developer.mozilla.org/en/OpenWebApps/The_Manifest
navigator.apps.install()

Those two vendors are converging.

link rel seems more appropriate for icons. Apple is taking that route.

HTML manifest files seem appropriate for requesting same-origin exceptions,
which is part of why I'm keeping an eye on this thread.

Those .install() calls are still a good idea, as vendors have their own 
unique extensions and distribution mechanisms.


I don't expect to get background pages / content pages into a web apps 
spec any time soon.
They're the same concept across Firefox/Chrome/Safari, but there are a 
lot of surrounding details.



-Charles



Please use public-test-infra list for testharness.js discussions [Was: RfC: WebApps Testing Process]

2011-04-04 Thread Arthur Barstow
Hi All - on March 31 I submitted a rough draft proposal to the WebApps 
WG related to testing processes [1]. That proposal includes re-using 
testharness.js which is used by at least the HTML WG and Web Performance 
WG. Garrett Smith submitted some comments about this harness and those 
comments are below.


Going forward, please use the following mail list for testharness.js 
discussions:


  public-test-in...@w3.org
  http://lists.w3.org/Archives/Public/public-test-infra/

Bugs for this harness should be submitted to the Testing product and 
testharness.js component:


  http://www.w3.org/Bugs/Public/describecomponents.cgi?product=Testing

James Graham (author of testharness.js) is default assignee of this 
component.


-Art Barstow

[1] http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1086.html

 Original Message 
Subject:Re: RfC: WebApps Testing Process
Date:   Sun, 3 Apr 2011 16:45:02 -0700
From:   ext Garrett Smith dhtmlkitc...@gmail.com
To: Arthur Barstow art.bars...@nokia.com
CC: public-webapps public-webapps@w3.org



On 3/31/11, Arthur Barstowart.bars...@nokia.com  wrote:


 4. http://www.w3.org/2008/webapps/wiki/Harness - short intro to the
 harness which is a reuse of the one used by the HTML WG (created and
 maintained by James Graham)


I'd rather see the `format_value` function broken up. It makes
non-standard expectations of host objects (`val`) and that there is a
global `Node` object. Which standard requires that?

Instead of making decisions based on what is on the object's prototype
chain, It is safer to make a more direct inference.

However, taking a step back, I want to know why the function is so generalized.

I see that the function `format_value` is called by `assert_equals`
and by itself, recursively. It is expected to take all of number,
string, and Node because assert_equals pushes down the requirement to
be generalized. I would rather see this functionality broken up so
that assertions about Node objects are passed Nodes, and then the
formatting can be in format_node, or stringify_node, etc. And it can
get worse when you have more object types or subtypes, such as any of
the various DOM collections.

I've attacked this `assert_*` multiplicity variance by using what is
called constraints in NUnit. Essentially, encapsulate the parts
that vary. In javascript, a constraint can be written very easily as a
function. That will also allow for cleanup of the messiness of `-0`
and NaN's and their accompanying obsolete comments.





Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Joran Greef
On 04 Apr 2011, at 4:39 PM, Jonas Sicking wrote:

 Hence it would still be the case that we would be relying on the
 SQLite developers to maintain a stable SQL interpretation...

SQLite has a fantastic track record of maintaining backwards compatibility.

IndexedDB has as yet no track record, no consistent implementations, no 
widespread deployment, only measurably poor performance and a lukewarm indexing 
and querying API.

If anything it's the other way round. You have yet to convince developers that 
IndexedDB will be faster, more stable, more powerful, more memory efficient 
than SQLite and with better test coverage at that.


Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
This is ignoring the possibility that something like RelationalDB could be
used, where a well defined common subset of SQL can be used (and I use
well-defined in the formal sense). This would allow a relatively thin
wrapper on top of most SQL implementations and would allow SQLite (or BDB)
to be used as the backend.

As a seasoned C++ programmer, I could even write a Firefox plugin using
XPCOM as a reference implementation using the same API as the JavaScript
RelationalDB implementation on my GitHub. Although I am not keen on putting
in the time to do this if nobody is interested.

To me is seems this thread is going in circles. RelationalDB does not have
the standardisation problem that WebSQL has, but is still a relatively thin
API layer that can be implemented over the top of a fast and well tested SQL
implementation. It is based on sound theory and research defining the
abstraction layer, and has a relationally complete API, so there should be
no need to change the core API in the development of a standard.


Cheers,
Keean.


On 4 April 2011 14:39, Jonas Sicking jo...@sicking.cc wrote:

 On Saturday, April 2, 2011, Joran Greef jo...@ronomon.com wrote:
  I am incredibly uncomfortable with the idea of putting the
  responsibility of the health of the web in the hands of one project.
  In fact, one of the main reasons I started working at Mozilla was to
  prevent this.
 
  / Jonas
 
  I agree with you. All the more reason to support both WebSQL and
 IndexedDB. It is not a case of either/or. It would be healthy to have
 competing APIs.

 Competition might be a great thing. But it doesn't address the issue
 in the least. It would still be the case that some developers would
 choose to use WebSQL, and browser makers would still have to support
 it, including support the SQL dialect it uses.

 Hence it would still be the case that we would be relying on the
 SQLite developers to maintain a stable SQL interpretation to keep a
 healthy and functional web.

 / Jonas




Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Joran Greef
On 04 Apr 2011, at 5:26 PM, Keean Schupke wrote:

 This is ignoring the possibility that something like RelationalDB could be 
 used, where a well defined common subset of SQL can be used (and I use 
 well-defined in the formal sense). This would allow a relatively thin wrapper 
 on top of most SQL implementations and would allow SQLite (or BDB) to be used 
 as the backend.

Yes, if an implementation of RelationalDB arrives which is solid and fast with 
support for set operations that would be great. The important thing is that we 
have two competing APIs (and preferably a strong API with a great track record).


Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
Yes, it already has well defined set operations. Solid is a matter of
testing by enough people (and if you wanted to try it and feed back that
would be a start). Fast should not be a problem, as the SQL database does
all the heavy lifting.

In more detail, Codd's six primitive operators are project, restrict,
cross-product, union and difference. Relations are an extension of Sets, so
intersection and difference on compatible relations behave like they would
on sets.

RelationalDB already implements the following 5 methods making it
relationally-complete. Meaning it can do anything you could possibly want to
do with relations using combinations of these 5 methods.

Relation.prototype.project = function(attributes) { // this implements
rename as well.
Relation.prototype.restrict = function(exp)
Relation.prototype.join = function(relation, exp) {
Relation.prototype.union = function() {};
Relation.prototype.difference = function() {};

Of course some things can be made easier, so the following methods, although
they can be defined in terms of the above 5, will be provided (in future
implementations) to keep user code concise and implementations thin and
fast.

// derived methods
Relation.prototype.intersection = function() {};
Relation.prototype.thetajoin = function() {};
Relation.prototype.semijoin = function() {};
Relation.prototype.antijoin = function() {};
Relation.prototype.divide = function() {};
Relation.prototype.leftjoin = function() {};
Relation.prototype.rightjoin = function() {};
Relation.prototype.fulljoin = function() {};


We also hope to provide the lattice operators meet and join:

http://en.wikipedia.org/wiki/Lattice_(order)

Just these two operators can replace all 5 of Codd's primitives (including
all set operations). With just these two you can do anything that you can
with _all_ the above. Meet is actually the same as Codd's natural-join
(unfortunately terminology in different mathematical fields is not
consistent here) and Join is a generalised union operator.

See:

http://www.arxiv.com/pdf/cs/0501053v2

To see how Meet and Join can be used to construct each of the above
operators.


Cheers,
Keean.


On 4 April 2011 15:36, Joran Greef jo...@ronomon.com wrote:

 On 04 Apr 2011, at 5:26 PM, Keean Schupke wrote:

  This is ignoring the possibility that something like RelationalDB could
 be used, where a well defined common subset of SQL can be used (and I use
 well-defined in the formal sense). This would allow a relatively thin
 wrapper on top of most SQL implementations and would allow SQLite (or BDB)
 to be used as the backend.

 Yes, if an implementation of RelationalDB arrives which is solid and fast
 with support for set operations that would be great. The important thing is
 that we have two competing APIs (and preferably a strong API with a great
 track record).


Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
On 4 April 2011 15:55, Keean Schupke ke...@fry-it.com wrote:

 Yes, it already has well defined set operations. Solid is a matter of
 testing by enough people (and if you wanted to try it and feed back that
 would be a start). Fast should not be a problem, as the SQL database does
 all the heavy lifting.

 In more detail, Codd's six primitive operators are project, restrict,
 cross-product, union and difference. Relations are an extension of Sets, so
 intersection and difference on compatible relations behave like they would
 on sets.


I missed 'rename' from my list of Codd's operators. Our 'project' function
provides both project  rename, so I overlooked it.


 ...



Cheers,
Keean.


Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Mikeal Rogers
i've mostly stayed out of this thread because i felt like i'd just being fanning the flames but i really can't stay out anymore.

databases are more that SQL, always have been.

SQL is a DSL for relational database access. all implementations of SQL have a similar set of tools they implement first and layer SQL on top of. those tools tend to be a storage engine, btree, and some kind of transactional model between them. under the ugly covers, most databases look like berkeleydb and the layer you live in is just sugar on top.

creating an in-browser specification/implementation on top of a given relational/SQL story is a terrible idea. it's unnecessarily limiting to a higher level api and can't be easily extended the way a simple set of tools like IndexedDB can.

suggesting that other databases be implemented on top of SQL rather than on top of the tools in which SQL is built is just backwards to anyone who's built a database.

it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable.

i implemented a CouchDB compatible datastore on top of IndexedDB, it took me less than a week at a time when there was only one implementation that was still changing and still had bugs. it would be much easier now.

https://github.com/mikeal/idbcouch

it needs to be updated to use the latest version of the spec which is a day of work i just haven't gotten to yet.

the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case.



Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Tab Atkins Jr.
On Mon, Apr 4, 2011 at 8:07 AM, Joran Greef jo...@ronomon.com wrote:
 On 04 Apr 2011, at 4:39 PM, Jonas Sicking wrote:
 Hence it would still be the case that we would be relying on the
 SQLite developers to maintain a stable SQL interpretation...

 SQLite has a fantastic track record of maintaining backwards compatibility.

 IndexedDB has as yet no track record, no consistent implementations, no 
 widespread deployment,

It's new.


 only measurably poor performance

Ironically, the poor performance is because it's using sqlite as a
backing-store in the current implementation.  That's being fixed by
replacing sqlite.


 and a lukewarm indexing and querying API.

Kinda the point, in that the power/complexity of SQL confuses a huge
number of develoeprs, who end up coding something which doesn't
actually use the relational model in any significant way, but still
pays the cost of it in syntax.

(I found normalization forms and similar things completely trivial
when I was learning SQL, but for some reason almost every codebase
I've looked at has a horribly-structured db.  As far as I can tell,
the majority of developers just hack SQL into being a linear object
store and do the rest in their application code.  We can reduce the
friction here by actually giving them a linear object store, which is
what IndexedDB is.)

~TJ



Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
Some thoughts:

On 4 April 2011 16:10, Mikeal Rogers mikeal.rog...@gmail.com wrote:

 i've mostly stayed out of this thread because i felt like i'd just being
 fanning the flames but i really can't stay out anymore.

 databases are more that SQL, always have been.


 SQL is a DSL for relational database access. all implementations of SQL
 have a similar set of tools they implement first and layer SQL on top of.
 those tools tend to be a storage engine, btree, and some kind of
 transactional model between them. under the ugly covers, most databases look
 like berkeleydb and the layer you live in is just sugar on top.


SQL is a standard language (or API) for talking to databases. Why should a
developer need to learn a different API for each database. W3 is about
standardising APIs. SQL is just an API standardised as a DSL. It is good for
all the reasons any standard is good. Add to that the sound mathematical
theory of relational-algebra, means it has a lot going for it. Although like
any standard is has its problems, most of those seem to be where it has
deviated away from the pure relational-algebra.



 creating an in-browser specification/implementation on top of a given
 relational/SQL story is a terrible idea. it's unnecessarily limiting to a
 higher level api and can't be easily extended the way a simple set of tools
 like IndexedDB can.


Its not limiting it provides a more powerful (higher level) interface, that
allows developers to concentrate on what to do with the data not how to do
it.



 suggesting that other databases be implemented on top of SQL rather than on
 top of the tools in which SQL is built is just backwards to anyone who's
 built a database.


RelationalDB is not a database its a relational-data model.



 it's not very hard to write the abstraction you're talking about on top of
 IndexedDB, and until you do it i'm going to have a hard time taking you
 seriously because it's clearly doable.


Surely its the API that is important, not how it is implemented? You can try
the API now implemented on top of WebSQL. The API will stay the same no
matter what underlying technology is used.



 i implemented a CouchDB compatible datastore on top of IndexedDB, it took
 me less than a week at a time when there was only one implementation that
 was still changing and still had bugs. it would be much easier now.

 https://github.com/mikeal/idbcouch

 it needs to be updated to use the latest version of the spec which is a day
 of work i just haven't gotten to yet.


I am not overly impressed by CouchDB.


 the constructs in IndexedDB are pretty low level but sufficient if you know
 how to implement databases. performance is definitely an issue, but making
 these constructs faster would be much easier than trying to tweak an off the
 shelf SQL implementation to your use case.


I look at the amount of man hours that have gone into developing SQLite, and
BDB and I think, hey if its so easy to write a high performance database,
those guys must have been wasting a lot of time?


Cheers,
Keean.


Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
On 4 April 2011 16:04, Tab Atkins Jr. jackalm...@gmail.com wrote:

 On Mon, Apr 4, 2011 at 8:07 AM, Joran Greef jo...@ronomon.com wrote:
  On 04 Apr 2011, at 4:39 PM, Jonas Sicking wrote:
  Hence it would still be the case that we would be relying on the
  SQLite developers to maintain a stable SQL interpretation...
 
  SQLite has a fantastic track record of maintaining backwards
 compatibility.
 
  IndexedDB has as yet no track record, no consistent implementations, no
 widespread deployment,

 It's new.


  only measurably poor performance

 Ironically, the poor performance is because it's using sqlite as a
 backing-store in the current implementation.  That's being fixed by
 replacing sqlite.


  and a lukewarm indexing and querying API.

 Kinda the point, in that the power/complexity of SQL confuses a huge
 number of develoeprs, who end up coding something which doesn't
 actually use the relational model in any significant way, but still
 pays the cost of it in syntax.

 (I found normalization forms and similar things completely trivial
 when I was learning SQL, but for some reason almost every codebase
 I've looked at has a horribly-structured db.  As far as I can tell,
 the majority of developers just hack SQL into being a linear object
 store and do the rest in their application code.  We can reduce the
 friction here by actually giving them a linear object store, which is
 what IndexedDB is.)


 ~TJ


SQLite has seen really good use in the mobile app community on both iPhone
and Android. I would have thought that if we wanted the same kind of
thriving app developer community around HTML5 web-apps, taking a few leaves
from the mobile developers book would not be a bad idea?

IMHO its those kind of developers HTML5 should be trying to attract, in
addition to the current web developers.


Cheers,
Keean.


CfC: publish new WDs of File API: {Writer, Directories and System}; deadline April 11

2011-04-04 Thread Arthur Barstow
This is a Call for Consensus to publish new Working Drafts of Eric's two 
File API specs (last published about 6 months ago):


1. File API: Writer
http://dev.w3.org/2009/dap/file-system/file-writer.html

2. File API: Directories and System:
http://dev.w3.org/2009/dap/file-system/file-dir-sys.html

If you have any comments or concerns about this proposal, please send 
them to public-webapps by April 11 at the latest.


As with all of our CfCs, positive response is preferred and encouraged 
and silence will be assumed to be agreement with the proposal.


Note that Eric may update these EDs during this CfC.

(I will follow up separately with Arun and Jonas re the status and plans 
for the File API spec.)


-Art Barstow




Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
I would point out that RelationalDB is relationally complete and the api
does not depend on the sqlite spec at all.

Cheers
Keean
On Apr 1, 2011 8:58 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Fri, Apr 1, 2011 at 12:39 PM, Glenn Maynard gl...@zewt.org wrote:
 Lastly, some vendors have expressed unwillingness to embed SQLite for
 legal reasons. Embedding other peoples code definitely exposes you to
 risk of copyright and patent lawsuits. While I can't say that I fully
 agree with this reasoning, I'm also not the one that would be on the
 receiving end of a lawsuit. Nor am I a lawyer and so ultimately will
 have to defer to people that know better. In the end it doesn't really
 matter as if a browser won't embed SQLite then it doesn't matter why,
 the end result is that the same SQL dialect won't be available cross
 browser which is bad for the web.

 If SQLite was to be used as a web standard, I'd hope that it wouldn't
show
 up in a spec as simply do what SQLite does, but as a complete spec of
 SQLite's behavior.  Browser vendors could then, if their lawyers
insisted,
 implement their own compatible implementation, just as they do with other
 web APIs.  I'd expect large portions of SQLite's test suite to be
adaptable
 as a major starting point for spec tests, too.

 Have you read the WebSQL spec?

 Creating such a spec would be a formidable task, of course.

 Indeed. One that the SQL community has failed in doing so far. And
 they have a lot more experience with SQL than we do.

 / Jonas



Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Joran Greef
On 04 Apr 2011, at 6:10 PM, Mikeal Rogers wrote:

 it's not very hard to write the abstraction you're talking about on top of 
 IndexedDB, and until you do it i'm going to have a hard time taking you 
 seriously because it's clearly doable.


You assume I have not written the abstraction I am talking about on top of 
IndexedDB?

 the constructs in IndexedDB are pretty low level but sufficient if you know 
 how to implement databases. performance is definitely an issue, but making 
 these constructs faster would be much easier than trying to tweak an off the 
 shelf SQL implementation to your use case.


How exactly would you make a schema-enforcing interface faster than a stateless 
interface?

How would you implement application-managed indices on top of IndexedDB without 
being slower than SQLite?

How would you implement set operations on indices in IndexedDB without being 
slower or less memory efficient than SQLite?

How would you create an index on an existing object store in IndexedDB 
containing more than 50,000 objects on an iPad, without incurring any object 
deserialization/serialization overhead, without being an order of magnitude 
slower than SQLite, and without bringing the iPad to its knees? If you can do 
it with even one IndexedDB implementation out there then kudos and hats off to 
you. :)

I understand your point of view. I once thought the same. You would think that 
IndexedDB would be more than satisfactory for these things. The question is 
whether IndexedDB provides adequate and performant database primitives, to the 
same degree as SQLite (and of course SQL is merely an interface to database 
storage primitives, I do not recalling saying otherwise).

You can build IndexedDB on top of SQLite (as some browsers are indeed doing), 
but you cannot build SQLite on IndexedDB.


Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Mikeal Rogers


  
  



  

  
  

  

Joran Greef
  April 4, 2011 April 4, 201110:18 AM
  

  
  
On 04 Apr 2011, at 6:10 PM, Mikeal Rogers wrote:



  it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable.




You assume I have not written the abstraction I am talking about on top of IndexedDB?



  the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case.




How exactly would you make a schema-enforcing interface faster than a stateless interface?

How would you implement application-managed indices on top of IndexedDB without being slower than SQLite?

  

this assumes your indexes can be describe well in sqlite and that
you want them generated at write time. one of the performance
optimizations couchdb makes is to generate secondary indexes at read
time which is much more efficient as you can batch the processing
and do a single bulk transaction which limits IO. if you need,
exactly, the indexes sqlite provides then i'm sure they will be well
optimized in sqlite, but for everyone else.

  

How would you implement set operations on indices in IndexedDB without being slower or less memory efficient than SQLite?

  

you can do pretty much anything if you write a good abstraction and
leverage transactions. if it's too slow complain to the vendor and
it'll get improved, that's how we've been improving performance in
the browser for many, many years and it seems to be working.

  

How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :)

  

the biggest bottleneck here in the current implementation would be
the transaction overhead on a database this size, which is because
of performance problems in sqlite which is underlying the
implementation. sqlite can't fix this, it's currently the problem.
the object serialization is not a huge performance issue,
performance issues in databases are almost always do to IO or
transaction locks.

  
I understand your point of view. I once thought the same. You would think that IndexedDB would be more than satisfactory for these things. The question is whether IndexedDB provides adequate and performant database primitives, to the same degree as SQLite (and of course SQL is merely an interface to database storage primitives, I do not recalling saying otherwise).

  

sqlite doesn't expose it's primitives well, it exposes it's
abstractions. berkeleydb exposes it's primitives well, and you could
make the case that it would be a better thing to standardize on,
although i wouldn't think that would be a great idea either.

  

You can build IndexedDB on top of SQLite (as some browsers are indeed doing), but you cannot build SQLite on IndexedDB.
  

you should most definitely be able build sqlite on top of IDB, there
would be a performance penalty of course, which we can address, but
you should be able to do it. if you can't then we need to extend the
specification.

  

  
  

  

Keean Schupke
  April 4, 2011 April 4, 20118:55 AM
  

  
  
Yes, it already has well defined set operations. Solid is a
matter of testing by enough people (and if you wanted to try it
and feed back that would be a start). Fast should not be a
problem, as the SQL database does all the heavy lifting.

  
  In more detail, Codd's six primitive operators are
project, restrict, cross-product, union and difference.
Relations are an extension of Sets, so intersection and
difference on compatible relations behave like they would on
sets.
  


RelationalDB already implements the following 5 methods
  making it relationally-complete. Meaning it can do
  anything you could possibly want to do with relations
  using combinations of these 5 methods.
  
 
  

Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Joran Greef
On 04 Apr 2011, at 6:04 PM, Tab Atkins Jr. wrote:

 It's new.

Do you think it would be wise then to advocate doing away with SQLite before 
IndexedDB has had a chance to prove itself? Surely two competing APIs would be 
the fastest way to bring IndexedDB up to speed?

 Ironically, the poor performance is because it's using sqlite as a
 backing-store in the current implementation.  That's being fixed by
 replacing sqlite.

Yes I am aware of this. There are some design flaws in IndexedDB. For instance, 
it does not regard objects as opaque (as would a typical key-value store), 
which means that creating an index on an existing object store would require 
deserializing/serializing every object therein. Doing that for 50,000 objects 
on an iPad would be breathtaking.

I have written object stores on top of SQLite and they are already an order of 
magnitude faster than IndexedDB with a more powerful and memory efficient API 
to boot.

 Kinda the point, in that the power/complexity of SQL confuses a huge
 number of develoeprs, who end up coding something which doesn't
 actually use the relational model in any significant way, but still
 pays the cost of it in syntax.

I was not referring to SQL but to the underlying primitives exposed through the 
SQL interface. For example, set operations on indices, or the ability to index 
objects with array values.



Re: Web SQL Database specification

2011-04-04 Thread Arthur Barstow
[ I don't know why this e-mail dated 31-March-2011 just showed up in my 
Inbox today ... ]


On Mar/31/2011 9:25 AM, ext Sylvain GREZE wrote:

Hello there,

I write this email to let you know about our surprise when we saw that 
the Web SQL Database is no longer part of the html5 specification.


Having experienced the Google Gears plugin, which implement the WebSQL 
Database, I can tell you that having SQL both on the client and the 
server side, even if the SQL is not exactly the same, makes it very 
practical to deal with synchronisation problems between client DB and 
Server DB (for example you execute an update statement on the client 
side, and then you report almost as is the statement on the server 
side).

Dealing with Indexed DB will make this kind of stuff very hard...

Is this part of the spec really going to disapear?


Sylvain - I don't understand your question but please note the Web 
Applications WG agreed in November 2010 to stop work on Web SQL 
Database. Please see [1] for the related e-mail thread.


-Art Barstow

[1] http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/0451.html


Regards,

Sylvain Greze
Développeur Informatique

ALPES CONTROLES
Développement informatique
3, impasse des prairies
74940 ANNECY LE VIEUX
Tél. 04.50.64.99.57 - Fax. 04.50.64.23.83

www.alpes-controles.fr





Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

2011-04-04 Thread Keean Schupke
On 4 April 2011 22:55, Aryeh Gregor simetrical+...@gmail.com wrote:

 On Fri, Apr 1, 2011 at 2:39 PM, Jonas Sicking jo...@sicking.cc wrote:
  There are several reasons why we don't want to rely exclusively on
  SQLite, other than solely W3C formalia.
 
  First of all, what should we do once the SQLite team releases a new
  version which has some modifications in its SQL dialect? We generally
  always need to embed the latest version of the library since it
  contains critical bug fixes, however SQLite makes no guarantees that
  it will always support exactly the same SQL statements. . . .

 These are good reasons, and I have no problem with them.  SQLite is
 designed with very different compatibility and security needs than the
 web platform has, and its performance goals might be different in some
 respects as well.  There are various ways that you could address this
 short of making up something completely different, but I'm not sure
 whether it would be a good idea.

 Anyway, I didn't intend to reignite this whole discussion.  The
 decision has been made, now we get to see what comes of it.

 On Mon, Apr 4, 2011 at 11:07 AM, Joran Greef jo...@ronomon.com wrote:
  SQLite has a fantastic track record of maintaining backwards
 compatibility.

 SQLite does not face anything close to the compatibility requirements
 that web browsers face.  There are perhaps billions of independent web
 pages, which don't have any control over what browser versions they're
 being run in.  These pages are expected to work in all browsers even
 if they were written ten years ago and no one has looked at them
 since, and even if they were written incompetently.  Just because
 something has an excellent compatibility track record by the standards
 of application libraries doesn't mean it's compatible enough for the
 web.


Something like RelationalDB gives you the power of a relational-db with no
dependence on a specific implementation of SQL, so it would be compatible
enough for the web.  It fixes all the problems with the standardisation of
WebSQL that have been talked about so far.  I think it would find no
technical issues that block its standardisation.  As a high level DB API it
does not need all the low-level features of IndexedDB, so its API can be
much simpler and cleaner. RelationalDB can at least be provided as a library
on top of IndexedDB, and it can use WebSQL where it is supported. My concern
with the library approach is performance when implemented on top of
IndexedDB.


Cheers,
Keean.