Re: Web Storage & SQL

[Robert Sayre]

On Fri, Apr 10, 2009 at 6:13 PM, Ian Hickson  wrote:
>
> There are many use cases, e.g. Google is interested in this to enable its
> applications to be taken offline. We recently released offline GMail using
> this SQL backend; one could easily imagine other applications like
> Calendar, Reader, Docs&Spreadsheets, etc, supporting offline mode.

These are so obfuscated they are basically closed source applications.
I have no problem with that, but their existence doesn't tell us that
much, since it's hard to see how they're actually using the proposed
API.

-- 

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

Re: Web Storage & SQL

[Nikunj Mehta]

On Apr 10, 2009, at 3:13 PM, Ian Hickson wrote:


On Fri, 10 Apr 2009, Nikunj Mehta wrote:


Can someone state the various requirements for Web Storage? I did not
find them enunciated anywhere.


There's only one requirement that I know of:

* Allow Web sites to store structured data on the client.

There are many use cases, e.g. Google is interested in this to  
enable its
applications to be taken offline. We recently released offline GMail  
using

this SQL backend; one could easily imagine other applications like
Calendar, Reader, Docs&Spreadsheets, etc, supporting offline mode. A  
while

back we released a demo of Reader using Gears' SQL database.


Last time I tried this trick I was asked to come back with more  
precise use cases [1]. Then I put together more detailed use cases  
[2], and even those were not considered to be written precisely  
enough. So it looks like the bar for what constitutes a use case or  
requirement seems to be quite high.



But we would rather use a standard API than rely on Gears.


I think if we are serious about building a good foundation for local  
persistence, then we should have more precise requirements for Web  
Storage. Otherwise, we risk prematurely standardizing some dialect of  
SQL supported by SQLite as "Web Storage".


Here's a compilation of requirements from what I have read in this  
thread and other work I have done in this area:
data items may be small and record like objects or large and document  
like.

Must support data items that are hundreds of megabytes or tens of bytes.
Must support data items that are binary or text
Should support sending and receiving large data items directly over  
the network without bringing them into memory.
handle large data sets efficiently; in particular, it should not  
require that the entire data set fit into memory at one time,

execute queries over the entire dataset,
queries may involve full-text search, parameterized or query by  
example search, or a combination

queries provide operators for all JavaScript data types
queries permit user defined functions
integrate well with JavaScript.
to enable offline operation, permit a local view over or update of  
data that may be remotely mastered

integrate well with Web - MIME types, URL, and HTTP methods
not require a rigid schema definition and support decentralized schema  
evolution

Application data may only be accessed by authorized users

[1] http://lists.w3.org/Archives/Public/public-webapps/2008AprJun/0079.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2008OctDec/0104.html

Re: Web Storage & SQL

[Ian Hickson]

On Fri, 10 Apr 2009, Nikunj Mehta wrote:
> 
> Can someone state the various requirements for Web Storage? I did not 
> find them enunciated anywhere.

There's only one requirement that I know of:

* Allow Web sites to store structured data on the client.

There are many use cases, e.g. Google is interested in this to enable its 
applications to be taken offline. We recently released offline GMail using 
this SQL backend; one could easily imagine other applications like 
Calendar, Reader, Docs&Spreadsheets, etc, supporting offline mode. A while 
back we released a demo of Reader using Gears' SQL database. But we would 
rather use a standard API than rely on Gears.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: Web Storage & SQL

[Boris Zbarsky]

Nikunj Mehta wrote:
One clear problem identified despite these examples is that we do not 
have a precise enough spec for the query language to make truly 
independent interoperable implementations possible.


There are several different query languages that can be interoperably 
implemented - Lucene provides one example, and Couch DB is another. What 
makes you say that a truly interoperable implementation is not possible?


What Maciej said is that truly interoperable implementations of the spec 
as currently written are not possible.  This is so because it doesn't 
define the query language at all other than saying it's "SQL".


This says nothing about implementing a spec that actually specifies a 
query language.  (I don't consider "SQL" to be specification of a 
well-defined query language, for reasons raised earlier in this thread.)


Can someone state the various requirements for Web Storage? I did not 
find them enunciated anywhere.


This would be very welcome, yes.

-Boris

Re: Web Storage & SQL

[Boris Zbarsky]

Maciej Stachowiak wrote:
I don't think this one point should be decisive by itself. But I don't 
think it should be given zero weight either.


Agreed (in case that wasn't clear).

Note that one of the clients in question is the offline-enabled mobile 
version of GMail. I think this demonstrates that the SQL-based Database 
Storage can serve the needs of an advanced and polished Web application. 
In addition, we have a rough demonstration that it is practically 
implementable in a modern Web engine.


Both quite true.

One clear problem identified despite these examples is that we do not 
have a precise enough spec for the query language to make truly 
independent interoperable implementations possible. It seems to me that 
significantly redesigning database storage is not necessary to address 
this.


Yes; the two issues are somewhat orthogonal, in my opinion.

-Boris

Re: CfC: FPWD of Server-Sent Events, Web Sockets API, Web Storage, and Web Workers; deadline April 10

[Ian Hickson]

On Fri, 10 Apr 2009, Nikunj Mehta wrote:
>
> Here's what Oracle would like to see in the abstract:
> 
> This specification defines two APIs for persistent data storage in Web 
> clients: one for accessing key-value pair data and another for accessing 
> structured data.

Done.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: Web Storage & SQL

[Nikunj Mehta]

On Apr 10, 2009, at 1:53 PM, Maciej Stachowiak wrote:

One clear problem identified despite these examples is that we do  
not have a precise enough spec for the query language to make truly  
independent interoperable implementations possible.


There are several different query languages that can be interoperably  
implemented - Lucene provides one example, and Couch DB is another.  
What makes you say that a truly interoperable implementation is not  
possible? Why does the query language have to be SQL?


It seems to me that significantly redesigning database storage is  
not necessary to address this. "X is underspecified so let's do Y or  
Z instead" is not a very strong argument in my opinion. Another  
issue raised is that a different database model (OODB for instance)  
may work better for content authors. I would say we do not have very  
compelling evidence yet that such a design would be better, or that  
it could meet the various requirements, and we do not even have a  
concrete strawman proposal that we could start evaluating.


Can someone state the various requirements for Web Storage? I did not  
find them enunciated anywhere.

Re: Web Storage & SQL

[Maciej Stachowiak]

On Apr 9, 2009, at 5:38 PM, Boris Zbarsky wrote:


Maciej Stachowiak wrote:
I agree that "no such thing as standard SQL" (or rather the fact  
that implementations all have extensions and divergences from the  
spec) is a problem. But I am not sure inventing a brand new query  
language and database model as proposed by Vlad is a good solution  
to this problem.


That's fine; I'm not sure of that either.  I have no particular  
opinion on that question, in fact.


1) Applications are starting to be deployed which use the SQL-based  
storage API, such as the mobile version of GMail. So it may be too  
late for us to remove SQL storage from WebKit entirely.


This is a price of early adoption, sure.

If we want this content to interoperate with non-WebKit-based user  
agents, then we will ultimately need a clear spec for the SQL  
dialect to use, even if we also added an OODB or a relational  
database using some other query language.


That's true, but it's not a given that we want this content to  
interoperate as-is.  Early adopters of known in-flux technologies  
typically realize that they might have to make changes; if a  
different data storage API is decided on, or if the subset of SQL  
that's decided on doesn't match what these apps are using, then  
they'll need to change.


So while I agree that it might be difficult for Webkit to remove the  
SQL support it shipped as soon as some other approach is decided on  
(if that even happens), it doesn't follow that other UAs would need  
to ship SQL support at that point.


There are strong arguments for not breaking existing content, of  
course, but there are also strong arguments for not having  
experimental implementations of early drafts completely dictate the  
standardization process.


I don't think this one point should be decisive by itself. But I don't  
think it should be given zero weight either. I do think the existence  
of an implementation of the current spec and Web content using it  
somewhat raises the burden of proof on anyone proposing a redesign.


Note that one of the clients in question is the offline-enabled mobile  
version of GMail. I think this demonstrates that the SQL-based  
Database Storage can serve the needs of an advanced and polished Web  
application. In addition, we have a rough demonstration that it is  
practically implementable in a modern Web engine.


One clear problem identified despite these examples is that we do not  
have a precise enough spec for the query language to make truly  
independent interoperable implementations possible. It seems to me  
that significantly redesigning database storage is not necessary to  
address this. "X is underspecified so let's do Y or Z instead" is not  
a very strong argument in my opinion. Another issue raised is that a  
different database model (OODB for instance) may work better for  
content authors. I would say we do not have very compelling evidence  
yet that such a design would be better, or that it could meet the  
various requirements, and we do not even have a concrete strawman  
proposal that we could start evaluating.


Regards,
Maciej

Re: CfC: FPWD of Server-Sent Events, Web Sockets API, Web Storage, and Web Workers; deadline April 10

[Nikunj Mehta]

Just a clarification about the charter...

On Apr 10, 2009, at 9:50 AM, Arthur Barstow wrote:

Regarding a WG Note, that doesn't seem appropriate in this case  
since the WG's plan of record (Charter) is to create a  
Recommendation for this spec.


The charter [1] includes "Offline APIs and Structured Storage for  
enabling local access to Web application resources when not connected  
to a network."


This is at variance to the abstract of the current editor's draft [2]  
which states in its abstract that the spec includes and API for "a  
database system with a SQL frontend." (emphasis mine)


The former does not imply the latter and is the cause for Oracle's  
objection to the current draft.


Respectfully,
Nikunj

[1] http://www.w3.org/2008/webapps/charter/
[2] http://dev.w3.org/html5/webstorage/

Re: CfC: FPWD of Server-Sent Events, Web Sockets API, Web Storage, and Web Workers; deadline April 10

[Nikunj Mehta]

Hi Art,

Oracle conditionally supports the publishing this draft as FPWD  
provided that the abstract is worded appropriately. The reason to  
clarify the abstract is so that the WG doesn't build an implicit  
expectation that it will /only/ produce a SQL-based API in Web Storage.


Here's what Oracle would like to see in the abstract:

This specification defines two APIs for persistent data storage in Web  
clients: one for accessing key-value pair data and another for  
accessing structured data.


Some developers around the world have assumed, without justification,  
that SQL is /the/ model of data access that will be supported inside  
the browser, e.g., Maciej expressing an expectation about SQLite [1].  
This is because of the history of this draft and I hope we can do  
something to temper that expectation at an early enough stage.


Nikunj

[1] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0133.html

On Apr 10, 2009, at 9:50 AM, Arthur Barstow wrote:


Hi Nikunj,

On Apr 10, 2009, at 10:42 AM, ext Nikunj Mehta wrote:

Oracle does not support the substance of the current Web Storage  
draft

[1][2][3]. This is a path-breaking change to the Web applications
platform and rushing such a major change without substantive
consideration of alternatives is not in its own best interest. Oracle
does not see it fit to advance the current draft along the
recommendation track

Still, we believe that the working group will benefit greatly from  
the

wide review of this draft. Has the chair exhausted all such
alternatives such as Working Group Note? At the very least the status
needs to be clear about the purpose of publishing the document. A
boilerplate status is not appropriate since there are important
concerns about the technique used for structured storage in the  
draft.


I agree it would be good to get broad review of the proposed FPWD  
and the formal publication will trigger a related note on both  
w3.org and the weekly Public newsletter.


Please note there is certainly a precedence for a WG to not have  
unanimous agreement regarding the entire "substance" of a FPWD.


Regarding a WG Note, that doesn't seem appropriate in this case  
since the WG's plan of record (Charter) is to create a  
Recommendation for this spec.


-Regards, Art Barstow




Nikunj Mehta

[1] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0131.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0136.html
[3] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0137.html

On Apr 2, 2009, at 12:59 PM, Arthur Barstow wrote:


This is a Call for Consensus (CfC) to publish the First Public
Working Draft of the specs below.

As with all of our CfCs, positive response is preferred and
encouraged and silence will be assumed to be assent. The deadline
for comments is April 10.

-Regards, Art Barstow


Begin forwarded message:


From: ext Ian Hickson 
Date: April 1, 2009 6:22:40 PM EDT
To: "public-webapps@w3.org" 
Subject: Request for FPWD publication of Server-Sent Events, Web
Sockets API,  Web Storage, and Web Workers
Archived-At: 

Re: HTML5 Origin (aka IETF Origin draft)

[Adam Barth]

This is to support things like data URLs that can't be represented as
a (scheme, host, port) tuple.

Adam


On Thu, Apr 9, 2009 at 9:48 AM, Bil Corry  wrote:
> I wanted to clarify something in the IETF Origin draft[1], which is now going 
> to serve as the basis for HTML5's Origin.
>
> Section 5 requires that when a user agent provides the Origin header, it must 
> either send "null" or the ASCII serialization of the origin.  ASCII 
> serialization (and Unicode serialization) stipulates that if an origin is not 
> a scheme/host/port tuple, then it must return "null".  Section 2 allows 
> implementations to define other types of origins in addition to the 
> scheme/host/port tuple.  So my question is, if a user agent defines another 
> type of origin, but is required to send "null" for it in the Origin header, 
> is there some other use for defining other types of origins?
>
>
> - Bil
>
> [1] http://www.ietf.org/internet-drafts/draft-abarth-origin-00.txt
>
>
>

Re: CfC: FPWD of Server-Sent Events, Web Sockets API, Web Storage, and Web Workers; deadline April 10

[Arthur Barstow]

Hi Nikunj,

On Apr 10, 2009, at 10:42 AM, ext Nikunj Mehta wrote:


Oracle does not support the substance of the current Web Storage draft
[1][2][3]. This is a path-breaking change to the Web applications
platform and rushing such a major change without substantive
consideration of alternatives is not in its own best interest. Oracle
does not see it fit to advance the current draft along the
recommendation track

Still, we believe that the working group will benefit greatly from the
wide review of this draft. Has the chair exhausted all such
alternatives such as Working Group Note? At the very least the status
needs to be clear about the purpose of publishing the document. A
boilerplate status is not appropriate since there are important
concerns about the technique used for structured storage in the draft.


I agree it would be good to get broad review of the proposed FPWD and  
the formal publication will trigger a related note on both w3.org and  
the weekly Public newsletter.


Please note there is certainly a precedence for a WG to not have  
unanimous agreement regarding the entire "substance" of a FPWD.


Regarding a WG Note, that doesn't seem appropriate in this case since  
the WG's plan of record (Charter) is to create a Recommendation for  
this spec.


-Regards, Art Barstow




Nikunj Mehta

[1] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/ 
0131.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/ 
0136.html
[3] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/ 
0137.html


On Apr 2, 2009, at 12:59 PM, Arthur Barstow wrote:


This is a Call for Consensus (CfC) to publish the First Public
Working Draft of the specs below.

As with all of our CfCs, positive response is preferred and
encouraged and silence will be assumed to be assent. The deadline
for comments is April 10.

-Regards, Art Barstow


Begin forwarded message:


From: ext Ian Hickson 
Date: April 1, 2009 6:22:40 PM EDT
To: "public-webapps@w3.org" 
Subject: Request for FPWD publication of Server-Sent Events, Web
Sockets API,  Web Storage, and Web Workers
Archived-At: 

Re: CfC: FPWD of Server-Sent Events, Web Sockets API, Web Storage, and Web Workers; deadline April 10

[Nikunj Mehta]

Oracle does not support the substance of the current Web Storage draft  
[1][2][3]. This is a path-breaking change to the Web applications  
platform and rushing such a major change without substantive  
consideration of alternatives is not in its own best interest. Oracle  
does not see it fit to advance the current draft along the  
recommendation track


Still, we believe that the working group will benefit greatly from the  
wide review of this draft. Has the chair exhausted all such  
alternatives such as Working Group Note? At the very least the status  
needs to be clear about the purpose of publishing the document. A  
boilerplate status is not appropriate since there are important  
concerns about the technique used for structured storage in the draft.


Nikunj Mehta

[1] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0131.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0136.html
[3] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/0137.html

On Apr 2, 2009, at 12:59 PM, Arthur Barstow wrote:

This is a Call for Consensus (CfC) to publish the First Public  
Working Draft of the specs below.


As with all of our CfCs, positive response is preferred and  
encouraged and silence will be assumed to be assent. The deadline  
for comments is April 10.


-Regards, Art Barstow


Begin forwarded message:


From: ext Ian Hickson 
Date: April 1, 2009 6:22:40 PM EDT
To: "public-webapps@w3.org" 
Subject: Request for FPWD publication of Server-Sent Events, Web  
Sockets API,  Web Storage, and Web Workers
Archived-At: 



The following drafts are relatively stable and would benefit  
greatly from

wider review:

  Server-Sent Events
  http://dev.w3.org/html5/eventsource/

  The Web Sockets API
  http://dev.w3.org/html5/websockets/

  Web Storage
  http://dev.w3.org/html5/webstorage/

  Web Workers
  http://dev.w3.org/html5/workers/

Assuming there is consensus in the working group to do so, could we
publish these as First Public Working Drafts?

Cheers,
--
Ian Hickson   U+1047E) 
\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _ 
\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'-- 
(,_..'`-.;.'

Re: Web Storage & SQL

[Ivan Enderlin @ Hoa]

Hi :-),

My opinion about the web storage & SQL is splitted in two parts.

On the one hand, making a subset of SQL instructions is very difficult. 
Each of us knows that and no-one would say the contrary.
On the other hand, I have made an ORM in the past, build on two layers. 
The first one is a database abstract layer (DAL) that enables user to 
choose the RDBMS which he would like to use (what the /type/ argument of 
the openDatabase() function expects). The second one is an 
object-relational mapping layer (ORM) that builds SQL query through an 
object interface. The SQL instructions builder is delagated in many 
classes (with encapsulation): one class per supported database 
(typically, one class per RDBMS).


A relevant and obvious compromise would be to make an ORM, with an API 
interface as jLinq proposes. It abstracts the database, proposes a 
standard API, enables multiple RDBMS in use etc.

Thoughts ?

Best regards.

--
Ivan Enderlin
Developper of Hoa Framework
http://hoa-project.net/