Pouchdb would use IDB in
Firefox. Replacing SQLite with a key/value store abstraction on top of
SQLite makes no sense. You are guaranteed to regress perf, footprint,
etc.
I think using our own storage as a pouchdb backend is a lot less risky,
likely easier.
Taras
Friday, July 26,
2013 13:30
Nobody is asking to move all core services. We are talking about
history, bookmarks and passwords, for now.
Andreas
Friday, July 26,
2013 13:27
I
did some experiments with pouchdb inside Gecko, using passwords and
bookmarks
as my two main use cases. As Dale pointed out, if we use a
shadow
database and then replicate the shadow database, we also have to
replicate
from the current storage backends (password, bookmark places,
history
places) into the local couchdb/pouchdb (I will use couchdb as an
abstraction
anything that looks like a couchdb interface/api/db). I
wrote a
draft of that code for passwords and its not really trivial. Its
probably
roughly as complex as doing the same and using a custom
replication
using the couchdb wire protocol into a remote couchdb on the
server.
Using
a custom replication using CouchDB wire protocol has a benefit of not
needing to convert the entire application to use CouchDB as the backing
datastore. Given that the effort could be "roughly as complex", I'd
rather explore the custom replication and not trade lock-in to one
datastore (SQLite) for another (CouchDB).
An
alternative to both would be to simply swap our current storage
backends
for couchdb, in particular because some of our current storage
code
is atrocious anyway (history is better, bookmarks are bad, password
backend
is horrific and totally synchronous). The main problem we are
running
into here is that the password manager for example is completely
synchronous.
The API is of the form x = searchLogin(needle). This can't
be
changed easily because there is layers and layers of APIs around this
API
all across the codebase that are also synchronous. Rewriting all of
that
to take a result callback would take at least a couple months. Its
righteous
work, but it takes time, and it doesn't really solve a
performance
problem because this code path is very rarely taken. The
solution
old sync took (make a synchronous call, spin the event loop
while
waiting, continue) is really terrible and we should not go down
that
path.
Let's not underestimate the time required to
move from SQLite to CouchDB for all core data in Firefox.
Friday, July 26,
2013 11:53
I did some experiments with pouchdb inside Gecko, using passwords
and
bookmarks as my two main use cases. As Dale pointed out, if we use a
shadow database and then replicate the shadow database, we also have to
replicate from the current storage backends (password, bookmark places,
history places) into the local couchdb/pouchdb (I will use couchdb as an
abstraction anything that looks like a couchdb interface/api/db). I
wrote a draft of that code for passwords and its not really trivial. Its
probably roughly as complex as doing the same and using a custom
replication using the couchdb wire protocol into a remote couchdb on the
server.
An alternative to both would be to simply swap our current storage
backends for couchdb, in particular because some of our current storage
code is atrocious anyway (history is better, bookmarks are bad, password
backend is horrific and totally synchronous). The main problem we are
running into here is that the password manager for example is completely
synchronous. The API is of the form x = searchLogin(needle). This can't
be changed easily because there is layers and layers of APIs around this
API all across the codebase that are also synchronous. Rewriting all of
that to take a result callback would take at least a couple months. Its
righteous work, but it takes time, and it doesn't really solve a
performance problem because this code path is very rarely taken. The
solution old sync took (make a synchronous call, spin the event loop
while waiting, continue) is really terrible and we should not go down
that path.
The problem to use one database is not actually to change the
password
manager's database to a couchdb. Thats trivial. The problem is that
couchdb is async, and we can't wait for the results of db queries from
within the password manager API.
We have a couple different options here. We could put couchdb into a
worker and then do a blocking call every time we talk to couchdb and
block the main thread from within the password manager API. That ...
well really blows, but it would work. But adding more code to the code
base that intentionally blocks the main thread will probably make Taras
hate me for life, even if its a fairly rare case. The other option we
have is to add a sync mozStorage backend to pouchdb, but thats rough as
shitty.
I don't think there are any clear winners here, but this is
definitely
an unpleasant problem.
Andreas
|
