Brett Wilson wrote:
Hi everybody,
I'm working on replacing a lot of Firefox's storage systems to use
sqlite. It has been going well so far except for one issue.
The database file is stored in the user's Mozilla profile directory.
In companies and Universities using Linux, this directory is often
hosted over the network. In these cases, sqlite performance can be
quite poor.
I am aware that writing performance can be bad in these cases, but we
don't do too many writes. I am mostly concerned about read
performance, since a lot of this stuff (history, bookmarks) drive the
UI. The lag, even over a fast network, can be noticable. I am also
concerned about file locking, since the documentation does not
recommend using files over the network.
Can anybody suggest what to do about this problem? This is not
something that can be avoided, since some people will have this
configuration and will not have any say about it. Firefox must perform
reasonably in these cases.
One thing that could work in our favor is that Mozilla already does
locking on the profile, so access will be restricted to our one
process. Is there anything that we can do to take advantage of this to
avoid having to query the file for reads even when the content is
cached? It looks like we will have multiple database connections from
this process.
I will work on minimizing the number of queries in the common cases,
but any little bit of perfomance will help here.
Thanks,
Brett
Brett,
If you can't tolerate the delays accessing the database over the
network, can you make a copy of the database in a temp directory on the
local machine on startup. If you copy the file after you lock the
profile it should be safe to copy down to local storage. Then use the
local database while the application runs, and finally copy the database
back to the server, if it has been modified, when the application quits.
If you want to be more resistant to data loss due to power failures or
program crashes, you could copy the modified database back to the server
after every write. This would be slower, but you say you are not as
concerned about write performance.
