Again asking the same question :
a. Does WAL during an insert/update in the log file do any internal
search/sort
and then insert/update to the log or
b. Just appends the WAL log with the incoming insert/update entry, thus
keeping the
writes sequential and during a checkpoint (manual or automatic) does the
merging with the actual tables,
which as you pointed out will have to search/sort causing alot of random
disk access?
c. Also we don't read the db in file IO path. The read are for scheduled data
maintenance scanners,
thus making the random read of disk a occasional event.
Just trying to assess if my understanding of sqlite WAL is correct.
~Joe
----- Original Message -----
From: "Valentin Davydov" <[email protected]>
To: "General Discussion of SQLite Database" <[email protected]>
Sent: Monday, November 24, 2014 4:37:12 PM
Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in
Glusterfs
On Fri, Nov 21, 2014 at 02:01:39PM -0500, Joseph Fernandes wrote:
> We wanted to known the following
> 1) How could we improve the performance on the write side so that we have
> minimal latency?
> 2) Will ther be any write performance hit when the number of records in the
> DB increase?
Generally speaking, you have to do some work to arrange your data (modification
times) in some ordered way. This work can be done eihter in advance, as you
suggest, or on demand, as some people have already told you. But anyway this
will eat up necessary resources, regardless of whose code would do it, either
yours or SQLite. In practice (given magnetic disks as underlying storage),
most scarce of the mentioned resources is rotational/seek latency, which
detrimentally affects all disk operations of any scheduled priority. SQLite
performs extensive random disk access (mostly reads) on most operation
scenarios - selects, inserts, indexing etc. with possible exception of
small updates of non-indexed data (which are accessed in a similar fashion
by later selects). The only way to cope with the slow disk is keeping all
necessary data somwhere else, for example, into the RAM cache. Of course,
cache itself should be populated in advance to give this benefit, and, given
current RAM prices, it seems not very feasible to steal available memory from
smart applications in favour of dumb cache.
Hope, this considerations will help you in tuning your code.
Valentin Davydov.
_______________________________________________
sqlite-users mailing list
[email protected]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
[email protected]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
