Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Nico Williams
On Sun, Nov 23, 2014 at 4:26 PM, James K. Lowden jklow...@schemamania.org wrote: Metadata updates to Posix filesystems are seen as so costly that fsync(2) on the datafile descriptor doesn't update them. A separate sync on the directory is required. Compared to an in-memory update (of

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Valentin Davydov
On Fri, Nov 21, 2014 at 02:01:39PM -0500, Joseph Fernandes wrote: We wanted to known the following 1) How could we improve the performance on the write side so that we have minimal latency? 2) Will ther be any write performance hit when the number of records in the DB increase? Generally

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Nico Williams
My advice is to borrow from other clustered filesystems' experience. If you want to adhere to POSIX semantics then st_mtime and st_size visibility will be a particular headache, especially since you don't know when it's OK to lie (i.e., which callers of stat() are using st_mtime/st_size for

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Nico Williams
BTW, the experience with dedup is that doing things off-line means never catching up, while doing them online means going slow. You might cache as much as you can in memory then go slow when you miss the cache... In practice I think it's best to separate data and metadata devices so that you can

[sqlite] Missing function in NativeDB.c : Java_org_sqlite_NativeDB_result_1error()

2014-11-24 Thread pelz
Hi Everyone! I wanted to use the user defined functions in Java language with sqlite-jdbc-3.8.7. (On Ubuntu Linux 14.04 LTS 64bits) I wanted to check the arguments number in my custom function. And I thought, that I send a message for user, when he gave too many arguments in the function. I

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Joseph Fernandes
- Original Message - From: James K. Lowden jklow...@schemamania.org To: sqlite-users@sqlite.org Sent: Monday, November 24, 2014 3:56:14 AM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs On Sat, 22 Nov 2014 23:25:16 -0500 (EST) Joseph Fernandes

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Joseph Fernandes
- Original Message - From: Nico Williams n...@cryptonector.com To: General Discussion of SQLite Database sqlite-users@sqlite.org Sent: Monday, November 24, 2014 4:35:59 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs On Sun, Nov 23, 2014 at

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Joseph Fernandes
Again asking the same question : a. Does WAL during an insert/update in the log file do any internal search/sort and then insert/update to the log or b. Just appends the WAL log with the incoming insert/update entry, thus keeping the writes sequential and during a checkpoint (manual or