Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Sun, 23 Nov 2014 14:34:23 -0500 (EST) Joseph Fernandes wrote: > Ok I get it now. Yes we are using a single db connection object, But > few questions, > 1) how would making sqlite3 single thread that improve the > performance? Is there a special advantage in this mode

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

4:37:12 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs On Fri, Nov 21, 2014 at 02:01:39PM -0500, Joseph Fernandes wrote: > We wanted to known the following > 1) How could we improve the performance on the write side so that we have > min

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

- Original Message - From: "Nico Williams" <n...@cryptonector.com> To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org> Sent: Monday, November 24, 2014 4:35:59 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

- Original Message - From: "James K. Lowden" <jklow...@schemamania.org> To: sqlite-users@sqlite.org Sent: Monday, November 24, 2014 3:56:14 AM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs On Sat, 22 Nov 2014 23:25:16 -0

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

BTW, the experience with dedup is that doing things off-line means never catching up, while doing them online means going slow. You might cache as much as you can in memory then go slow when you miss the cache... In practice I think it's best to separate data and metadata devices so that you can

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

My advice is to borrow from other clustered filesystems' experience. If you want to adhere to POSIX semantics then st_mtime and st_size visibility will be a particular headache, especially since you don't know when it's OK to lie (i.e., which callers of stat() are using st_mtime/st_size for

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Fri, Nov 21, 2014 at 02:01:39PM -0500, Joseph Fernandes wrote: > We wanted to known the following > 1) How could we improve the performance on the write side so that we have > minimal latency? > 2) Will ther be any write performance hit when the number of records in the > DB increase?

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Sun, Nov 23, 2014 at 4:26 PM, James K. Lowden wrote: > Metadata updates to Posix filesystems are seen as so costly that > fsync(2) on the datafile descriptor doesn't update them. A separate > sync on the directory is required. Compared to an in-memory update > (of

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Sat, 22 Nov 2014 23:25:16 -0500 (EST) Joseph Fernandes wrote: > 2) Using the changelog to feed the db has another issue i.e freshness > of data in the DB w.r.t the IO. Few of our data maintainer scanners > would require the freshness of the feed to be close to real. [...]

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs On Sat, 22 Nov 2014 11:01:45 -0500 (EST) Joseph Fernandes <josfe...@redhat.com> wrote: > Thanks Eduardo. > > Answers inline JOE>> > > a)If you use Sqlite in single thread, compile it witho

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Sat, 22 Nov 2014 11:01:45 -0500 (EST) Joseph Fernandes wrote: > Thanks Eduardo. > > Answers inline JOE>> > > a)If you use Sqlite in single thread, compile it without thread > support. > JOE>> We are running it in multithread mode, as the database will be > JOE>> fed by

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

Thanks James Please find the answers inline JOE>> - Original Message - From: "James K. Lowden" <jklow...@schemamania.org> To: sqlite-users@sqlite.org Sent: Saturday, November 22, 2014 10:29:01 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Da

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Fri, 21 Nov 2014 14:01:39 -0500 (EST) Joseph Fernandes wrote: > 4) Therefore, we are looking at a datastore that can give us a very > quick write(almost zero latency, as the recording is done inline > w.r.t file IO) and that as good data querying facilities(Slight >

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

Thanks Eduardo. Answers inline JOE>> - Original Message - From: "Eduardo Morras" <emorr...@yahoo.es> To: sqlite-users@sqlite.org Sent: Saturday, November 22, 2014 9:20:46 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs On

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

On Fri, 21 Nov 2014 14:01:39 -0500 (EST) Joseph Fernandes wrote: > Hi There, > > 1) We are trying to use sqlite3 in Glusterfs as a Change Time > Recording Data Store i.e using sqlite3 db to record any modification > attempts that happens on the glusterfs per file inode, So

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

uot;Keith Medcalf" <kmedc...@dessus.com> To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org> Sent: Saturday, November 22, 2014 9:02:30 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs Your Glusterfs does not s

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

: nothing works and no one knows why. >-Original Message- >From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- >boun...@sqlite.org] On Behalf Of Joseph Fernandes >Sent: Friday, 21 November, 2014 12:02 >To: sqlite-users@sqlite.org >Subject: [sqlite] Using Sqlite

[sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

Hi There, 1) We are trying to use sqlite3 in Glusterfs as a Change Time Recording Data Store i.e using sqlite3 db to record any modification attempts that happens on the glusterfs per file inode, So that it can be used to indicate the hotness of the file in the cluster. 2) We have developed a