----- Original Message ----- From: "Nico Williams" <[email protected]> To: "General Discussion of SQLite Database" <[email protected]> Sent: Monday, November 24, 2014 4:35:59 PM Subject: Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs
On Sun, Nov 23, 2014 at 4:26 PM, James K. Lowden <[email protected]> wrote: > Metadata updates to Posix filesystems are seen as so costly that > fsync(2) on the datafile descriptor doesn't update them. A separate > sync on the directory is required. Compared to an in-memory update > (of metadata, in kernel space) and a single fsync call, the price of a > SQLite transaction is enormous, at a guess an order of magnitude more. This. Updates of mtime and atime in particular are expensive. Another problem (for Lustre-style clusters) is stat(), since it returns information that only a metadata service might know (e.g., file type, file ID (st_dev and st_ino), and link count) but also information that it might not (file size, various timestamps), which then presents enormous headaches for implementors. There are also write visibility rules as to stat() that some applications depend on... This is why "ls -l" can be slow on some such clusters. JOE>> Glusterfs is not replacing POSIX atime,mtime and ctime with this db (Sqlite is not a metadata db). i.e stat will always read from actual atime/mtime/ctime of the inode. Therefore as mention in [1] we don't do any db read operations in the file IO path. The read consumer of the db are data maintaining scanners that are scheduled, and which want a smarter way to find the "HOT" and "COLD" files in the cluster. Infact there will be a one db per storage unit(brick) of glusterfs therefore no centralized db. Nico -- _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
