Hi There, 1) We are trying to use sqlite3 in Glusterfs as a Change Time Recording Data Store i.e using sqlite3 db to record any modification attempts that happens on the glusterfs per file inode, So that it can be used to indicate the hotness of the file in the cluster.
2) We have developed a prototype that inserts/updates inode records in sqlite3 db(one instance of the db file per glusterfs storage unit or as we call it a brick) to record the modification that are happening on the inode. These insert/updates are in the IO path of the file i.e any data or metadata change time will be record inline, w.r.t the IO, in the DB. 3) We only save the modification time on the to the db, so that we can easily query for "What files have change during a specific period of time" or "What files have not change during a specific period of time". The query is done by a scanner that runner on each storage unit or brick on a regular interval and chooses HOT or COLD files as suggested by the DB for data maintenance operations. 4) Therefore, we are looking at a datastore that can give us a very quick write(almost zero latency, as the recording is done inline w.r.t file IO) and that as good data querying facilities(Slight latency in the read is fine but the fresh of that record data should be spot on). 5) Please find our DB setting, Journal mode : WAL SYNC MODE : NORMAL Cache Size : 1000 - 4096 Pages (Default Page Size) AutoCheck-Pointing: 1000 - 1000000 Pages We wanted to known the following 1) How could we improve the performance on the write side so that we have minimal latency? 2) Will ther be any write performance hit when the number of records in the DB increase? Thanks in advance ~Joe _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
