On Thu, Feb 14, 2008 at 6:19 PM, <[EMAIL PROTECTED]> wrote: > "Samuel Neff" <[EMAIL PROTECTED]> wrote: > > If the images you're storing are larger than the defined page size for > the > > database (which is most likely the case) then you can get better > performance > > and reduced memory consumption by storing the images in the file system > and > > store only paths to the files in the database. This means reading the > large > > amount of data directly from the file system instead of from sqlite's > > linked-list of pages and bypassing the page caching layer (which you > > probably don't want for images anyways) and freeing up more of the page > > cache for real database data. > > > > One would think. And yet experiments suggest otherwise. It > turns out to be faster to read images directly out of SQLite > BLOBs until the image gets up to about 15KB on windows and > up to about 60KB on linux. And even for much larger images, > the performance difference between reading from SQLite and > reading from a file is not that great, so it is a reasonable > thing to do to read from SQLite if transactions are important > to you or if it is just more convenient. > > -- > D. Richard Hipp <[EMAIL PROTECTED]> > > <http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users> >
How much larger? (Have the experiments been published?) These are exactly the factors we're trading off: performance versus reliability and programmer convenience. Reliability is paramount. If there is no way to achieve transactional semantics using files outside of SQLite, then any discussions about performance and programmer convenience are purely academic. The application must behave deterministically. Performance is next most important. Our images are typically 20k JPEGs, as captured with the current Nokia Tablet's VGA camera. Our audio streams are encoding with Speex at about 2k/second, so a typical 1-minute recording is about 120k. However, audio recordings can grow arbitrarily large, so performance with 10-minute (1.2M) or 100-minute (12M) recordings could be relevant. Also, we can anticipate video recordings in the future, with a steeper slope. Naturally, we are willing to sacrifice programmer convenience for the sake of end-user reliability and performance. In this case, it appears that reliability and programmer convenience are congruent, with transactions. Thanks for a great database, Dr. Hipp. It suits the burgeoning portable applications perfectly. It's transactional incremental I/O enables rich media on portable devices. Norm. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
