Howdy, I wrote a note the other day on BLOBS that did not go through I guess but since BLOBs came up again I want to throw in my 2 cents.
I have YET to see a database, small to massively scalable that could handle BLOBS worth anything. Databases, especially relational ones give you quick power to sort, find, query records quickly. The only way I have ever seen BLOBS handled correctly, and most effectively, is with story blobs on the hard drive, then storing the path in the database. The database table for the blobs had meta data (fields/columns) that contained information to find your blobs on the hard drive fast. Now, last but not least the hard drive directory structure being what it is (on Windows NT and above) computers, you get serious performance degradation when you hit adding more than 500 folders to a directory, or 500 files under a directory, so you have to come up with a good mechanism to limit this on your drive for a ton of blobs. Personally, since SQL Server, Oracle, etc. can't handle then that well, it seems to be kind of a waste of time to implement in Sqlite. I prefer the simplicity talk given early. If someone wants blobs, do it the old fashioned way! Thanks, Allan -----Original Message----- From: D. Richard Hipp [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 04, 2003 4:40 AM To: [EMAIL PROTECTED] Subject: Re: [sqlite] Performance problem Avner Levy wrote: > > We have just finished testing the same scenario with MySql at > amazingly they continued to insert 1500-3000 rows per second even when > the database had 60,000,000 records. I don't know how this magic is done... Nor do I. If anybody can clue me in, I would appreciate it. I suspect the secret must be in the file format for their indices. Does anybody have any idea what that format is? Or how indices work in MySQL? FWIW: Big changes are looming on the horizon for SQLite. The format of indices will probably have to change at some point in the future in order to accomodate BLOBs. This will result in an an incompatible file format change (version 3.0.0). The change is not imminent (think summer of 2004) nor inevitable. But if other changes could be made at the same time to support faster inserts of non-localized data, the transition from 2.x.x to 3.x.x would be a good time to toss them in. -- D. Richard Hipp -- [EMAIL PROTECTED] -- 704.948.4565 --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
