Re: [sqlite] beat 120,000 inserts/sec

2005-04-11 Thread Thomas Steffen
On Apr 11, 2005 4:06 PM, Christian Smith <[EMAIL PROTECTED]> wrote: > The test given is clearly CPU bound. All the big numbers are from people > with big CPUs, with equally big RAM performance as well, probably. I have done a few database test recently, and I often found them to be CPU bound, at

Re: [sqlite] beat 120,000 inserts/sec

2005-04-11 Thread Christian Smith
On Sat, 9 Apr 2005, Al Danial wrote: >On Apr 9, 2005 12:43 AM, Andy Lutomirski <[EMAIL PROTECTED]> wrote: >> Al Danial wrote: >> > The attached C program measures insert performance for populating >> > a table with an integer and three random floating point values with >> > user defined

Re: [sqlite] beat 120,000 inserts/sec

2005-04-09 Thread bbum
On Apr 9, 2005, at 8:49 AM, Al Danial wrote: I did try SYNCHRONOUS=off but that didn't seem to have an effect; I'll study the docs to make sure I've got it right. This isn't surprising. fsync() is largely a no-op on just about any operating system. It doesn't actually guarantee that the bytes

Re: [sqlite] beat 120,000 inserts/sec

2005-04-09 Thread Andy Lutomirski
Andrew Piskorski wrote: On Sat, Apr 09, 2005 at 11:49:17AM -0400, Al Danial wrote: Thanks to everyone who posted performance numbers and machine setup info. Some results were counterintuitive (I'd have guessed SCSI drives would come out on top) but many variables are at work It is basically

Re: [sqlite] beat 120,000 inserts/sec

2005-04-09 Thread Andrew Piskorski
On Sat, Apr 09, 2005 at 11:49:17AM -0400, Al Danial wrote: > Thanks to everyone who posted performance numbers and machine > setup info. Some results were counterintuitive (I'd have guessed > SCSI drives would come out on top) but many variables are at work It is basically impossible that a 10k

Re: [sqlite] beat 120,000 inserts/sec

2005-04-09 Thread Al Danial
On Apr 9, 2005 12:43 AM, Andy Lutomirski <[EMAIL PROTECTED]> wrote: > Al Danial wrote: > > The attached C program measures insert performance for populating > > a table with an integer and three random floating point values with > > user defined transaction size. Usage is: > > > >

Re: [sqlite] beat 120,000 inserts/sec

2005-04-09 Thread Clay Dowling
Jeremy Hinegardner wrote: > ./sqlite_insert 10 5 10 inserts to /tmp/a.db in 0.671 s = 149057.52 inserts/s ./sqlite_insert 200 2 200 inserts to /tmp/a.db in 14.437 s = 138535.38 inserts/s ./sqlite_insert 200 5 200 inserts to /tmp/a.db in 15.322 s = 130530.52

Re: [sqlite] beat 120,000 inserts/sec

2005-04-09 Thread bbum
There are also pragmas to control page size and in-memory caching. You will want to play with those, as well. If SQLite is in the middle of a transaction and you load it up with commands, it will create a journal file in /tmp/ to start pages that don't fit in the in-memory page cache (or

Re: [sqlite] beat 120,000 inserts/sec

2005-04-08 Thread Jeremy Hinegardner
On Fri, Apr 08, 2005 at 11:01:02PM -0400, Al Danial wrote: [...] > What kind of insert performance do you see on your machine? If it > is substantially better than 120 kinserts/s what kind of hardware > do you have? I'm especially interested in how much faster the > code runs on systems with

Re: [sqlite] beat 120,000 inserts/sec

2005-04-08 Thread Andy Lutomirski
Al Danial wrote: The attached C program measures insert performance for populating a table with an integer and three random floating point values with user defined transaction size. Usage is: ./sqlite_insert All of these are on Gentoo, Athlon 64 3200+, running 64 bit. Writing to /tmp,

Re: [sqlite] beat 120,000 inserts/sec

2005-04-08 Thread Dan Nuffer
Al Danial wrote: > What kind of insert performance do you see on your machine? If it > is substantially better than 120 kinserts/s what kind of hardware > do you have? I'm especially interested in how much faster the > code runs on systems with multiple disks in a RAID 0 configuration. > Are

Re: [sqlite] beat 120,000 inserts/sec

2005-04-08 Thread Eric Bohlman
Al Danial wrote: A scientific application I work with has clumsy data retrieval options. I dumped the application's output--integer and floating point numbers--into an SQLite database and soon after began to enjoy the power of SQL to pull out interesting results. The main complaint for making the

Re: [sqlite] beat 120,000 inserts/sec

2005-04-08 Thread Dan Kennedy
Random thought: You might squeeze some more performance out by trying a couple different filesystems. i.e. if you're using ext3, try some different journaling options, or try ext2. --- Al Danial <[EMAIL PROTECTED]> wrote: > A scientific application I work with has clumsy data retrieval >

[sqlite] beat 120,000 inserts/sec

2005-04-08 Thread Al Danial
A scientific application I work with has clumsy data retrieval options. I dumped the application's output--integer and floating point numbers--into an SQLite database and soon after began to enjoy the power of SQL to pull out interesting results. The main complaint for making the transfer to