> With FB 2.5.2 SC 64-bit on Windows 7 Prof.
>
> While copying a 18GB database from folder A to B on the same spinning
> physical disk at ~33MB/s read + ~33MB/s write, thus 66MB/s total, doing
> a select count(*) on that database (8K page size) for a table with ~6Mio
> records at a physical disk read rate (according to perfmon) of only
> ~7,8MB/s.
>
> The system has been freshly rebooted, thus the database is not in the
> file system cache nor in the FB connection page cache.
>
> A very naive test, but as I would expect a COUNT(*) with cold caches to
> be purely I/O bound, with max. ~7,8MB/s we are far away from nearly
> fully utilizing disk I/O.
I'm neither surprised or shocked by those numbers. The copy command is
doing very large block buffered sequential reads that the disk controller
can both execute out of order and logically anticipate subsequent
sequential reads. Likewise, the write side is doing large block
buffered sequential transfers that, again, the disk controller can optimize.
A more reasonable measure of disk efficiency is to instrument a Firebird
engine to record page numbers for pages fetched by your "count (*)", then
write a trivial C program to fetch those pages in the given order. If you
subtract off the database startup time from the "count (*)" time, my guess
is that the result will compare fairly with the simple page reader. If
not, something is serious wrong and a system profile is in order.
Disks suck. Disks have always sucked. Modern disks suck ony slightly less
and older disks (though, admittedly, they're a lot cheaper). SSDs suck
too, except in comparison to disks (they have to use an I/O path designed
for sucky physical disks). The only way to make a database perform well is
to factor the disk out of the performance equation. See my Distributed
Firebird Roadmap for one example of how do this. There are many other
ways, particularly give the availability of huge, dirt cheap, and
blindingly fast RAM. Put lots of RAM between the database engine and the
(sucky) disk is the only way to fly. But more disk buffers really doesn't
hack it.
--
Jim Starkey
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
