On Sunday, 16 December 2012 at 07:47:48 UTC, Rob T wrote:
On Sunday, 16 December 2012 at 05:37:57 UTC, SomeDude wrote:
Isn't the memory management completely negligible when
compared to the database access here ?
Here are the details ...
My test run selects and returns 206,085 records with 14 fields
per record.
With all dynamic memory allocations disabled that are used to
create the data structure containing the returned rows, a run
takes 5 seconds. This does not return any data, but it runs
exactly through all the records in the same way but returns to
a temporary stack allocated value of appropriate type.
If I disable the GC before the run and re-enable it immediately
after, it takes 7 seconds. I presume a full 2 seconds are used
to disable and re-enable the GC which seems like a lot of time.
With all dynamic memory allocations enabled that are used to
create the data structure containing the returned rows, a run
takes 28 seconds. In this case, all 206K records are returned
in a dynamically generate list.
If I disable the GC before the run and re-enable it immediately
after, it takes 11 seconds. Since a full 2 seconds are used to
disable and re-enable the GC, then 9 seconds are used, and
since 5 seconds are used without memory allocations, the
allocations are using 4 seconds, but I'm doing a lot of
allocations.
In my case, the structure is dynamically generated by
allocating each individual field for each record returned, so
there's 206,085 records x 14 fields = 2,885,190 allocations
being performed. I can cut the individual allocations down to
206,000 by allocating the full record in one shot, however this
is a stress test designed to work D as hard as possible and
compare it with an identically stressed C++ version.
You cannot expect the GC to perform like manual memory
management. It's a completely unrealistic microbenchmark to
allocate each individual field, even for manual MM. The least you
can do to be a little bit realistic is indeed to allocate one row
at a time. I hope that's what you intend to do. But usually,
database drivers allow the user to tweak the queries and decide
how many rows can be fetched at a time, and it's pretty common to
fetch 50 or 100 rows at a time, meaning one allocation only each
time. It would be interesting to compare the performance of the
two languages in these situations, i.e one row at a time, and 50
rows at a time.
