The venti surveys have stopped trickling in. As promised,
here is a summary.
Below is a table summarizing the venti data.
Each line is one server that someone submitted.
I am surprised that there are people out there
with 4+ year old servers. You take very good care
of your disks. I suspect that most of the large but new
servers replaced older ones.
Most people who replied had less than 2 million blocks,
and I suspect most people who didn't reply have servers like
the bottom half of the table rather than the top half.
You could keep a million-block index in 50 MB of memory
and do away with the index hash table completely.
It might be worth having venti detect when the index
cache can fit the entire index and operate entirely out
of the cache, without ever needing to read from the
hash table. You could get much better performance
out of venti doing that, and not even need a bloom filter.
In fact, in that mode you wouldn't even need to configure
an index disk.
Then there are the power users, with their tens or hundreds
of millions of blocks. One GB of index cache only gets you
thirty million index entries, so these people have essentially
no choice but to maintain the hash table index with its
heavy seek penalties. Maybe in a few years flash storage will
save the day.
Russ
GB GB days
clumps cclumps raw compr age
338708170 279714997 1158.6 593.4 681 most clumps
243894713 207350734 5614.7 1657.2 69 most bytes
148582718 105977246 447.2 186.5 561
45255915 23277726 135.0 87.0 169
27395797 15624078 95.9 62.9 7
22095841 12085340 78.9 53.4 66
10194765 6088883 38.8 22.7 -
7991230 6622335 34.4 17.3 1768
6618670 2068276 46.2 39.5 60
6158824 4209478 37.7 21.2 1780
5971129 4325778 40.4 23.8 361
5805550 2246660 40.2 30.4 41
4900543 3749792 23.4 12.2 132
4151139 274551 31.3 30.4 214
2465171 1065357 16.3 12.4 1044
1756289 324119 12.6 11.4 1984 oldest
1544611 17057 23.4 23.3 359
1353686 901896 2.4 1.1 -
1308981 981760 7.3 4.0 1413
1279725 1073807 7.2 3.1 1544
1197798 961554 6.8 3.3 1436
1136953 609235 6.7 4.8 220
1082686 971796 6.9 1.9 1119
837292 347263 5.5 4.2 239
834808 549286 5.3 3.2 1164
742362 609642 4.1 1.8 490
641818 457784 3.8 2.2 1631 slowest growth
279319 222898 1.6 0.8 -
250992 168172 1.4 0.8 547
238121 215040 1.4 0.5 268
213772 183795 1.2 0.5 42
151431 134091 0.9 0.4 108
141257 121809 0.8 0.3 208
122807 105275 0.6 0.3 32
70 67 0.0 0.0 0 newest
