Today I've also seen this benchmark in Chinese websites. SequoiaDB seems
come from a Chinese startup company, and in db-engines ranking
http://db-engines.com/en/ranking it's score is 0.00. So IMO I have to say
I think this benchmark is a soft sell. They compare three databases, two
written by c++ and one by java, and use a very tricky testcase to make
Cassandra can not hold all data in memtables. After all, java need more
memory than c++. For a on-disk database, generally data size of one node is
much larger than RAM, and it's performance of memory query is less
important than disk query.
So I think this benchmark have no value at all.
2014-12-19 14:47 GMT+08:00 Wilm Schumacher wilm.schumac...@gmail.com:
Hi,
I'm always interessted in such benchmark experiments, because the
databases evolve so fast, that the race is always open and there is a lot
motion in there.
And of course I askes myself the same question. And I think that this
publication is unreliable. For 4 reasons (from reading very fast, perhaps
there is more):
1.) It is unclear what this is all about. The title is NoSQL Performance
Testing. The subtitle is In-Memory Performance Comparison of SequoiaDB,
Cassandra, and MongoDB. However, in the introduction there is not one
word about in memory performance. The introduction could be a general
introduction for a general on-disk-nosql benchmark. So ... only the
subtitle (and a short sentence in the Result Summary) says what this is
actually about.
2.) There are very important databases missing. For in memory e.g.
redis. If e.g. redis is not a valid candidate in this race, why is this
so?MySQL is capable of in memory distributed databanking, too.
3.) The methodology is unclear. Perhaps I'm the only one, but what does
Run workload for 30 minutes (workload file workload[1-5]) mean for mixed
read/write ops? Why 30 min? Okay, I can image, that the authors estimated
the throughput, preset the number of 100 Mio rows and designed it to be
larger than the estimated throughput in x minutes. However, all this
information is missing. And why 45% and 22% of RAM? My first Idea would be
a VERY low ration, like 2% or so, and a VERY large ratio, like 80-90%. And
than everything in between. Is 22% or 45% somehow a magic number?
Furthermore in the Result summary there 1/2 and 1/4 of RAM are discussed.
Okay, 22% is near 1/4 ... but where does the difference origin from? And
btw. ... 22% of what? Stuff to insert? Stuff already insererted? It's all
deductable, but it's strange that the description is so sloppy.
4.) There is no repetion of the loads (as I understand). Its one run, one
result ... and it's done. I don't know a lot of cassandra in in-memory use.
But either the experiment should be repeated quite some runs OR it should
be explained why this is not neccessary.
Okay, perhaps 1 is a little picky, and 4 is a little fussy. But 3 is
strange and 2 stinks.
Well, just my first impression. And that's Cassandra is very fast ;).
Best regards
Wilm
Am 19.12.2014 um 06:41 schrieb diwayou:
i just have read this benchmark pdf, does anyone have some opinion
about this?
i think it's not fair about cassandra
url:
http://www.bankmark.de/wp-content/uploads/2014/12/bankmark-20141201-WP-NoSQLBenchmark.pdf
http://msrg.utoronto.ca/papers/NoSQLBenchmark
