[replying to list, with permission]
On Mon, Feb 22, 2010 at 12:05 AM, jeremey.barr...@nokia.com wrote:
I'm looking for a very scalable primary data store for a large web/API
application. Our data consists largely of lists of things, per user. So a
user has a bunch (dozens to hundreds) of thing A, some of thing B, a few of
thing C, etc. There are social elements to the app w/ shared data, so that
could be modeled with each user having a list of pointers, but with writes
being super cheap I'm more inclined to write everything everywhere (that's a
side issue, but it's in the back of my mind). Users number in the millions.
So basically I'm looking for something scalable, available, fast, and with
native support for range scans (given that almost every query is fetching
some list of things). This is where my questions lie... I'm pretty familiar
with the Bigtable model and it suits my needs quite well, I would store thing
A under a row key of userid.thingid (or similar) and then a range scan over
userid. will pick them all up at once.
HBase has been top of my list in terms of data model, but I ran across a
performance study which suggested it's questionable and the complexity of
components gives me some pause. So Cassandra seems the other obvious choice.
However, the data model isn't as clear to me (at least, not yet, which is
probably just a terminology problem).
My questions:
1) would you consider Cassandra (0.5+) safe enough for a primary data
store?
Yes. Several companies are deploying 0.5 in production. It's pretty
solid. (We'll have a 0.5.1 fixing some minor issues RSN, and a 0.6
beta.) And I agree that it's significantly simpler to deploy (and
keep running) than HBase.
2) is the row key model I suggested above the best approach in Cassandra, or
is there something better? My testing so far has been using get_range_slice
with a ColumnParent of just the CF and SlicePredicate listing the columns I
want (though really I want all columns, is there a shorthand for that?)
Cassandra deals fine with millions of columns per row, and allows
prefix queries on columns too. So an alternate model would be to have
userX as row key, and column keys A:1, A:2, A:3, ..., B:1, B:2, B:3,
This will be marginally faster than splitting by row, and has
the added advantage of not requiring OPP.
You could use supercolumns here too (where the supercolumn name is the
thing type). If you always want to retrieve all things of type A at a
time per user, then that is a more natural fit. (Otherwise, the lack
of subcolumn indexing could be a performance gotcha for you:
http://issues.apache.org/jira/browse/CASSANDRA-598).
3) schema changes (i.e. adding a new CF)... seems like currently you take
the whole cluster down to accomplish this... is that likely to change in the
future?
You have to take each node down, but a rolling restart is fine. No
reason for the whole cluster to be down at once.
We're planning to make CF changes doable against live nodes for 0.7,
in https://issues.apache.org/jira/browse/CASSANDRA-44.
4) any tuning suggestions for this kind of setup? (primary data store using
OrderPreservingPartitioner doing lots of range scans, etc.)
Nothing unusual -- just the typical try to have enough RAM to cache
your 'hot' data set.
5) I noticed mention in some discussion that the OrderPreserving mode is not
as well utilized and is probably in need of optimizations... how serious is
that, and are there people working on that, or is help needed?
We have range queries in our stress testing tool now, and with Hadoop
integration coming in 0.6 I expect it will get a lot more testing.
Certainly anyone who wants to get their hands dirty is welcome. :)
6) hardware... we could certainly choose to go with pretty beefy hardware,
especially in terms of RAM... is there a point where it just isn't useful?
Some recommdendations in
http://wiki.apache.org/cassandra/CassandraHardware. In general, don't
go beyond the knee of the price/performance curve, since you can
always add more nodes instead.
Past enough for your memtables
(http://wiki.apache.org/cassandra/MemtableSSTable), RAM is only useful
for caching reads, it won't help write performance. So that's the
main factor in how much do I need.
-Jonathan
