Hi, On 09.07.2011, at 23:37, Dan Kuebrich wrote:
> Perhaps I misunderstand your proposal, but it seems that even with your > manual key placement schemes, the row would still be huge, no matter what > node it gets placed on. A better solution might be figuring out how to make > each row into a few smaller ones to get better balancing of load and also > faster reads. I probably could have been more clear. The idea is to randomly choose one node among several for a single key each time some data is added to a row. Say, a particular key is normally assigned to node n. Then, for each write to that key, we randomly choose one of the nodes n, n+1, n+2, ..., n+k to write the data (or we could choose the node with the least load). Of course, if one wants to read all data for that key, all these nodes have to be queried, because each node will store a chunk of the data for the key. > - Can you segment the column(s) of the row into different, predictably-named > rows? Yes and no. We actually do not know in advance which of the rows will be the ones that grow that large. However, and this is the second solution I described, it would be possible to randomly choose some suffix that is added when some data is added to a key. For example, we might have a key "abc" and a predefined list of suffixes (1, 2, 3). When adding data instead of writing to key "abc", we randomly choose one of the suffixes and then, for example, write to "abc1". Of course, the number of suffixes determines how high the probability is that the modified keys will actually be stored on different nodes. > - Or segment into different rows and use a secondary index to find the rows > that are part of a particular RDF? This is actually something I hadn't looked at yet, thanks for the pointer! > - And/or compress the RDF data (maybe you're already doing that) to reduce > the impact of large rows? While compression would certainly help, it does not really change the underlying problem, just delay its effect. Actually, I hope that Cassandra itself will at some point take care of compression ;) Another problem is that you can't actually just increase cluster size to scale to larger datasets, because the constraint is the disk space on single nodes. Cheers, Günter > > On Sat, Jul 9, 2011 at 4:27 PM, Günter Ladwig <[email protected]> wrote: > Hi all, > > we are currently looking at using Cassandra to store highly skewed RDF data. > With the indexes we use it may happen that a single row contains up to 20% of > the whole dataset, meaning that it can grow larger than available disk space > on single nodes. In [1], it says that this limitation is not likely to change > in the future, but I was wondering if anybody has looked at this problem? > > One thing that comes to mind is a simple approach to DHT load-balancing [2], > where keys are assigned to one node of several random alternatives (which > means that for reading, all these nodes have to be queried). This is a bit > similar to replication, except, of course, that only one copy of the data is > stored. As this would require changes to the Cassandra code base, we could > "simulate" this by randomly choosing one of several predefined suffixes and > appending it to a key before storing it. By modifying a key this way, we > could be somewhat sure that it will be stored at a different node. The first > solution would certainly be preferable. > > Any thoughts or experiences? Failing that, maybe someone can give me a > pointer into the Cassandra code base, where something like the [2] should be > implemented. > > Cheers, > Günter > > [1] http://wiki.apache.org/cassandra/CassandraLimitations > [2] Byers at el.: Simple Load Balancing for Distributed Hash Tables, > http://www.springerlink.com/content/r9r4qcqxc2bmfqmr/ > > -- > > Dipl.-Inform. Günter Ladwig > > Karlsruhe Institute of Technology (KIT) > Institute AIFB > > Englerstraße 11 (Building 11.40, Room 250) > 76131 Karlsruhe, Germany > Phone: +49 721 608-47946 > Email: [email protected] > Web: www.aifb.kit.edu > > KIT – University of the State of Baden-Württemberg and National Large-scale > Research Center of the Helmholtz Association > > -- Dipl.-Inform. Günter Ladwig Karlsruhe Institute of Technology (KIT) Institute AIFB Englerstraße 11 (Building 11.40, Room 250) 76131 Karlsruhe, Germany Phone: +49 721 608-47946 Email: [email protected] Web: www.aifb.kit.edu KIT – University of the State of Baden-Württemberg and National Large-scale Research Center of the Helmholtz Association
smime.p7s
Description: S/MIME cryptographic signature
