If your proposed solution is crazy depends on your needs :) It sounds like you can live with not-realtime data. So it is ok to cache it. Why preproduce the results if you only need 5% of them? Why not use redis as a cache with expiring sorted sets that are filled on demand from cassandra partitions with counters? So redis has much less to do and can scale much better. And you are not limited on keeping all data in ram as cache data is volatile and can be evicted on demand. If this is effective also depends on the size of your sets. CS wont be able to sort them by score for you, so you will have to load the complete set to redis for caching and / or do sorting in your app on demand. This certainly won't work out well with sets with millions of entries.
2017-01-13 23:14 GMT+01:00 Mike Torra <mto...@demandware.com>: > We currently use redis to store sorted sets that we increment many, many > times more than we read. For example, only about 5% of these sets are ever > read. We are getting to the point where redis is becoming difficult to > scale (currently at >20 nodes). > > We've started using cassandra for other things, and now we are > experimenting to see if having a similar 'sorted set' data structure is > feasible in cassandra. My approach so far is: > > 1. Use a counter CF to store the values I want to sort by > 2. Periodically read in all key/values in the counter CF and sort in > the client application (~every five minutes or so) > 3. Write back to a different CF with the ordered keys I care about > > Does this seem crazy? Is there a simpler way to do this in cassandra? > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
