Re: implementing a 'sorted set' on top of cassandra

You could store the key -> score pairs in Cassandra, pull out the full partition and repopulate the cache in redis with the top N whatever you need. I'd only read the Cassandra values directly in order to repopulate the cache. I wouldn't try to score the score -> key values, the perf will be a

Re: implementing a 'sorted set' on top of cassandra

On Tue, Jan 17, 2017 at 11:47 AM, Mike Torra wrote: > Thanks for the feedback everyone! Redis `zincryby` and `zrangebyscore` is > indeed what we use today. > > Caching the resulting 'sorted sets' in redis is exactly what I plan to do. > There will be tens of thousands of

Re: implementing a 'sorted set' on top of cassandra

Thanks for the feedback everyone! Redis `zincryby` and `zrangebyscore` is indeed what we use today. Caching the resulting 'sorted sets' in redis is exactly what I plan to do. There will be tens of thousands of these sorted sets, each generally with <10k items (with maybe a few exceptions going

Re: implementing a 'sorted set' on top of cassandra

Mike mentioned "increment" in his initial post. That let me think of a case with increments and fetching a top list by a counter like https://redis.io/commands/zincrby https://redis.io/commands/zrangebyscore 1. Cassandra is absolutely not made to sort by a counter (or a non-counter numeric

Re: implementing a 'sorted set' on top of cassandra

Sorted sets don't have a requirement of incrementing / decrementing. They're commonly used for thing like leaderboards where the values are arbitrary. In Redis they are implemented with 2 data structures for efficient lookups of either key or value. No getting around that as far as I know. In

Re: implementing a 'sorted set' on top of cassandra

Sorting on an "incremented" numeric value has always been a nightmare to be done properly in C* Either use Counter type but then no sorting is possible since counter cannot be used as type for clustering column (which allows sort) Or use simple numeric type on clustering column but then to

Re: implementing a 'sorted set' on top of cassandra

If your proposed solution is crazy depends on your needs :) It sounds like you can live with not-realtime data. So it is ok to cache it. Why preproduce the results if you only need 5% of them? Why not use redis as a cache with expiring sorted sets that are filled on demand from cassandra

Re: implementing a 'sorted set' on top of cassandra

Not if you want to sort by score (a counter) Am 14.01.2017 08:33 schrieb "DuyHai Doan" : > Clustering column can be seen as sorted set > > Table abstraction == Map> > > > On Sat, Jan 14, 2017 at 2:28 AM, Edward Capriolo > wrote: > >> >> >> On Fri,

Re: implementing a 'sorted set' on top of cassandra

Clustering column can be seen as sorted set Table abstraction == Map> On Sat, Jan 14, 2017 at 2:28 AM, Edward Capriolo wrote: > > > On Fri, Jan 13, 2017 at 8:14 PM, Jonathan Haddad > wrote: > >> I've thought about this for years and have never

Re: implementing a 'sorted set' on top of cassandra

On Fri, Jan 13, 2017 at 8:14 PM, Jonathan Haddad wrote: > I've thought about this for years and have never arrived on a particularly > great implementation. Your idea will be maybe OK if the sets are very > small and if the values don't change very often. But in a system

Re: implementing a 'sorted set' on top of cassandra

I've thought about this for years and have never arrived on a particularly great implementation. Your idea will be maybe OK if the sets are very small and if the values don't change very often. But in a system where the values of the keys in the set change frequently (lots of tombstones) or the

Re: implementing a 'sorted set' on top of cassandra

On Fri, Jan 13, 2017 at 5:14 PM, Mike Torra wrote: > We currently use redis to store sorted sets that we increment many, many > times more than we read. For example, only about 5% of these sets are ever > read. We are getting to the point where redis is becoming difficult

implementing a 'sorted set' on top of cassandra

We currently use redis to store sorted sets that we increment many, many times more than we read. For example, only about 5% of these sets are ever read. We are getting to the point where redis is becoming difficult to scale (currently at >20 nodes). We've started using cassandra for other