I also had a hard time reading this. It sounds like the tl;dr is that your are proposing storing each redis collection in a single region entry, rather than a a partition region? I guess the question is whether users will have a few very large collections that need to be partitioned, or lots and lots of really tiny collections. I'm not quite sure what the more common use case is.
-Dan On Tue, Feb 14, 2017 at 3:22 PM, Swapnil Bawaskar <sbawas...@pivotal.io> wrote: > The Redis adapter was designed so that we can scale all the Redis data > structures horizontally. If you bring the data structures to region entry > level, there is no reason for anyone to use our implementation over Redis. > > On Tue, Feb 14, 2017 at 3:15 PM Jason Huynh <jhu...@pivotal.io> wrote: > >> Hi Hitesh, >> >> Not sure about everyone else, but I had a hard time reading this, >> however I think I figured out what you were describing... the only part I >> still am unsure about is Feedback/vote: both behaviour is desirable. Do >> you mean you want feedback and voting on whether both behaviors are >> desired? As in old implementation and new implementation? >> >> 2,3,4) The new implementation would mean all the data for a specific >> data structure is contained in a single bucket. So the individual data >> structures are not quite scalable. How would you allow scaling of a single >> data structure? >> >> On Tue, Feb 14, 2017 at 3:05 PM Real Wes <thereal...@outlook.com> wrote: >> >> In what format do you want the feedback Hitesh? For now I’ll just >> comment: >> >> 1. Redis Type String >> No comments except that a future Geode value-add would be to extend the >> Jedis client so that the K/V’s are not compressed. In this way OQL and CQ >> will work. The tradeoff of this is that the data cannot be read by a >> native redis client but for Geode users it’s great. Call the new client >> Geodis. >> >> 2. List/ Hash/ Set/ SortedSet >> Creating a separate region for each creates a constraint that the keys >> are limited to the characters for region names, which are A-z/0-9/ - and >> _. Everything else is out. Redis users might start asking questions why >> their list named ++^^/## throws an error. Your suggestion to make it a key >> rather than a region solves this. Furthermore, creating a new region every >> time a new Redis collection is created is going to be slow. I’m not sure >> why a region was created but I’m sure it made sense to the developer at the >> time. >> >> 7. Default Config >> Can’t we configure a gfsh option to default to the region types we want? >> Customer A will want PARTITION but Customer B will want >> PARTITION_REDUNDANT_EXPIRATION_PERSISTENT. >> I wonder if we can consider a geode> create region —redisType=PARTITION_ >> REDUNDANT_EXPIRATION_PERSISTENT that makes _all_ Redis regions of that >> type? >> >> >> >> On Feb 14, 2017, at 5:36 PM, Hitesh Khamesra <hitesh...@yahoo.com<mailto: >> hitesh...@yahoo.com>> wrote: >> >> Current GeodeRedisAdapter implementation is based on >> https://cwiki.apache.org/confluence/display/GEODE/ >> Geode+Redis+Adapter+Proposal. >> We are looking for some feedback on Redis commands and their mapping to >> geode region. >> >> 1. Redis Type String >> a. Usage Set k1 v1 >> b. Current implementation creates "STRING_REGION" >> geode-partition-region upfront >> c. This k1/v1 are geode-region key/value >> d. Any feedback? >> >> 2. List Type >> a. usage "rpush mylist A" >> b. Current implementation maps each list to geode-partition-region(i.e. >> mylist is geode-partition-region); with the ability to get item from >> head/tail >> c. Feedback/vote >> -- List type operation at region-entry level; >> -- region-key = "mylist" >> -- region-value = Arraylist (will support all redis list ops) >> d. Feedback/vote: both behavior is desirable >> >> >> 3. Hashes >> a. this represents field-value or java bean object >> b. usage "hmset user1000 username antirez birthyear 1977 verified 1" >> c. Current implementation maps each hashes to >> geode-partition-region(i.e. user1000 is geode-partition-region) >> d. Feedback/vote >> -- Should we map hashes to region-entry >> -- region-key = user1000 >> -- region-value = map >> -- This will provide java bean sort to behaviour with 10s of >> field-value >> -- Personally I would prefer this.. >> e. Feedback/vote: both behaviour is desirable >> >> 4. Sets >> a. This represents unique keys in set >> b. usage "sadd myset 1 2 3" >> c. Current implementation maps each sadd to geode-partition-region(i.e. >> myset is geode-partition-region) >> d. Feedback/vote >> -- Should we map set to region-entry >> -- region-key = myset >> -- region-value = Hashset >> e. Feedback/vote: both behaviour is desirable >> >> 5. SortedSets >> a. This represents unique keys in set with score (usecase Query top-10) >> b. usage "zadd hackers 1940 "Alan Kay"" >> c. Current implementation maps each zadd to geode-partition-region(i.e. >> hackers is geode-partition-region) >> d. Feedback/vote >> -- Should we map set to region-entry >> -- region-key = hackers >> -- region-value = Sorted Hashset >> e. Feedback/vote: both behaviour is desirable >> >> 6. HyperLogLogs >> a. A HyperLogLog is a probabilistic data structure used in order to >> count unique things (technically this is referred to estimating the >> cardinality of a set). >> b. usage "pfadd hll a b c d" >> c. Current implementation creates "HLL_REGION" geode-partition-region >> upfront >> d. hll becomes region-key and value is HLL object >> e. any feedback? >> >> 7. Default config for geode-region (vote) >> a. partition region >> b. 1 redundant copy >> c. Persistence >> d. Eviction >> e. Expiration >> f. ? >> >> 8. It seems; redis knows type(list, hashes, string ,set ..) of each key. >> Thus for each operation we need to make sure type of key. In current >> implementation we have different region for each redis type. Thus we have >> another region(metaTypeRegion) which keeps type for each key. This makes >> any operation in geode slow as it needs to verify that type. For instance, >> creating new key need to make sure its already there or not. Whether we >> should allow type change or not. >> a. Feedback/vote >> -- type change of key >> -- Can we allow two key with same name but two differnt type (as it >> will endup in two different geode-region) >> String type "key1" in string region >> HLL type "key1" in HLL region >> b. any other feedback >> >> 9. Transactions: >> a. we will not support transaction in redisAdapter as geode transaction >> are limited to single node. >> b. feedback? >> >> 10. Redis COMMAND (https://redis.io/commands/command) >> a. should we implement this "COMMAND" ? >> >> 11. Any other redis command we should consider? >> >> >> Thanks. >> Hitesh >> >> >>
