Redis has three modes standalone, sentinel, and cluster. Sentinel is an HA configuration with replicas and auto failover, but each node has a full copy of the data.
Cluster is a sharded key space so data is divided among the nodes. In order to implement atomic compare and set operations required by the distributed cache client, we need to use watches and transactions which are not supported in a cluster. You can create a highly available setup using Redis sentinel. To Boris’s point about the single point of failure, that is only true when using the built in DMC client and server. If you use an alternate client implementation like Redis or HBase then it can be highly available. On Thu, Feb 14, 2019 at 8:15 AM Boris Tyukin <[email protected]> wrote: > I am not NiFi dev, but personally, after looking at DistributedCache > processors we decided not to use them. There is nothing "distributed" about > them. It will be a single point of failure in your flows, difficult to > manage (you have to use NiFi to read/write/delete keys). And I also looked > at Redis (Did you see Bryan's tutorial? > https://bryanbende.com/development/2017/10/09/apache-nifi-redis-integration) > but I saw limitations with using a cluster as well. > > On the other hand, it is really easy to create your own custom groovy > processors or NiFi processors and use your own Redis cluster. > > Check Matt's blog and you will have your custom groovy processor done in a > few days, using his examples and recipes > http://funnifi.blogspot.com/ > > https://community.hortonworks.com/articles/75032/executescript-cookbook-part-1.html > > > Boris > > > > On Thu, Feb 14, 2019 at 3:20 AM Jianan Zhang <[email protected]> > wrote: > >> Hi all, >> >> I am trying to configure the RedisDistributedMapCacheClientService, and >> nifi told me that this controller service does not support the redis >> running in cluster mode. >> In our business, we already have a redis cluster, and I do not want to >> set up a new redis alone for this controller service. So can anybody tell >> me why RedisDistributedMapCacheClientService does not support clustered >> mode redis? >> >> Thanks, >> >> Jianan >> > -- Sent from Gmail Mobile
