You can also use the Cassandra DMC, which is something we are starting to use a lot where I work.
Admittedly, the documentation is non-existent at the moment, but we also open sourced an experimental delegating DMC client that can be used to chain multiple DMCs together so you can do Redis for hot caching and something like Cassandra for broader cold caches https://github.com/Domestic-Resilience-FOSS/nifi-delegating-distributedmapcache-bundle On Wed, Nov 2, 2022 at 5:25 PM Peter Turcsanyi <[email protected]> wrote: > > Embedded Hazelcast can also be an option. In that case, there is no > need to set up an external cache but the Hazelcast instances are > running on the NiFi nodes (in the same JVM as NiFi). > Please note: no security/authentication is supported in embedded mode. > > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hazelcast-services-nar/1.18.0/org.apache.nifi.hazelcast.services.cacheclient.HazelcastMapCacheClient/index.html > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hazelcast-services-nar/1.18.0/org.apache.nifi.hazelcast.services.cachemanager.EmbeddedHazelcastCacheManager/index.html > > On Wed, Nov 2, 2022 at 5:10 PM Bryan Bende <[email protected]> wrote: > > > > The DMC Server does not support high availability. If you have a 3 > > node nifi cluster, each node will have a DMC client and DMC server, > > but the clients all have to point at only one of the servers, and if > > the node where that server is running goes down, there is no > > replication or failover to another node. So it is really up to you to > > decide if that is acceptable for your use case. If its not then you > > need to use a different DMC client implementation that can communicate > > with an external HA cache, like Redis. > > > > On Wed, Nov 2, 2022 at 11:38 AM Greene (US), Geoffrey N > > <[email protected]> wrote: > > > > > > I make heavy use of DistributedMapCacheServer in my nifi flows (one node; > > > not clustered). > > > > > > > > > > > > I seem to remember reading that the DistributedMapCacheServer is not to > > > be used in production; it’s a reference implementation only, and it is > > > not really recommended for production. > > > > > > > > > > > > Unfortunately, I can no longer find the reference saying that > > > DistributedMapCacheServer is not trustworthy for prod. > > > > > > I don’t have an HDFS implementation anywhere, but I do need the cacheing > > > part. > > > > > > > > > > > > Can someone explain? Can I use DistributedMapCacheServer in my > > > production flows? > > > > > > > > > > > > > > > > > >
