Hi,
I have one thing where i am looking for a solution. Maybe someone can
help me out or give me hint how to do.
Problem:
I often use a NiFi Clusters with "DistributedMapCacheClientService"
which uses a "DistributedMapCacheServer" for cluster wide key/value
storage. Per default the DMCS uses "in memory" and sockets for
synchronization. We use a persistence directory to make the data
persistent and to avoid that the data is gone after restarting the
entire cluster. But in the case, if the primary node changes, i think
the data will be outdated or used from a potential outdated other node.
If this other Node takes the primary node role, old data will be used
for next FecthDistrubutedMapCache. The latest updates over the old
primar node are gone.
Is there a service using e.g. zookeeper "int the backgroud" to get a
real distributed persitent Cache - even after restarting the entire
cluster / all nodes?
I know, the standard cache is able to provide a hugh frequent
read/update servise if the data is in memory. But if we need just one or
max a few updates per minute ...
Yes, using another system like a Database (as persistent singleton) can
be a solution - a not really matching solution. Why is there no standard
service in NiFi for this? Isn't it a good idea or i am the only one with
this problem in the past?
Thanks in advance for answers,
Jörg (Hammerbacher)
