Hi everyone.
I'm running a single instance of NiFi on a Kubernetes cluster using the the
Helm chart from Cetic (I can't share the link here, it's on GitHub).
This Helm chart uses PVCs for flow file storage, which works fine for
persisting flows between restarts of NiFi.
The problem is that our PVCs aren't backed up, nor are they resilient to an
availability zone failure.
So I'm trying to find a method for backing up the flow files, but I've not
found anything for Kubernetes.
Ideally I would like to write the flow files to a PostgreSQL database, which is
already present in the Kubernetes cluster and is backed up. But from what I can
read (and I couldn't easily be wrong!) I don't think NiFi can persist flow
files in a PostgreSQL database?
The other option would be to write the flow files to object storage, like an S3
or Azure blob storage bucket, but I don't think NiFi can do that either?
My question for anyone else running NiFi on Kubernetes - how do you back up the
flow files so they can be resolved in the event that the PVCs are permanently
lost?
Thanks.
