You'd only need to do that if you have strict ordering requirements like
reading directly from a transaction log and replicating it. If yes I'd
skip nifi unless your also doing other cases with it.
Sounds like Matts path gets you going though so that might work out just
fine.
Thanks
Joe
On Oct
Thanks Joe and Matt.
@Joe, based on your comment, I need to use NiFi as a producer which puts
the data on Kafka queue and then have NiFi consumer, which writes the data
back to the destination. Is my understanding correct?
@Matt, My use case is for the DynamoDB. I will look into whether
incrementa
Rai,
There are incremental data movement processors in NiFi depending on
your source/target. For example, if your sources are files, you can
use ListFile in combination with FetchFile, the former will keep track
of which files it has found thus far, so if you put new files into the
location (or up
Rai,
NiFi can certainly be used for some data replication scenarios and
quite often is. If you can treat the source like a continuous data
source then there is some way to keep state about what has been pulled
already, what has changed or needs yet to be pulled, and it can just
keep running then
Hi All,
I am learning NiFi as well as trying to deploy it in production for few
use cases. One of the use case is ETL and another use case is, using NiFi
as a backup solution, where it takes the data from one source and moves to
another database|file. Is anyone using NiFi for this purpose? Does Ni
