Hi Ravi,
To distribute QueryDatabaseTable workload, I'd suggest using
GenerateTableFetch instead. Because it can generate SQLs to query
updated records. And those SQL FlowFiles can be distributed among NiFi
nodes by RPG.
-
Following lines are just to share my thoughts on the topic for
Ravi
The pattern of List -> -> Fetch -> Merge -> Put is common
and is the correct way to use it.
ListFile can often be used across the cluster all at once and work out
fine since each node is often accessing a unique resource like a local
file system. However, in this case you're pointing them
Hi,
We are creating processgroups to transfer data from File system to HDFS/HBASE
(separate process groups for each destination).
Simple Example: FileSystem to HDFS process group: We have ListFile, FetchFile
and PutHDFS process to transfer data from File system to HDFS. When “ListFile”
