I would also be interested in the question below. Anyone?
On 14/10/2015 05:17, [email protected] wrote:
I am researching to implement Change Data Capture(CDC) solution by observing MySQL binlogs and routing that data through NiFi to detect inserts , updates and deletes and finally storing/sync the data into NoSQL(MongoDB) and HDFS. For this use case the data in source system is normalized (OLTP) and sink system is denormalized (OLAP), so I have to accumulate some data in NiFi to compose aggregated record and store it into MongoDB. I would like to know if any NiFi users building CDC solution using NiFi? Is NiFi a good choice for this kind of dataflow / real-time ETL? I found some Java based CDC frameworks here: https://github.com/wushujames/mysql-cdc-projects/wiki Thanks Sumanth
