Hi everyone, I'm wondering if there is a Detect Duplicate processor that can read records from a flow file and as output gives just the non-duplicates (can be single records or a group of non-duplicates would be better). I want to use a record reader to avoid splitting the json content into 10000s of flow files to detect the duplicates. Immediately after this flow is a record reader/writer going to HBase.
Jeremy
