[
https://issues.apache.org/jira/browse/NIFI-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklas resolved NIFI-5861.
--------------------------
Resolution: Won't Fix
Just noticed that the DetectDuplicate doesn't actually remove things from cache.
> Notify: Add support for age of duration
> ---------------------------------------
>
> Key: NIFI-5861
> URL: https://issues.apache.org/jira/browse/NIFI-5861
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Niklas
> Priority: Major
> Original Estimate: 4h
> Remaining Estimate: 4h
>
>
> Since the Notify processor is writing to the cache it would make sense to
> have a "age of duration" property like the DetectDuplicate processor.
>
> For example: I've 2 seperate flows which put Data from one DB to another DB.
> Flow 2 depends on Flow 1 due to Foreign Key. Both flows have
> GenerateTableFetch which listen to Updates in source DB.
> At the end of Flow 1 it'll put all INSERTED IDs to a Notify Processor which
> put everything in the Cache.
> Flow 2 runs simultaneously and it'll check if the Foreign Key Value exist in
> the DB that it'll stream the data. If the key doesn't exist it'll be routed
> to a DetectDuplicate and if it's a Duplicate (this ID already run in Flow 1
> but isn't inserted yet) it'll be routed to a Wait Processor. The wait
> processor now waits for the Notify node on Flow 1.
>
> This whole construct works fine. The wait processor removes the key from the
> cache if it's match but I'm writing all IDs to the cache, even IDs that
> doesn't hang in a wait processor. I've tried to do a workaround with
> DetectDuplicate Processor that it'll only use Notify for things that hang in
> a wait processor but it didn't work that great due to bad timings.
>
> It would be good to have a TTL setting similar to the DetectDuplicate
> processor for the Notify processor. Additionally to this issue is that the
> Nifi Cache Server doesn't have a default TTL option to clear the cache
> automatically.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
