[
https://issues.apache.org/jira/browse/NIFI-7971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dennis Jaheruddin updated NIFI-7971:
------------------------------------
Summary: List-Fetch-Delete pattern (was: Create List-Fetch-Delete pattern)
> List-Fetch-Delete pattern
> -------------------------
>
> Key: NIFI-7971
> URL: https://issues.apache.org/jira/browse/NIFI-7971
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.12.1
> Reporter: Dennis Jaheruddin
> Priority: Minor
>
> *Challenge*
> There are several processors for getting data. Originally we had GetHDFS,
> GetSFTP and these are now being phased out for List Fetch processors.
> I think the List-Fetch pattern is a significant improvement, but what I see
> now is that people are looking for a List-Fetch-Delete pattern.
> *Example usecase:*
> Gather data, process and write in destination, delete
> There are several ways to achieve this (native but inefficient ones, like 'do
> a second Fetch with remove source file and terminate success', or efficient
> but scripted ones, like 'use the executestreamprocessor to send the delete
> command').
> As far as I understand, this would not be needed when using Stateless Nifi,
> but that is not relevant for many usecases.
> *Proposal*
> However, I propose we find a solution that allows people to easily achieve a
> List-Fetch-Delete pattern.
> Two lines of thought:
> # Expand Fetch processors with a 'deleteonly' mode, which would not actually
> copy in the data
> # Create dedicated processors, like 'DeleteSFTP' or 'ExecuteSFTP'
> I have written the example for SFTP, but think this would be a natural
> extension for most places where we have the List-Fetch pattern currently.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
