[
https://issues.apache.org/jira/browse/NIFI-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noura Ali updated NIFI-11891:
-----------------------------
Description:
Currently, when ListGCS runs for the first time, it lists all the available
files in the specified bucket directory, and when it runs in the next times, it
lists the files created/modified since last run, using the state stored in the
processor.
What is required is that every time the processor runs(runs after first one),
to list all existing files in GCS specified directory, not the files
created/modified since last run.
As I understand this is called "No Tracking" listing strategy that is available
in most List* processors, but it doesn't exist in ListGCSBucket.
Without this capability, I had to build a whole extra pipeline to stop the
processor–> clear the state -->start processor, before the ListGCS runs, which
is prone to issues with extra work and maintenance effort.
Appreciate your help and support.
was:
Need that every time the processor runs, to list all existing files in GCS, not
the files created/modified since last run.
As I understand this is called "No Tracking" listing strategy that is available
in most List* processors, but it doesn't exist in ListGCSBucket.
Without this capability, I had to build a whole extra pipeline to stop the
processor–> clear the state -->start processor, before the ListGCS runs, which
is prone to issues with extra work and maintenance effort.
Appreciate your help and support.
> Add No Tracking Strategy to ListGCS
> -----------------------------------
>
> Key: NIFI-11891
> URL: https://issues.apache.org/jira/browse/NIFI-11891
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Noura Ali
> Priority: Major
>
> Currently, when ListGCS runs for the first time, it lists all the available
> files in the specified bucket directory, and when it runs in the next times,
> it lists the files created/modified since last run, using the state stored in
> the processor.
> What is required is that every time the processor runs(runs after first one),
> to list all existing files in GCS specified directory, not the files
> created/modified since last run.
> As I understand this is called "No Tracking" listing strategy that is
> available in most List* processors, but it doesn't exist in ListGCSBucket.
>
> Without this capability, I had to build a whole extra pipeline to stop the
> processor–> clear the state -->start processor, before the ListGCS runs,
> which is prone to issues with extra work and maintenance effort.
>
> Appreciate your help and support.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
