Re: Updating schedule information using REST API without disturbing other processor configuration

[Koji Kawamura]

Hi Ravi,

How does your request JSON sent to the PUT /processors/{id} endpoint look like?
If you don't need to update any processor properties, then you don't
have to send /component/config/properties element in a request JSON.
You can debug how NiFi UI sends REST requests using web browser
Developer tools [1].

[1] https://developers.google.com/web/tools/chrome-devtools/

Thanks,
Koji

On Sat, Jan 27, 2018 at 7:41 AM, Ravi Papisetti (rpapiset)
 wrote:
> Hi,
>
> We are trying to update "Run Schedule" and "Scheduling Strategy" of a NiFi
> Processor using REST API (PUT /processors/{id}). This works for most of the
> processors, but when it is updated for GetSFTP, it overwrites password field
> that results authfail exception. If I update scheduling information using
> NiFi UI, it works fine.
>
> Any thought how we can update scheduling information without bothering about
> other processor configuration.
>
> Appreciate any help.
>
> Thanks,
>
> Ravi Pipisetti
>
>

Re: Generalizing a List / Fetch workflow

[Jeff]

It might take a bit more than allowing input on that processor.  One thing
you might want to check is how the processor retains state for the path
given in the configuration.  If this path changes regularly due to EL or
incoming flowfiles, the state management for the processor might not be
consistent.  From memory, I believe the processor only stores timestamps
related to the path, but not the path itself.  That's most likely one of
the reasons why the processor does not allow incoming flowfiles; the state
of multiple directories would have to be tracked with no way to
automatically clear unused/stale state, without NiFi being able to
determine of a path's state is still needed.  If you allow input in
ListFile, you'll have to make sure that state of the various paths gets
tracked properly.  There's potentially a sort of memory leak at that point,
although admittedly you'd have to send a lot of unique paths through that
processor before it became a problem.  At the moment, I don't have a good
strategy when the "stale" paths could be removed from state.

On Sat, Jan 27, 2018 at 7:21 PM James McMahon  wrote:

> This sounds promising. I can give it a try. Assuming this is the proper
> place to download the code for ListFile, can you tell me how to pull the
> correct source code? I've not yet done this. I would definitely go the
> route of fixed folder path with Expression Language support. -Jim
>
> On Sat, Jan 27, 2018 at 7:09 PM, Mike Thomsen 
> wrote:
>
>> It's input requirement is set to INPUT_FORBIDDEN. It shouldn't be too
>> hard to set that to INPUT_ALLOWED and make it able to handle a flowfile or
>> a fixed folder path (or even better, fixed folder path w/ EL support). If
>> you do a patch, I'll try to find time to do a review.
>>
>> On Fri, Jan 26, 2018 at 4:03 PM, James McMahon 
>> wrote:
>>
>>> My customer has posted a requirement for a generalized List / Fetch
>>> workflow. I would not be able to express the target directory in the
>>> ListFile as a fixed value because I would need to reset the input directory
>>> each workflow cycle, and the target input directories will not share any
>>> common parent.
>>>
>>> Has anyone recently explored an approach to feed the ListFile or
>>> equivalent its target input directory dynamically? It does not seem
>>> possible to do that. I've failed to figure this out before, not being able
>>> to determine a way to precede the ListFile in the workflow.
>>>
>>
>>
>