It is possible the directory could get created and then something else
fails, but personally I can't think of any downside to that, and seems
like a better trade-off than routing to success, but if anyone else
has thoughts please let me know.
On Wed, Oct 17, 2018 at 12:07 PM Shawn Weeks <swe...@weeksconsulting.us> wrote:
>
> Took a look at the pull request and that should handle the issue I was
> seeing. Is there going to be an issue with the directory being left there if
> something else fails? For my case the empty directory is fine.
>
>
> Thanks
>
> Shawn Weeks
>
> ________________________________
> From: Bryan Bende <bbe...@gmail.com>
> Sent: Wednesday, October 17, 2018 10:48:20 AM
> To: users@nifi.apache.org
> Subject: Re: FetchFile Completion Strategy
>
> Hi Shawn,
>
> The way the processor works is the following...
>
> 1) Perform checks to try and ensure the completion strategy will work later
> 2) Perform the fetch and if successful then transfer to success and
> commit the session
> 3) Perform the completion strategy
>
> The issue is if the completion fails in step 3, it is too late to
> route to failure because in step 2 the session was already committed.
>
> The reason it is done this way is because if we fetched the file
> without committing the session and then executed the completion
> strategy, it would be possible NiFi could crash at this moment before
> committing the session, and now the source file could be gone from the
> completion strategy and the data is also gone from NiFi since it was
> never committed, so there would be data loss.
>
> All that being said, I think I was able to figure out an improvement
> to protect against the case where there is a permission problem...
>
> https://github.com/apache/nifi/pull/3088
>
> Thanks,
>
> Bryan
>
> On Mon, Oct 15, 2018 at 4:10 PM Shawn Weeks <swe...@weeksconsulting.us> wrote:
> >
> > I was just wondering if it's expected behavior for the FetchFile Processor
> > to eat errors related to the completion strategy. If for example your
> > completion strategy is move and you specify a directory that doesn't exist
> > or that NiFi doesn't have permissions on I'd have expected the Processor to
> > route to failure not success.
> >
> >
> > Thanks
> >
> > Shawn Weeks
