Took a look at the pull request and that should handle the issue I was seeing. Is there going to be an issue with the directory being left there if something else fails? For my case the empty directory is fine.
Thanks Shawn Weeks ________________________________ From: Bryan Bende <[email protected]> Sent: Wednesday, October 17, 2018 10:48:20 AM To: [email protected] Subject: Re: FetchFile Completion Strategy Hi Shawn, The way the processor works is the following... 1) Perform checks to try and ensure the completion strategy will work later 2) Perform the fetch and if successful then transfer to success and commit the session 3) Perform the completion strategy The issue is if the completion fails in step 3, it is too late to route to failure because in step 2 the session was already committed. The reason it is done this way is because if we fetched the file without committing the session and then executed the completion strategy, it would be possible NiFi could crash at this moment before committing the session, and now the source file could be gone from the completion strategy and the data is also gone from NiFi since it was never committed, so there would be data loss. All that being said, I think I was able to figure out an improvement to protect against the case where there is a permission problem... https://github.com/apache/nifi/pull/3088 Thanks, Bryan On Mon, Oct 15, 2018 at 4:10 PM Shawn Weeks <[email protected]> wrote: > > I was just wondering if it's expected behavior for the FetchFile Processor to > eat errors related to the completion strategy. If for example your completion > strategy is move and you specify a directory that doesn't exist or that NiFi > doesn't have permissions on I'd have expected the Processor to route to > failure not success. > > > Thanks > > Shawn Weeks
