I have solved this problem using what you had provided (for the most part) Here is how i ended up doing it. 1 Processor group for keeping the oauth token up to date every 55 minutes GenerateFlowFile -> InvokeHTTP -> PutDistributedMapCache 1 processor group to List files and upload them List File -> Fetch File -> FetchDistrubtedMapCache -> InovkeHTTP
This way i always have an up to date token, and the flow for Invoking my HTTP request always works!! Thanks! On Mon, Aug 13, 2018 at 8:40 PM Andy LoPresto <[email protected]> wrote: > I also realize the sleep may seem extraneous with the flowfile > penalization that NiFi offers, but I am not sure if applying that penalty > is identical across relationships, so I didn’t want to penalize the > “not-found” relationship that went to the Get Token processor. You may be > interested in the open feature request for a PenalizeFlowFile processor I > found here [1]. > > [1] https://issues.apache.org/jira/browse/NIFI-4805 > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Aug 13, 2018, at 6:36 PM, Andy LoPresto <[email protected]> wrote: > > Hi Travis, > > I have not executed this flow myself, so this is hypothetical — could you > use the DistributedMapCache services to persist the token when you acquire > it and retrieve the token from there before running the second InvokeHTTP > processor (the “POST" processor)? You would remove the first InvokeHTTP > (the “token” processor) from the flow directly and have it alongside the > flow with a trigger every 55 minutes or whatever you like, which would > persist it to the cache. In your “main” flow, you can use a > FetchDistributedMapCache processor to pull the existing token and use that? > > I’ve attached a screenshot, but again, I have not run this flow to verify. > > In this case, the Get OAuth Token Every 55 min does exactly what it says. > Each time it runs, it puts that token value in the DMC (you probably need > an ExtractText or ReplaceText processor in between; I can’t remember if the > OAuth Token comes back in the response content or as a header). On the > right side, you have the ListFile -> FetchFile, and then it goes to Fetch > Token From DMC. If the token is there, it routes the flowfile to success > relationship and POSTs the file. If the token is not there, it sends the > flowfile to two processors: (realizing now this isn’t quite complete) the > Get Token processor and an ExecuteScript processor set up with a simple > Groovy script to sleep for 5 seconds. You would actually need to change how > the Get Token processor runs for this to work; it should be event driven, > and there should be a GenerateFlowFile processor leading to it that is > timer driven every 55 minutes. That way, either the normal scheduling > flowfile arriving or the “no token present” flowfile would trigger the > token retrieval. Anyway, once the token is back in the DMC, the sleep > processor should finish and the Fetch Token will try to run again, this > time getting the token and triggering the POST. > > Hope this works for you. If you have to many any adjustments to this, > please reply here to document what works for you. > > <Screen Shot 2018-08-13 at 6.30.23 PM.png> > > > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Aug 13, 2018, at 5:47 PM, Travis Vaske <[email protected]> wrote: > > I have a flow that looks like this > > list file -> get file -> invokehttp (gets oauth token) -> invokehttp (post > file with token) > > This works fine, but my token is good for a certain amount of time (maybe > 1 hour), is there a setting where 1 processor runs every 30 minutes (to get > the token) but when a get file triggers it, it will pass the token that i > already have without invoking every time? > > Thanks! > > > >
