Re: Retry logic for rest api - NIFI
Thanks Matt. I got it. We implemented a similar approach to retry in database failures. I will try that now. On Thu, Jul 30, 2020 at 3:31 PM Matt Burgess wrote: > Asmath, > > InvokeHttp routes the original flowfile to a number of different > relationships based on things like the status code. For example if > you're looking for a 2xx code but want to retry on that for some > reason, you'd use the "Original" relationship. If you want a retryable > code (5xx) you can use the "Retry" relationship, and so on. If you > need a specific code, you can use RouteOnAttribute to match the > "invokehttp.status.code" attribute to the code(s) you want, and all > that match can be sent back to the original InvokeHttp processor. > > That's a simpler pattern but they can get more complex. For example > you can start the flowfile with an attribute "retries" set to 5, and > before you send the flow file back to InvokeHttp, you'd decrement the > counter with UpdateAttribute and then perhaps drop the flowfile or > send it to some other mechanism after retries becomes zero. Also you > could put a delay in the flow so you're not retrying the same thing as > fast as possible (that could constitute a Denial-Of-Service attack on > the HTTP endpoint you're trying to reach). I can't remember which > relationships from InvokeHttp penalize the flowfile, so there's a > chance that the processor will handle the delay for you (see the > User's Guide section on penalized flowfiles). > > Regards, > Matt > > On Thu, Jul 30, 2020 at 3:38 PM KhajaAsmath Mohammed > wrote: > > > > can you please let me know how to use this in NIFI. > > > > On Thu, Jul 30, 2020 at 11:19 AM Otto Fowler > wrote: > >> > >> nipyapi does something like that: > https://github.com/Chaffelson/nipyapi/blob/164351ee2d92f8c4a75989310662bbad0f7bafc4/nipyapi/utils.py#L210 > >> > >> > >> > >> > >> On July 30, 2020 at 11:22:29, KhajaAsmath Mohammed ( > mdkhajaasm...@gmail.com) wrote: > >> > >> Hi, > >> > >> I am looking for some information on how to do retry logic on restapi > until we get specific status code. Please let me know if you have any > approach/templates for this > >> > >> Thanks, > >> Asmath >
Re: Retry logic for rest api - NIFI
Asmath, InvokeHttp routes the original flowfile to a number of different relationships based on things like the status code. For example if you're looking for a 2xx code but want to retry on that for some reason, you'd use the "Original" relationship. If you want a retryable code (5xx) you can use the "Retry" relationship, and so on. If you need a specific code, you can use RouteOnAttribute to match the "invokehttp.status.code" attribute to the code(s) you want, and all that match can be sent back to the original InvokeHttp processor. That's a simpler pattern but they can get more complex. For example you can start the flowfile with an attribute "retries" set to 5, and before you send the flow file back to InvokeHttp, you'd decrement the counter with UpdateAttribute and then perhaps drop the flowfile or send it to some other mechanism after retries becomes zero. Also you could put a delay in the flow so you're not retrying the same thing as fast as possible (that could constitute a Denial-Of-Service attack on the HTTP endpoint you're trying to reach). I can't remember which relationships from InvokeHttp penalize the flowfile, so there's a chance that the processor will handle the delay for you (see the User's Guide section on penalized flowfiles). Regards, Matt On Thu, Jul 30, 2020 at 3:38 PM KhajaAsmath Mohammed wrote: > > can you please let me know how to use this in NIFI. > > On Thu, Jul 30, 2020 at 11:19 AM Otto Fowler wrote: >> >> nipyapi does something like that: >> https://github.com/Chaffelson/nipyapi/blob/164351ee2d92f8c4a75989310662bbad0f7bafc4/nipyapi/utils.py#L210 >> >> >> >> >> On July 30, 2020 at 11:22:29, KhajaAsmath Mohammed (mdkhajaasm...@gmail.com) >> wrote: >> >> Hi, >> >> I am looking for some information on how to do retry logic on restapi until >> we get specific status code. Please let me know if you have any >> approach/templates for this >> >> Thanks, >> Asmath
Re: Retry logic for rest api - NIFI
can you please let me know how to use this in NIFI. On Thu, Jul 30, 2020 at 11:19 AM Otto Fowler wrote: > nipyapi does something like that: > https://github.com/Chaffelson/nipyapi/blob/164351ee2d92f8c4a75989310662bbad0f7bafc4/nipyapi/utils.py#L210 > > > > > On July 30, 2020 at 11:22:29, KhajaAsmath Mohammed ( > mdkhajaasm...@gmail.com) wrote: > > Hi, > > I am looking for some information on how to do retry logic on > restapi until we get specific status code. Please let me know if you have > any approach/templates for this > > Thanks, > Asmath > >
Re: Retry logic for rest api - NIFI
nipyapi does something like that: https://github.com/Chaffelson/nipyapi/blob/164351ee2d92f8c4a75989310662bbad0f7bafc4/nipyapi/utils.py#L210 On July 30, 2020 at 11:22:29, KhajaAsmath Mohammed (mdkhajaasm...@gmail.com) wrote: Hi, I am looking for some information on how to do retry logic on restapi until we get specific status code. Please let me know if you have any approach/templates for this Thanks, Asmath
