klesh opened a new issue, #3756: URL: https://github.com/apache/incubator-devlake/issues/3756
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement. ### Use case For users who have to collect data from a slow JIRA Server Instance, the chance of pipelines failing is too high to cope: 1. API timeout too many times causing the RETRY mechanism to give up. 2. High concurrency causing jira API to be even slower or crash In this situation, it is nearly impossible for the user to collect jira data, it would fail over and over again for any reasonable jira board. Apache Devlake should be able to cope with the situation: 1. Be able to collect data even if the API was insanely slow 2. Be able to collect data ASAP even if the API response time fluctuates from time to time ### Description 1. set RETRY to a relative high value 2. keep track of the RETRY history and adjust the RATE LIMIT dynamically ### Related issues _No response_ ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
