klesh commented on issue #1433: URL: https://github.com/apache/incubator-devlake/issues/1433#issuecomment-1178616889
@hezyin @CamilleTeruel The investigation of github graphql shows that it is a promising direction: 1. it has a much higher rate limit, the algorithm of rate limit calculation can be found here https://docs.github.com/en/graphql/overview/resource-limitations 2. it can request multiple and nested resources in one request, so the total number of requests can be much lower, thus potentially much faster. Based on information from @likyh, we decided to expand the investigation scope from `github`, to other data sources as well, to find out the availability of graphql among different data sources, we may consider bringing in graphql if more than one data sources are supporting it, and share the similar features (in terms of higher rate limit and multiple/nested resources support). However, the github graphql rate limit calculation is quite complex, so it may not be possible to be converted to a steady rate. Although the Lazy RateLimit Strategy could be applied here, it leads to a UX complication. We need input from @Startrekzky and @yumengwang03 . @likyh suggested we should adopt a Dynamic Rate Control Algo like Binary Back Off sth in that nature. I'm afraid this would make the Plugin Interface further complex since rate limit information is data-source-specific. Please take these factors into account for the Plugin Development Improvement plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
