tzssangglass commented on issue #7919: URL: https://github.com/apache/apisix/issues/7919#issuecomment-1247593588
> 3\. I disabled the `proxy-mirror` plugin, but it's helpless, the P99 latency is still high. Can you show the grafana monitoring after removing the `proxy-mirror` plugin? I am interested in the average latency of about 5s. This is not normal. Are there some error logs? It looks like you captured the exception stack. The phenomena and conclusions about `prometheus` plugins causing P99 long-tail requests are known. ref: #5755 Note: This phenomenon usually occurs when the number of indicators exceeds 10,000. Based on previous experience, the latency of long-tail requests is nearly a hundred times higher than the average latency(with an average delay of a few tens of milliseconds and a few seconds for long-tail requests.). But your garfana monitoring shows: long-tail requests are about four times more likely to be delayed than the average. This makes me feel strange(with an average latency of 5 seconds and 20 seconds for long-tail requests). As a comparison test, you can remove the proimetheus plugin and test again to see if the average 5s latency and the P99 latency appear. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
