Oh, I got what you said. But before used in the product env, we have to assess the performance about the collector cluster.
ES itself is ok, because it is used a lot as the factual standards, we could only care about how to adjust its config. Does the data transfer between agent and collect use the tcp long connection? (It seems used long connection based on jetty). Then how about the efficiency between collector and es? What's about its mechanism? I mean, if the input request rate from collector gt the process rate to es, what will happen? Is there exist disk cache or similar way to process? It seems not appropriate to use memory to do this, data is too much. In the product env of internet, this is very common: there will be 100+ micro-service instance, and the qps will be 20K total around, 1K qps at least in the high load instance, and it will increase of course in the future. According to the performance report, the 1K qps per instance, then we need assess the collector scale. If these conditions are satisfied as below: a.Trace data transport is based on tcp long connection. b.The package about trace is not large. c.Collector have mechanism to write trace data to es in the high efficiency. Then 10K qps per collector is very ok, I think. Is that mean, we need at least three collector nodes in the env that based on giving consideration to usability of collector cluster and request qps scale? At last, I mean, is there some advice or default config about collector cluster scale based on the request scale above? [ Full content available at: https://github.com/apache/incubator-skywalking/issues/1596 ] This message was relayed via gitbox.apache.org for [email protected]
