Using elastic search index stats apihttps://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html#indices-stats <https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html#indices-stats>
> 2020年4月17日 下午11:49，Bruno Roustant <bruno.roust...@gmail.com> 写道： > > Hi, > > Next time please reply/send these technical questions to > java-u...@lucene.apache.org <mailto:java-u...@lucene.apache.org> or > dev@lucene.apache.org <mailto:dev@lucene.apache.org> to get more audience. > > 538 segments seems huge. What is your configuration to get this number of > segments? > > Le jeu. 16 avr. 2020 à 16:28, 张海雷 <zhanghai...@bytedance.com > <mailto:zhanghai...@bytedance.com>> a écrit : > > >> >> Hi, all >> In our Elastic Search benchmark, found out that IndexReader‘s >> registerParentReader method takes a lot of CPU time >> Environment: >> ES Version: 7.5.1 >> Lucene Version: 8.3.0 >> >> Index: >> Shard Num 1 >> Replicator’s Num:1 >> Segment Count: 538 >> Size:49.5GB >> >> Benchmark result: qps:5200,pct99 600ms >> CPU Util:93% >> >> Use flame graph find the hot method IndexReader‘s registerParentReader. >> dive into code find out that Have a race condition that on the >> synchronizedSet Because of only one >> Shard and one shard have too many segments, this intensify competition >> >> >> >> >> To verify this assumption,execute force merge on this index，after force >> merge: >> QPS: more then 5600 >> PCT99: 60ms >> CPU Util: 11% >> The performance improved by a factor of 10x >> >> So we can use ConcurrentWeakKeyHashmap instead of synchronizedSet to reduce >> lock competition? >> >> >> >> >> >
