Hi,
For whatever it's worth, I've occasionally seen this, or something
similar, happening as well. Restarting Solr on the offending server
always fixes the issue until it starts happening again. For some reason
the problem seems to manifest after a rolling server restart (OS
updates). I haven't been able to find anything interesting in Solr logs.
--Ere
Jesús Roca kirjoitti 18.5.2022 klo 20.34:
Hello,
We are having problema
We have a cluster with Solr 8 (15 nodes running RHEL) and ZooKeeper 3.6.2
(5 nodes) and only one collection of around 48 millions of docs with 10
shards and a replication factor of 3, so every server holds 2 cores.
A couple of weeks ago we performed an upgrade from Solr 8.9.0 to 8.11.1 and
we upgraded the Java version too from 1.8.0_322 to 11.0.14 and since then
some cores are constantly growing even when the number of docs is
practically the same.
I have upload an image where you can see how the size of a core grows over
time (we upgraded on 05/02), and another graphic where you can see the docs
number is quite stable compared to their size:
https://we.tl/t-pvV4RfXhYe
Currently the size of that replica of shard2 is 22,6 GB while leader shard
is 9,66 GB
We don't have any custom merge policy in solr.config file, we are using the
default merge policy.
When we restart an instance with a core that grows indefinitely it release
space. For ex, I have restarted the instance which graphic I shared and now
its size is 9,9 GB.
But after a while (some days) some cores grow again.
I have upload a COLSTATUS too with information about shard leaders and
their segments.
Any ideas about this issue?
Thanks!
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
