Hello Solr Community,
We are observing unexpected behavior when running an optimize command on a
multi-shard SolrCloud collection without specifying the maxSegments
parameter. Despite completing successfully, the process does not appear to
merge segments or reclaim disk space from deleted documents.
Environment & Topology :
Solr Version: 9.10.1 (SolrCloud mode)
Collection Name: collection1
Router: Implicit (router.field=sh)
Shards: 2
Replicas / Shard: 2 (TLOG replicas only)
Total Cores: 4
Collection Health: GREEN
All replicas ACTIVE
Shard Statistics (Prior to Request):
Shard 0:
Size: 56.71 GB
Total Documents: 39,741,327
Deleted Documents: 7,900,358 (16.58%)
Segment Count: 42
Shard 1:
Size: 58.86 GB
Total Documents: 32,003,917
Deleted Documents: 9,045,036 (22.03%)
Segment Count: 37
MergePolicy:
<mergePolicyFactory class="org.apache.solr.index.SortingMergePolicyFactory">
<str name="sort">_12 desc</str>
<str name="wrapped.prefix">inner</str>
<str
name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str>
<int name="inner.maxMergeAtOnce">10</int>
<int name="inner.segmentsPerTier">10</int>
<double name="inner.noCFSRatio">0.1</double>
</mergePolicyFactory>
<mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler">
<int name="maxMergeCount">5</int>
<int name="maxThreadCount">3</int>
</mergeScheduler>
Note: The collection uses SortingMergePolicyFactory wrapping
TieredMergePolicyFactory with index sorting on field "_12 desc".
Observed Behavior:
When invoking the update handler without arguments:
POST /solr/collection1/update?optimize=true
The request returns an HTTP 200 OK status almost immediately, but we
observe no state changes across any shard:
Deleted Documents: Retained (no reduction in percentage)
Segment Count: Remains 42 and 37 respectively
Disk Footprint: Zero space reclaimed
However, if we explicitly set maxSegments=1 (or specify a target segment
count):
POST /solr/collection1/update?optimize=true&maxSegments=1
The segment merging triggers as expected and purges the deleted documents.
Expected Behavior :
We expected /update?optimize=true to initiate a segment merge and reclaim
disk space from deleted documents by default, even when maxSegments is
omitted.
Questions
1. Default maxSegments Behavior: Since Solr 7.5+ / Lucene changes, what is
the default behavior of optimize=true when maxSegments is omitted?
2. Is the observed behavior expected when using SortingMergePolicyFactory
wrapping TieredMergePolicyFactory, particularly when optimize=true is
invoked without explicitly specifying maxSegments?
3. Merge Policy Thresholds: Are there specific TieredMergePolicy bounds
(e.g., deletesPctAllowed or maxMergeAtOnce) preventing segments with
~16–22% deleted documents from qualifying for natural merges?
4. Topology Impact: Could TLOG replica syncs, active transaction logs, or
soft-deletes interfere with optimize eligibility?
Any insights into the underlying TieredMergePolicy / forceMerge logic for
this scenario would be greatly appreciated.
Thanks,
Ayush Agrawal
