Andrzej Bialecki created SOLR-12941:
----------------------------------------
Summary: IndexSizeTrigger and splitMethod=link problems
Key: SOLR-12941
URL: https://issues.apache.org/jira/browse/SOLR-12941
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.6, master (8.0)
Reporter: Andrzej Bialecki
Assignee: Andrzej Bialecki
{{IndexSizeTrigger}} can be configured to use {{splitMethod=link}}
(SOLR-12730), which uses hard-linking for creating sub-shards.
However, if the trigger uses {{aboveBytes}} condition the resulting sub-shards
will not immediately decrease in size, until all of the deleted documents will
be expunged (either by gradual merges or by explicit and costly expungeDeletes
command). As a result the new sub-shards will still exceed the {{aboveBytes}}
threshold, which will cause the trigger to keep generating new split requests.
I see two options how to solve this:
* disallow using {{aboveBytes}} with {{splitMethod=link}}. This unfortunately
is a very desirable combination because it monitors the actual index size and
uses the fast splitting method.
* calculate an internal estimate of "eventual index size" for an index with
deletions, and use this estimate when checking with {{aboveBytes}} instead of
the real index size. This of course introduces a potentially significant
estimation error but allows to properly treat hard-linked sub-shards with
deletions as (eventually) significantly smaller than the parent shard.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
