Greetings everyone,
After building the Solr HNSW index on 9.8.0, I tried to optimize it to a
single segment. However the optimization doesn't happen and no errors
found in solr log.
Schema:
<fieldType name="knn_vector_float_256" class="solr.DenseVectorField"
vectorDimension="256" similarityFunction="dot_product" vectorEncoding=
"FLOAT32" hnswMaxConnections="16" hnswBeamWidth="200"/> <fieldType name=
"knn_vector_byte_256" class="solr.DenseVectorField" vectorDimension="256"
similarityFunction="dot_product" vectorEncoding="BYTE" hnswMaxConnections=
"16" hnswBeamWidth="200"/> <field name="item_vector_float" type=
"knn_vector_float_256" indexed="true" stored="false"/>
<field name="item_vector_byte" type="knn_vector_byte_256" indexed="true"
stored="false"/>
Solrconfig for merge:
<mergePolicyFactory class=
"${mergePolicyFactory:org.apache.solr.index.TieredMergePolicyFactory}">
<int name="maxMergeAtOnce">25</int> <int name="segmentsPerTier">25</int> <
int name="maxMergedSegmentMB">100000</int> </mergePolicyFactory>
In total 30M docs are indexed and I trigger optimization with
/update?optimize=true&waitSearcher=false&maxSegments=1. When using the BYTE
type, optimization finishes successfully. Howeverwith FLOAT32 type it does
not work, and ends up with ~33G index in ~80 segments. Also I noticed there
isn't any reduction in segment number.
The solr node has ~200G memory and plenty of disk space.
Thanks,
Wei
