slow-J commented on PR #16631:
URL: https://github.com/apache/lucene/pull/16631#issuecomment-5558608482
Ran knnPerfTest.py for 3 iterations each of baseline and candidate with
reindexing each time.
Setup: Cohere v3 wikipedia 1024d, 1M docs, maxConn=64, beamWidth=250,
topK=100, 10K queries, forceMerge. m6g.16xlarge, JDK 25.0.2.
Changed the `quantizeBits` to be only `(4,)`.
Set `INDEX_ONLY = True` and skipped search + stats.
Used the 1M doc test file.
```
doc_vectors={..}/cohere-v3-wikipedia-en-scattered-1024d.docs.first1M.vec
query_vectors={..}/cohere-v3-wikipedia-en-scattered-1024d.queries.first200K.vec
```
Indexing timings:
```
+ re_index_time = re.compile(r"^reindex takes ([\d.]+) sec \(plus
([\d.]+) sec waiting for merges\)")
+ re_force_merge_time = re.compile(r"^Force merge done in ([\d.]+) sec")
+ re_index_path = re.compile(r"^Index Path = (.+)$")
```
`index + merge_wait + force_merge` summed over all builds:
rep | baseline | candidate | delta
-- | -- | -- | --
1 | 4841.2 s | 4652.3 s | −3.90%
2 | 4807.1 s | 4643.1 s | −3.41%
3 | 4593.7 s | 4526.9 s | −1.45%
Looks better for the rerank=false only run.
| rep | baseline | candidate | delta |
| --- | --- | --- | --- |
| 1 | 858.12 s | 821.60 s | −4.26% |
| 2 | 852.94 s | 802.14 s | −5.96% |
| 3 | 858.25 s | 829.70 s | −3.33% |
The `rerank=false` phase split: index +0.47% (t=0.59), merge_wait −7.72%,
force_merge −7.01%.
For all 4:
| rerank | total delta | t |
| --- | --- | --- |
| false | −4.51% | −5.93 |
| true, rqb=4 | `−10.44% | −2.17 |
| true, rqb=8 | −0.88% | −0.52 |
| true, rqb=32 | +0.90% | 0.94 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
