Pranshu-S commented on PR #16731:
URL: https://github.com/apache/lucene/pull/16731#issuecomment-6442671253
> However, I feel like the duplicate ratio across docs in a sane dataset
would be <10% -- can you perform one run along these lines? (preferably with
more vectors, say 100K)
Here is the result from a skewed 100k result set with 10% redundant copies
| Workload | Distinct R@10 | Document R@10 (tie-aware) |
| --- | --- | --- |
| All-unique control (100K docs, 0% duplicates) | 0.9985 ± 0.0002 | 0.9985
± 0.0002 |
| **Skewed duplicates (100K docs, 10% duplicates, Zipf s=1, largest group
828)** | **0.9320 ± 0.0133** | **0.9357 ± 0.0132** |
Note the number of queries where the R < 0.1 -> shows a formation of a sink
but not as catastropic however sever enough to cause a 0.7 point recall
degradation
| Build | Distinct R@10 | Queries with R ≤ 0.1 (distinct) | Queries with R =
1.0 (distinct) | Document R@10 (tie-aware) | Queries with R ≤ 0.1 (tie-aware) |
Queries with R = 1.0 (tie-aware) |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 0.9310 | 38 | 703 | 0.9360 | 34 | 722 |
| 2 | 0.9192 | 57 | 733 | 0.9224 | 55 | 757 |
| 3 | 0.9457 | 39 | 818 | 0.9487 | 36 | 830 |
| Mean | 0.9320 | | | 0.9357 | | |
So yes, this behaviour is also in 10%, but not has severe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
