mccullocht commented on PR #14978:
URL: https://github.com/apache/lucene/pull/14978#issuecomment-3114145559
I ran a macrobenchmark on 1M 3072 dim vectors or ~12GB using angular
similarity and k=128 on an M2 Mac:
Baseline (lucene main): 0.005832984s avg latency
Experiment (native RandomVectorAccessor + prefetching): 0.003986186s avg
latency
The improvement is about 30%. I think this improvement will be durable on
other processors because I inlined prefetching in the dot computation so I'm
not relying on the CPU to decode instructions way ahead and reorder:
```rust
/// Score `q` against `d`; prefetch the vector for `p` (next to score).
pub fn score_and_prefetch(&self, q: &[f32], d: &[f32], p: &[f32]) -> f32
{
match self {
Self::DotProduct => {
let dot = unsafe {
let mut dot = vdupq_n_f32(0.0);
for i in (0..q.len()).step_by(4) {
if i % 16 == 0 {
_prefetch::<_PREFETCH_READ, _PREFETCH_LOCALITY3>(
p.as_ptr().add(i) as *const i8
);
}
let qv = vld1q_f32(q.as_ptr().add(i));
let dv = vld1q_f32(d.as_ptr().add(i));
dot = vfmaq_f32(dot, qv, dv);
}
vaddvq_f32(dot)
};
0.0f32.max((1.0f32 + dot) / 2.0f32)
}
_ => self.score(q, d),
}
}
```
I will try to stand this up on a graviton host and confirm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
