I filed OPENNLP-1895 as a follow-up to the static embedding engine proposed
in OPENNLP-1877 (PR #1152, still in review).

The idea: static embedding tables are big (a 500k x 300 float32 table is
about 600 MB), and recent work on scalar quantization gets near-optimal
distortion at 2-4 bits per dimension. That would put the same table at
38-75 MB with measured, small quality loss, and make embedding faster, since
the workload is memory-bound and we would read 4-16x fewer bytes.

The specific construction I want to implement is TurboQuant
(arXiv:2504.19874): a seeded randomized Hadamard rotation, per-coordinate
optimal scalar quantizers, and an optional 1-bit residual that makes inner
product estimates unbiased. It is a small amount of code: the rotation, the
precomputed quantizer grids, and a packed 2/4-bit decode kernel. Pure JVM,
implemented from the paper with citation, no new dependencies, no ports of
existing implementations.

Scope is deliberately narrow: quantized table storage and lookup behind the
same interface as the float table, plus an offline converter CLI. No
approximate nearest-neighbor indexing; that belongs a the search engine
layer, not OpenNLP.

The PR would come with measurements: round-trip distortion and a doccat
parity fixture at each bit width, throughput and memory vs the float table,
and an honest comparison against a Rust implementation of the same algorithm
on identical tables so we know what the JVM costs us.

This stacks conceptually on OPENNLP-1877, so I would not open the PR until
that review has moved. I'm raising this now for early opinions on scope and
where the file format should live.

Thoughts?

Kristian

Reply via email to