I filed OPENNLP-1895 as a follow-up to the static embedding engine proposed in OPENNLP-1877 (PR #1152, still in review).
The idea: static embedding tables are big (a 500k x 300 float32 table is about 600 MB), and recent work on scalar quantization gets near-optimal distortion at 2-4 bits per dimension. That would put the same table at 38-75 MB with measured, small quality loss, and make embedding faster, since the workload is memory-bound and we would read 4-16x fewer bytes. The specific construction I want to implement is TurboQuant (arXiv:2504.19874): a seeded randomized Hadamard rotation, per-coordinate optimal scalar quantizers, and an optional 1-bit residual that makes inner product estimates unbiased. It is a small amount of code: the rotation, the precomputed quantizer grids, and a packed 2/4-bit decode kernel. Pure JVM, implemented from the paper with citation, no new dependencies, no ports of existing implementations. Scope is deliberately narrow: quantized table storage and lookup behind the same interface as the float table, plus an offline converter CLI. No approximate nearest-neighbor indexing; that belongs a the search engine layer, not OpenNLP. The PR would come with measurements: round-trip distortion and a doccat parity fixture at each bit width, throughput and memory vs the float table, and an honest comparison against a Rust implementation of the same algorithm on identical tables so we know what the JVM costs us. This stacks conceptually on OPENNLP-1877, so I would not open the PR until that review has moved. I'm raising this now for early opinions on scope and where the file format should live. Thoughts? Kristian
