ColinLeeo opened a new pull request, #16907:
URL: https://github.com/apache/lucene/pull/16907
### Description
The Java-backed tests in #16795 exercise dispatch and argument/result
handling, but cannot detect failures in real library loading, symbol lookup, or
the FFM calling convention. This adds the small native-integration follow-up
discussed in #15840, based directly on current `main` and the native contract
from #16812. It can be reviewed independently of #16795.
A scalar C fixture implements six representative functions: byte dot product
and cosine, float dot product/distance/cosine, and in-place normalization. The
build produces both `lucene_`-prefixed and unprefixed variants. Native counters
and argument observations verify that calls really reached C, including element
counts and the normalization flag. The fixture checks lengths before accessing
arrays, so an incorrect length fails a Java assertion instead of reading
outside a heap array.
Five tests each launch a fresh JVM to cover:
- Real calls through array and static `MemorySegment` entry points, using
heap and off-heap byte segments, empty inputs, and boundary lengths; fractional
cosine results and normalization's null-pointer/exception behavior.
- Fallback for missing symbols, and rejection of unprefixed symbols even
when the library loads successfully.
- Missing optional and required libraries, and strict-mode failure on a
missing prefixed symbol.
`-Ptests.nativevector.run=true` builds the fixtures with `cc` on Linux/macOS
and enables the tests. `-Plucene.tool.cc=<compiler>` selects another compiler.
Ordinary test runs skip this suite and do not invoke a C compiler. A dedicated
workflow runs the integration suite on Linux x86-64 and AArch64.
Production Java and C implementations are unchanged. This is an initial
real-library test harness, not complete native coverage: the remaining 16
functions, strict-mode success with a complete library, and Windows fixture
builds remain follow-up work. Relates to #15840.
### Validation
On macOS AArch64, Temurin 25.0.5+8, with Clang:
```sh
./gradlew :lucene:core:test \
--tests 'org.apache.lucene.internal.vectorization.*' \
--tests 'org.apache.lucene.util.TestVectorUtil' \
:lucene:core:check \
-Ptests.nativevector.run=true \
-Ptests.defaultvectorization=true -Pvalidation.errorprone=true
```
- 369 tests, no failures, 17 existing skips; all five integration tests ran.
Seed: `A6D22205733C9209`.
- Default mode also passed with `lucene.tool.cc` set to a nonexistent
compiler: 369 tests, 22 skips (the five integration tests are disabled), with
no native compilation tasks run.
- Core checks passed with Error Prone enabled. Repository checks on the
changed files passed, including actionlint, zizmor, ast-grep, and
spelling/whitespace checks.
- Isolated mutation checks confirmed failures for byte counts passed to
float operations, an incorrect byte-cosine return descriptor, ignored null
normalization results, missing symbol prefixes, and an incorrect normalization
flag. These experiments did not modify the repository's production sources.
- All five child-JVM scenarios also passed when compiled against the
implementation from #16795, confirming compatibility with its lazy native
holder.
Linux results will come from the new CI jobs. The full repository test suite
has not been run locally.
### Review status
Prepared with Codex assistance under the author's direction. This draft is
for review of the native-test approach and its CI/build integration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
