ryankert01 opened a new pull request, #1480:
URL: https://github.com/apache/mahout/pull/1480
### Related Issues
Part of #1477 (3/5). Builds on #1479: this PR's diff against `main` includes
that PR's commits; review the last commit only, or diff against
`qdp-one-path/2-kernel-registry`. **Behaviour changes below; please read.**
### Changes
- [x] Refactoring
- [x] New feature: phase from CUDA tensors, basis from float64 CUDA tensors,
streaming Parquet for all encodings, NumPy float32 input
### Why
Each encoder is implemented eight times (f32/f64 × single/batch ×
host/device) in Rust, and the PyO3 layer hand-matches (encoding, ndim, dtype)
onto those methods. #1268 added one f32 variant of angle and touched 9 files
for 1,135 lines. This PR collapses the matrix so an encoding is one descriptor.
### How
- `gpu::kernels`: a `Kernel` trait with one `launch` on device-resident
data, `Input` (host f64/f32 slice, or a device pointer on a stream), `Shape`,
and a single `encode` driver that validates, uploads (dual-stream pipeline for
large host batches), allocates and launches. A single sample is a batch of one.
- Launches are asynchronous: `launch` returns a `Pending` (buffers the
kernels read, plus checks that need device results such as validation and norm
flags), settled after one stream sync. The Parquet streaming path keeps
copy/compute overlap for every encoding.
- One descriptor per encoding. Amplitude uses the wide single-sample
reduction for one sample and a CPU norm with a single kernel for small host
samples; its finalize kernel raises a flag and writes 0 for zero or non-finite
norms. Basis casts host indices on the CPU. The angle encoder goes from 868
lines to under 90.
- `QdpEngine::encode(input, shape, num_qubits, encoding)` replaces 24 named
methods; host-batch and f64/int64 device-pointer shorthands remain in
`compat.rs`.
- Bindings: `PyInput::from_py` converts lists, NumPy (f64, f32), CPU tensors
and CUDA tensors; `QdpEngine.encode` is one call. CUDA dtype acceptance is
derived from `Kernel::supports`.
- Deleted: `QuantumEncoder`, `gpu/encodings/*`,
`encoding/{amplitude,angle,basis}.rs`.
**Behaviour changes**
1. `encode_from_parquet` / `encode("file.parquet")` now honours engine
precision (previously always complex128). Conversion happens through a bounded
float64 staging buffer, so peak memory is lower than before, not higher.
2. NumPy float32 and CPU float32 tensors are accepted on the host path.
Under a float64 engine they are widened on the CPU first, so float64 accuracy
is kept.
3. Phase encodes from CUDA tensors; basis accepts float64 CUDA tensors.
4. Streaming Parquet works for iqp, iqp-z and phase.
Error message wording was kept wherever a test asserted it. The public
Python API is otherwise unchanged; the existing suites pass unmodified apart
from the cases above.
Verification: Rust workspace tests parallel and `--test-threads=1`, 378
Python tests including the parity grid. Performance: see #1477; no encoding is
measurably slower on the repo's standard benchmarks, and CUDA-tensor paths are
3–12× faster because validation no longer round-trips inside the launch.
## Checklist
- [x] Added or updated unit tests for all changes
- [x] Added or updated documentation for all changes
---
Part of a five-PR series from the same branch, each adding one commit on top
of the previous; the diff shown is cumulative against `main`. Series: #1478 →
#1479 → this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
