viiccwen opened a new pull request, #1464:
URL: https://github.com/apache/mahout/pull/1464
### Related Issues
Closes #1463
### Changes
- [x] Bug fix
- [ ] New feature
- [ ] Refactoring
- [ ] Documentation
- [x] Test
- [ ] CI/CD pipeline
- [ ] Other
### Why
The streaming Parquet encoder passed an `f64` element count to a helper that
expects a byte count. Consequently, amplitude and angle encoders copied only
one eighth of every input chunk to GPU staging memory. The CUDA copy completed
successfully, so callers received a state tensor with no error even though its
amplitudes were incorrect.
Existing integration tests only checked the returned DLPack shape and
therefore did not detect this numerical corruption.
### How
- Convert the Parquet chunk length from `f64` elements to bytes before
calling `PipelineContext::async_copy_to_device`.
- Use `checked_mul(size_of::<f64>())` and return a memory-allocation error
if the byte-count calculation overflows.
- Extend the existing angle streaming integration test instead of adding a
separate overlapping test:
- encode three distinct angle samples from Parquet;
- download the returned complex128 DLPack tensor;
- compare all 24 complex amplitudes with the analytical angle-encoding
formula;
- retain the existing shape and cleanup checks.
The regression test was also run against the previous implementation. It
failed as expected: `state[0]` was `1.0`, while the analytical result was
`0.9316157966884513`. The test passes with this change.
## Checklist
- [x] Added or updated unit tests for all changes
- [x] Added or updated documentation for all changes (the copy-unit contract
is documented inline; there is no user-facing API change)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
