0lai0 opened a new pull request, #1407: URL: https://github.com/apache/mahout/pull/1407
### Related Issues Closes #1341 ### Changes - [ ] Bug fix - [x] New feature - [ ] Refactoring - [ ] Documentation - [ ] Test - [ ] CI/CD pipeline - [ ] Other ### Why <!-- Why is this change needed? --> `PipelineConfig.dtype` is silently ignored when loading from files. `read_file_by_extension` always returned `Vec<f64>`, so `BatchData::F64` was always produced and `encode_batch_f32_for_pipeline` was never reached from a file source. `ParquetReader<T> / ParquetStreamingReader<T> `already had f32/f64 support from #1393 (issue #1340) — this PR just wires the dtype through. ### How <!-- What was done? --> - InMemoryProducer and StreamingProducer are now generic over `T: FloatElem (default f64)`. A pub(crate) ToBatchData trait handles wrapping/unwrapping BatchData inside produce(). - read_file_by_extension takes dtype: Precision and returns BatchData. Parquet dispatches to ParquetReader::<f32|f64> directly. Other formats (Arrow IPC, NumPy, PyTorch, TensorFlow) read as f64 and cast with a narrowing warning — native f32 readers for those formats are a follow-up. - new_from_file_streaming's reader setup was extracted into build_streaming_producer::<T> to avoid duplicating ~40 lines. ## Checklist - [x] Added or updated unit tests for all changes - [x] Added or updated documentation for all changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
