Glatzel opened a new pull request, #10343:
URL: https://github.com/apache/arrow-rs/pull/10343
# Which issue does this PR close?
/
# Rationale for this change
`arrow-csv`'s CSV reader did not support parsing columns into
`DataType::Float16`.
When a schema explicitly declared a `Float16` field (schema inference never
produces
`Float16` on its own), parsing would fail with an "Unsupported data type"
error since
there was no corresponding match arm in the primitive-array builder
dispatch. Since
`Float16Type` is a standard Arrow primitive type like `Float32`/`Float64`,
the CSV
reader should be able to build arrays for it when given an explicit schema.
# What changes are included in this PR?
- Added a `DataType::Float16` match arm in `parse()`
(`arrow-csv/src/reader/mod.rs`)
that dispatches to `build_primitive_array::<Float16Type>`, matching the
existing
handling for `Float32`/`Float64`.
- Added a `test_float_precision` test that exercises `Float16`, `Float32`,
and
`Float64` columns side by side against the same input values, verifying:
- values that are exactly representable in binary (e.g. `1.5`, `0.25`,
`-2.5`, `0`)
round-trip correctly at all three precisions,
- values with more precision than `f16`/`f32` can hold are correctly
truncated at
the appropriate width while `f64` retains full precision,
- empty CSV fields decode to null across all three columns.
# Are these changes tested?
Yes, added a `test_float_precision` fn to test parsing csv with different
precision
(`Float16`, `Float32`, `Float64`), including precision-loss and null
handling.
- `cargo test -p arrow-csv`
# Are there any user-facing changes?
Yes. Previously, attempting to read a CSV column into a schema field typed as
`DataType::Float16` would fail with an `Parser error: Unsupported data type
Float16`.
This is now supported, consistent with the existing
`Float32`/`Float64`handling.
No existing behavior changes for other types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
