JingsongLi commented on code in PR #1009:
URL: https://github.com/apache/paimon-rust/pull/1009#discussion_r4177621158
##########
crates/paimon/src/variant.rs:
##########
@@ -2797,6 +2808,25 @@ fn cast_variant_to_f64(variant: VariantRef<'_>) ->
Option<f64> {
}
}
+pub(crate) fn cast_variant_to_f32(variant: VariantRef<'_>) -> Option<f32> {
+ match variant.kind().ok()? {
+ VariantKind::Boolean => Some(if variant.get_boolean().ok()? {
+ 1.0
+ } else {
+ 0.0
+ }),
+ VariantKind::Long => Some(variant.get_long().ok()? as f32),
+ VariantKind::Float => variant.get_float().ok(),
+ VariantKind::Double => Some(variant.get_double().ok()? as f32),
+ VariantKind::Decimal => {
+ let decimal = variant.get_decimal().ok()?;
+ Some((decimal.unscaled as f64 / 10f64.powi(decimal.scale as i32))
as f32)
+ }
+ VariantKind::String =>
variant.get_string().ok()?.trim().parse::<f32>().ok(),
Review Comment:
[P2] Use Java whitespace rules for Variant string-to-FLOAT casts. Rust
str::trim removes Unicode whitespace such as NBSP (U+00A0), whereas Java
VariantGet delegates through
StringToNumericPrimitiveCastRule/BinaryStringUtils.toFloat to Float.parseFloat,
which rejects a NBSP-wrapped number. With a real table containing
{"x":"\u00a01.5\u00a0"}, the new native FLOAT projection returns 1.5 for
failOnError=false instead of null, and also returns 1.5 for failOnError=true
instead of failing. Both Python/Parquet probes fail; a Java runtime probe
rejects the same string, and the previous Rust f64 cast also rejected it. This
silently changes data/error semantics across Java and Rust. Trim only the
characters accepted by Java and cover both error policies while retaining
ordinary ASCII whitespace support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
