jackylee-ch opened a new pull request, #980: URL: https://github.com/apache/paimon-rust/pull/980
CSV boolean columns were cast with `arrow_cast`, whose string->boolean mapping reads `1`, `yes`, `t`, `on` and `y` as true and unrecognized tokens such as `2` or `invalid` as null. Java `CsvParser` reads BOOLEAN via `Boolean.parseBoolean`: only `true` (case-insensitive) is true, every other non-null token is false, and it never yields null. Reading the same CSV format-table file through Rust therefore flipped boolean values — `1`/`yes`/`t` became true instead of false — and turned tokens like `invalid` into null, so `WHERE flag = true` returned different rows than Java, Spark or Flink over the same file. Parse CSV boolean fields with the same rule so a format table reads identically across engines; null fields stay null. Verified by a unit test that fails under the old `arrow_cast` behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
