Re: [PR] feat: add `row_group_is_[max/min]_value_exact` to StatisticsConverter [arrow-rs]

[via GitHub]

etseidl commented on code in PR #7574:
URL: https://github.com/apache/arrow-rs/pull/7574#discussion_r2122773095


##########
parquet/tests/arrow_reader/mod.rs:
##########
@@ -987,6 +991,33 @@ fn create_data_batch(scenario: Scenario) -> 
Vec<RecordBatch> {
                 make_utf8_batch(vec![Some("e"), Some("f"), Some("g"), 
Some("h"), Some("i")]),
             ]
         }
+        Scenario::TruncatedUTF8 => {
+            // Make utf8 batch with strings longer than 64 bytes
+            // to check truncation of row group statistics
+            vec![
+                make_utf8_batch(vec![
+                    Some(&("a".repeat(64) + "1")),
+                    Some(&("b".repeat(64) + "2")),
+                    Some(&("c".repeat(64) + "3")),

Review Comment:
   I'll have to study the API used here in more detail, but I would assume one 
would usually provide a schema prior to writing batches. The issue here AFAICT 
is that a null value is written, but the schema says values are not nullable, 
so the null is converted to an empty string. I think I'd prefer an error be 
thrown in this instance.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org