dwsmith1983 commented on code in PR #6116:
URL: https://github.com/apache/datafusion-comet/pull/6116#discussion_r4100392301
##########
native/core/src/parquet/schema_adapter.rs:
##########
@@ -2959,6 +2968,595 @@ mod test {
Ok(())
}
+ /// The message every rejected read carries, from
`SparkError::ParquetMissingFieldIds`.
+ const MISSING_IDS: &str = "Parquet file schema doesn't contain any field
Ids";
Review Comment:
The scan tests are gone, along with `PlannerScan`, `scan_file_via_planner`
and `write_parquet`. What stays in Rust is
`contains_field_ids_sees_ids_on_any_node` with a `key_value` case, the
`errors.rs` unwrap test, and one reader test next to the check: a reader built
with `require_field_ids` refuses a file without ids on its first `get_metadata`
and names the file, reads a file that carries an id, and a factory without the
flag reads the file without ids. The repeated group cases moved to Scala as one
test over a file written with
`createParquetWriter(MessageTypeParser.parseMessageType(...))`, with ids only
on the `list` and `key_value` groups, read with the flag off and on and
compared against Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
