dqkqd commented on code in PR #17796:
URL: https://github.com/apache/datafusion/pull/17796#discussion_r2383659583
##########
datafusion/datasource-csv/src/file_format.rs:
##########
@@ -593,15 +593,23 @@ fn build_schema_helper(names: Vec<String>, types:
&[HashSet<DataType>]) -> Schem
.zip(types)
.map(|(field_name, data_type_possibilities)| {
// ripped from
arrow::csv::reader::infer_reader_schema_with_csv_options
- // determine data type based on possible types
- // if there are incompatible types, use DataType::Utf8
- match data_type_possibilities.len() {
- 1 => Field::new(
+ // determine data type based on possible types, ignoring
DataType::Null,
Review Comment:
Thanks for pointing out. I simplified the code as suggested.
These tests failed because there were columns with nulls only, after
removing them, the match arm fell to `Utf8` case. But they asserted that nulls
columns should have data type `DataType::Null`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
