dqkqd commented on code in PR #17796:
URL: https://github.com/apache/datafusion/pull/17796#discussion_r2383659583
##########
datafusion/datasource-csv/src/file_format.rs:
##########
@@ -593,15 +593,23 @@ fn build_schema_helper(names: Vec<String>, types:
&[HashSet<DataType>]) -> Schem
.zip(types)
.map(|(field_name, data_type_possibilities)| {
// ripped from
arrow::csv::reader::infer_reader_schema_with_csv_options
- // determine data type based on possible types
- // if there are incompatible types, use DataType::Utf8
- match data_type_possibilities.len() {
- 1 => Field::new(
+ // determine data type based on possible types, ignoring
DataType::Null,
Review Comment:
Thanks for pointing out. I simplify the code as suggested.
These tests failed because there were columns with only `NULL`, after
removing them, the match arm fell to `Utf8` case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
