lidavidm commented on a change in pull request #10270:
URL: https://github.com/apache/arrow/pull/10270#discussion_r629409104
##########
File path: cpp/src/arrow/dataset/file_csv.cc
##########
@@ -104,6 +115,13 @@ static inline Result<csv::ConvertOptions>
GetConvertOptions(
// Properly set conversion types
convert_options.column_types[field->name()] = field->type();
}
+ if (convert_options.include_columns.empty()) {
+ // We know which columns we want - so if none were specified, override the
default
+ // behavior of reading all columns by reading a missing fake null column
+ convert_options.include_missing_columns = true;
+ convert_options.include_columns.push_back("fabricated");
Review comment:
Ok, now I've added an explicit CSV reader option - it still uses a
fabricated null column internally for convenience but this way that won't leak
out into consumers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
