lidavidm commented on code in PR #12829:
URL: https://github.com/apache/arrow/pull/12829#discussion_r861088396
##########
cpp/src/parquet/arrow/reader_internal.cc:
##########
@@ -409,22 +441,36 @@ Status TransferDate64(RecordReader* reader, MemoryPool*
pool,
Status TransferDictionary(RecordReader* reader,
const std::shared_ptr<DataType>& logical_value_type,
- std::shared_ptr<ChunkedArray>* out) {
+ bool nullable, std::shared_ptr<ChunkedArray>* out) {
auto dict_reader = dynamic_cast<DictionaryRecordReader*>(reader);
DCHECK(dict_reader);
*out = dict_reader->GetResult();
if (!logical_value_type->Equals(*(*out)->type())) {
ARROW_ASSIGN_OR_RAISE(*out, (*out)->View(logical_value_type));
}
+ if (!nullable) {
+ // Reconstruct each chunk without nulls.
Review Comment:
What would happen with nested fields is that the parent array's null buffer
would get folded into the children and then on reading it back, there would be
nulls: https://github.com/apache/arrow/pull/12706#issuecomment-1086357554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
