chrisavl commented on a change in pull request #8589:
URL: https://github.com/apache/arrow/pull/8589#discussion_r517942949
##########
File path: cpp/src/parquet/column_writer.cc
##########
@@ -1230,8 +1230,9 @@ class TypedColumnWriterImpl : public ColumnWriterImpl,
public TypedColumnWriter<
buffers[0] = bits_buffer_;
// Should be a leaf array.
DCHECK_EQ(array->num_fields(), 0);
- return arrow::MakeArray(std::make_shared<ArrayData>(
- array->type(), array->length(), std::move(buffers), new_null_count));
+ return arrow::MakeArray(std::make_shared<ArrayData>(array->type(),
array->length(),
+ std::move(buffers),
Review comment:
Ah, you are right the offset also applies to the null buffer. But I am
not sure how to best slice the other buffers, since ArrayData::Slice won't help
here and using SliceBuffer directly would require figuring out the physical
offset from the logical offset in ArrayData. Any suggestions or pointers here
would be appreciated!
Fow now, I pushed a not so elegant workaround that allocates a bitmap for
the whole column instead of for only a batch_size. That way the offset for the
bitmap and the other buffers should be the same, I think.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
