kszucs commented on code in PR #45360:
URL: https://github.com/apache/arrow/pull/45360#discussion_r2005306344
##########
cpp/src/parquet/column_writer.cc:
##########
@@ -1332,13 +1337,38 @@ class TypedColumnWriterImpl : public ColumnWriterImpl,
public TypedColumnWriter<
bits_buffer_->ZeroPadding();
}
- if (leaf_array.type()->id() == ::arrow::Type::DICTIONARY) {
- return WriteArrowDictionary(def_levels, rep_levels, num_levels,
leaf_array, ctx,
- maybe_parent_nulls);
+ if (this->properties_->cdc_enabled()) {
+ ARROW_ASSIGN_OR_RAISE(auto boundaries,
+ content_defined_chunker_.GetBoundaries(
+ def_levels, rep_levels, num_levels,
leaf_array));
+ for (auto chunk : boundaries) {
+ auto chunk_array = leaf_array.Slice(chunk.value_offset);
Review Comment:
Theoretically the same value index advancing logic is used in the writer
implementations.
I can track the length of each value chunk just in case though. Can you
think of an example which I could test against?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
