zanmato1984 commented on code in PR #43763:
URL: https://github.com/apache/arrow/pull/43763#discussion_r1738055428
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -270,14 +329,17 @@ class ARROW_EXPORT RowEncoder {
}
int32_t num_rows() const {
- return offsets_.size() == 0 ? 0 : static_cast<int32_t>(offsets_.size() -
1);
+ return offsets_.empty() ? 0 : static_cast<int32_t>(offsets_.size() - 1);
}
private:
ExecContext* ctx_{nullptr};
std::vector<std::shared_ptr<KeyEncoder>> encoders_;
+ // The offsets of each row in the encoded bytes.
+ // The size would be num_rows + 1 if there are rows.
std::vector<int32_t> offsets_;
std::vector<uint8_t> bytes_;
+ // A constant row with all its columns encoded as null, referred as
kRowIdForNulls.
Review Comment:
```suggestion
// A constant row with all its columns encoded as null. Useful when the
caller is certain that an entire row is null and then uses kRowIdForNulls to
refer to it.
```
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -250,6 +263,51 @@ struct ARROW_EXPORT NullKeyEncoder : KeyEncoder {
}
};
+/// RowEncoder encodes ExecSpan to a variable length byte sequence
+/// created by concatenating the encoded form of each column. The encoding
+/// for each column depends on its data type.
+///
+/// This is used to encode columns into row-major format, which will be
beneficial for grouping and joining operations.
+///
+/// Unlike DuckDB and arrow-rs, currently this row format can not help
+/// sortings because the row-format is uncomparable.
+///
+/// The row format is composed of the the KeyColumn encodings for each,
+/// and the column is encoded as follows:
+/// 1. A null byte for each column, indicating whether the column is null.
+/// "1" for null, "0" for non-null.
+/// 2. The "fixed width" encoding for the column, it would exists whether
Review Comment:
```suggestion
/// 2. The "fixed width" encoding for the column, it would exist whether
```
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -250,6 +263,51 @@ struct ARROW_EXPORT NullKeyEncoder : KeyEncoder {
}
};
+/// RowEncoder encodes ExecSpan to a variable length byte sequence
+/// created by concatenating the encoded form of each column. The encoding
+/// for each column depends on its data type.
+///
+/// This is used to encode columns into row-major format, which will be
beneficial for grouping and joining operations.
+///
+/// Unlike DuckDB and arrow-rs, currently this row format can not help
+/// sortings because the row-format is uncomparable.
+///
+/// The row format is composed of the the KeyColumn encodings for each,
+/// and the column is encoded as follows:
+/// 1. A null byte for each column, indicating whether the column is null.
+/// "1" for null, "0" for non-null.
+/// 2. The "fixed width" encoding for the column, it would exists whether
+/// the column is null or not.
+/// 3. The "variable width" encoding for the column, it would exists only
+/// for non-null string/binary columns.
Review Comment:
This description is misleading. If by `"variable width" encoding` you mean
the `length + var-length-bytes`, then the `length` always occupies
`sizeof(Offset)` bytes, even for null.
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -250,6 +263,51 @@ struct ARROW_EXPORT NullKeyEncoder : KeyEncoder {
}
};
+/// RowEncoder encodes ExecSpan to a variable length byte sequence
+/// created by concatenating the encoded form of each column. The encoding
+/// for each column depends on its data type.
+///
+/// This is used to encode columns into row-major format, which will be
beneficial for grouping and joining operations.
+///
+/// Unlike DuckDB and arrow-rs, currently this row format can not help
+/// sortings because the row-format is uncomparable.
+///
+/// The row format is composed of the the KeyColumn encodings for each,
+/// and the column is encoded as follows:
+/// 1. A null byte for each column, indicating whether the column is null.
+/// "1" for null, "0" for non-null.
+/// 2. The "fixed width" encoding for the column, it would exists whether
+/// the column is null or not.
+/// 3. The "variable width" encoding for the column, it would exists only
+/// for non-null string/binary columns.
+///
Review Comment:
```suggestion
/// 4. Specially, if all columns in a row are null, the caller may decide to
refer to kRowIdForNulls instead of actually encoding/decoding it. See the
comment for encoded_nulls_.
///
```
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -270,14 +329,17 @@ class ARROW_EXPORT RowEncoder {
}
int32_t num_rows() const {
- return offsets_.size() == 0 ? 0 : static_cast<int32_t>(offsets_.size() -
1);
+ return offsets_.empty() ? 0 : static_cast<int32_t>(offsets_.size() - 1);
}
private:
ExecContext* ctx_{nullptr};
std::vector<std::shared_ptr<KeyEncoder>> encoders_;
+ // The offsets of each row in the encoded bytes.
+ // The size would be num_rows + 1 if there are rows.
Review Comment:
```suggestion
// The size would be num_rows + 1 if not empty.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
