zanmato1984 commented on code in PR #43763:
URL: https://github.com/apache/arrow/pull/43763#discussion_r1739750515
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -336,11 +354,20 @@ class ARROW_EXPORT RowEncoder {
private:
ExecContext* ctx_{nullptr};
std::vector<std::shared_ptr<KeyEncoder>> encoders_;
- // The offsets of each row in the encoded bytes.
- // The size would be num_rows + 1 if not empty.
+ // offsets_ vector stores the starting position (offset) of each encoded row
+ // within the bytes_ vector. This allows for quick access to individual rows.
+ //
+ // The size would be num_rows + 1 if not empty, the last element is the total
+ // length of the bytes_ vector.
std::vector<int32_t> offsets_;
+ // The encoded bytes of all "non kRowIdForNulls" rows.
std::vector<uint8_t> bytes_;
- // A constant row with all its columns encoded as null. Useful when the
caller is certain that an entire row is null and then uses kRowIdForNulls to
refer to it.
+ // A pre-computed constant row with all its columns encoded as null. Useful
when
Review Comment:
```suggestion
// A pre-encoded constant row with all its columns being null. Useful when
```
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -336,11 +354,20 @@ class ARROW_EXPORT RowEncoder {
private:
ExecContext* ctx_{nullptr};
std::vector<std::shared_ptr<KeyEncoder>> encoders_;
- // The offsets of each row in the encoded bytes.
- // The size would be num_rows + 1 if not empty.
+ // offsets_ vector stores the starting position (offset) of each encoded row
+ // within the bytes_ vector. This allows for quick access to individual rows.
+ //
+ // The size would be num_rows + 1 if not empty, the last element is the total
+ // length of the bytes_ vector.
std::vector<int32_t> offsets_;
+ // The encoded bytes of all "non kRowIdForNulls" rows.
Review Comment:
```suggestion
// The encoded bytes of all non "kRowIdForNulls" rows.
```
##########
cpp/src/arrow/compute/row/row_encoder_internal.h:
##########
@@ -267,37 +272,44 @@ struct ARROW_EXPORT NullKeyEncoder : KeyEncoder {
/// created by concatenating the encoded form of each column. The encoding
/// for each column depends on its data type.
///
-/// This is used to encode columns into row-major format, which will be
beneficial for grouping and joining operations.
+/// This is used to encode columns into row-major format, which will be
+/// beneficial for grouping and joining operations.
///
/// Unlike DuckDB and arrow-rs, currently this row format can not help
/// sortings because the row-format is uncomparable.
///
+/// # Key Column Encoding
+///
/// The row format is composed of the the KeyColumn encodings for each,
/// and the column is encoded as follows:
/// 1. A null byte for each column, indicating whether the column is null.
/// "1" for null, "0" for non-null.
/// 2. The "fixed width" encoding for the column, it would exist whether
/// the column is null or not.
-/// 3. The "variable width" encoding for the column, it would exists only
+/// 3. The "variable payload" encoding for the column, it would exists only
Review Comment:
Does it count into the "length" part for the var-length column (which will
exist no matter the column is null or not)?
##########
cpp/src/arrow/compute/row/row_encoder_internal.cc:
##########
@@ -159,17 +159,19 @@ Status FixedWidthKeyEncoder::Encode(const ExecValue&
data, int64_t batch_length,
};
if (data.is_array()) {
ArraySpan viewed = data.array;
+ // The original type might not FixedSizeBinaryType, but it would
Review Comment:
```suggestion
// The original type might not be FixedSizeBinaryType, but it would
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
