tustvold commented on code in PR #3251:
URL: https://github.com/apache/arrow-rs/pull/3251#discussion_r1037283594
##########
arrow/src/row/mod.rs:
##########
@@ -381,13 +430,29 @@ enum Codec {
/// A row converter for the child fields
/// and the encoding of a row containing only nulls
Struct(RowConverter, OwnedRow),
+ /// A row converter for the child field
+ List(RowConverter),
}
impl Codec {
fn new(sort_field: &SortField) -> Result<Self> {
match &sort_field.data_type {
DataType::Dictionary(_, _) =>
Ok(Self::Dictionary(Default::default())),
d if !d.is_nested() => Ok(Self::Stateless),
+ DataType::List(f) | DataType::LargeList(f) => {
+ // The encoded contents will be inverted if descending is set
to true
+ // As such we set `descending` to false and negate nulls first
if it
+ // it set to true
+ let options = SortOptions {
+ descending: false,
+ nulls_first: sort_field.options.nulls_first
+ != sort_field.options.descending,
+ };
+
+ let field = SortField::new_with_options(f.data_type().clone(),
options);
Review Comment:
Only the elements are encoded with descending false, they are then encoded
using variable length encoding which may reorder them. Yes it is mind-bending
:sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
