jayzhan211 commented on issue #9403:
URL: https://github.com/apache/datafusion/issues/9403#issuecomment-2171579091
I try to an approach that take multiple columns into consideration but found
that the time spend on `insert_accounted` largely increase.
```rust
self.map.insert_accounted(
new_header,
|header| header.hash,
&mut self.map_size,
);
```
The hash entry includes vector like
```rust
struct Entry<O>
where
O: OffsetSizeTrait,
{
/// hash of the value (stored to avoid recomputing it in hash table
check)
hash: u64,
/// if len =< [`SHORT_VALUE_LEN`]: the data inlined
/// if len > [`SHORT_VALUE_LEN`], the offset of where the data starts
offset_or_inline: Vec<usize>,
/// length of the value, in bytes (use O here so we use only i32 for
/// strings, rather 64 bit usize)
len: Vec<Option<O>>,
group_id: usize,
}
```
It seems that we need avoid adding `Vec` into hash entry. It seems
`ArrowBytesMap`'s idea couldn't help much :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
