alamb commented on code in PR #23473:
URL: https://github.com/apache/datafusion/pull/23473#discussion_r3597744534
##########
datafusion/spark/src/function/math/hex.rs:
##########
@@ -166,40 +167,59 @@ where
I: Iterator<Item = Option<T>>,
T: AsRef<[u8]> + 'a,
{
- let mut builder = StringBuilder::with_capacity(len, len * 64);
- let mut buffer = Vec::with_capacity(64);
let lookup = if lowercase {
&HEX_LOOKUP_LOWER
} else {
&HEX_LOOKUP_UPPER
};
+ // Write hex digits directly into one growing value buffer, tracking
offsets
+ // ourselves. Each input byte becomes exactly two output bytes, so there is
+ // no per-row `String`/`StringBuilder` copy — the hex digits are written
once
+ // into the final buffer.
+ let mut values: Vec<u8> = Vec::with_capacity(len * 64);
+ let mut offsets: Vec<i32> = Vec::with_capacity(len + 1);
+ offsets.push(0);
+ let mut nulls = NullBufferBuilder::new(len);
Review Comment:
Another optimization is to special case the "has nulls" path (aka if the
input NullBuffer is None, there is no need to check each value for `Some` /
`None` too -- instead we could have two loops
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
