Jefffrey commented on code in PR #23473:
URL: https://github.com/apache/datafusion/pull/23473#discussion_r3577957866
##########
datafusion/spark/src/function/math/hex.rs:
##########
@@ -166,40 +167,59 @@ where
I: Iterator<Item = Option<T>>,
T: AsRef<[u8]> + 'a,
{
- let mut builder = StringBuilder::with_capacity(len, len * 64);
- let mut buffer = Vec::with_capacity(64);
let lookup = if lowercase {
&HEX_LOOKUP_LOWER
} else {
&HEX_LOOKUP_UPPER
};
+ // Write hex digits directly into one growing value buffer, tracking
offsets
+ // ourselves. Each input byte becomes exactly two output bytes, so there is
+ // no per-row `String`/`StringBuilder` copy — the hex digits are written
once
+ // into the final buffer.
+ let mut values: Vec<u8> = Vec::with_capacity(len * 64);
+ let mut offsets: Vec<i32> = Vec::with_capacity(len + 1);
+ offsets.push(0);
+ let mut nulls = NullBufferBuilder::new(len);
Review Comment:
we can probably avoid a null builder by just cloning the input nulls? though
i guess need to plumb the functions to pass those through
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
