This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new fa40708411 perf: optimize ascii in datafusion-functions (#23462)
fa40708411 is described below
commit fa40708411f59bed7a0ce9dbf34306a63b660d9a
Author: Andy Grove <[email protected]>
AuthorDate: Sat Jul 11 00:37:08 2026 -0600
perf: optimize ascii in datafusion-functions (#23462)
## Which issue does this PR close?
N/A
## Rationale for this change
Optimize existing expression.
## What changes are included in this PR?
Rewrote calculate_ascii to skip per-element null checks in the no-null
case, use value_unchecked to avoid bounds checks, and add an ASCII
leading-byte fast path that avoids char-iterator decoding for ASCII
strings.
## Are these changes tested?
Existing tests.
Benchmark (criterion):
Wins:
- ascii_string_ascii_only (null_density=0.5): 23.96% faster (base 8224ns
-> cand 6253ns)
- ascii_string_utf8 (null_density=0.5): 17.558% faster (base 10529ns ->
cand 8680ns)
- ascii_string_ascii_only (null_density=0): 35.842% faster (base 6577ns
-> cand 4219ns)
- ascii_string_utf8 (null_density=0): 23.205% faster (base 11179ns ->
cand 8585ns)
- ascii_string_view_utf8 (null_density=0): 20.23% faster (base 11948ns
-> cand 9531ns)
- ascii_string_view_ascii_only (null_density=0): 24.794% faster (base
7063ns -> cand 5312ns)
- ascii_string_view_ascii_only (null_density=0.5): 13.057% faster (base
9270ns -> cand 8060ns)
- ascii_string_view_utf8 (null_density=0.5): 9.403% faster (base 11671ns
-> cand 10573ns)
Within noise:
- ascii_scalar_utf8view: -1.162% faster (base 44ns -> cand 44ns)
- ascii_scalar_utf8: -2.266% faster (base 43ns -> cand 44ns)
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---
datafusion/functions/src/string/ascii.rs | 57 ++++++++++++++++++++++++--------
1 file changed, 44 insertions(+), 13 deletions(-)
diff --git a/datafusion/functions/src/string/ascii.rs
b/datafusion/functions/src/string/ascii.rs
index bb5a8d0125..4447d1f174 100644
--- a/datafusion/functions/src/string/ascii.rs
+++ b/datafusion/functions/src/string/ascii.rs
@@ -98,7 +98,7 @@ impl ScalarUDFImpl for AsciiFunc {
ScalarValue::Utf8(Some(s))
| ScalarValue::LargeUtf8(Some(s))
| ScalarValue::Utf8View(Some(s)) => {
- let result = s.chars().next().map_or(0, |c| c as i32);
+ let result = first_char_code(&s);
Ok(ColumnarValue::Scalar(ScalarValue::Int32(Some(result))))
}
_ => {
@@ -118,22 +118,53 @@ impl ScalarUDFImpl for AsciiFunc {
}
}
+/// Returns the Unicode scalar value of the first character of `s`, or 0 when
+/// `s` is empty. Reads the leading byte first so the common all-ASCII case
+/// avoids constructing a `char` iterator and decoding a multi-byte sequence.
+#[inline]
+fn first_char_code(s: &str) -> i32 {
+ match s.as_bytes().first() {
+ None => 0,
+ // ASCII byte: the codepoint equals the byte value.
+ Some(&b) if b < 0x80 => b as i32,
+ // Leading byte of a multi-byte sequence: decode the first char.
+ Some(_) => s.chars().next().map_or(0, |c| c as i32),
+ }
+}
+
fn calculate_ascii<'a, V>(array: &V) -> Result<ArrayRef, ArrowError>
where
V: StringArrayType<'a, Item = &'a str>,
{
- let values: Vec<_> = (0..array.len())
- .map(|i| {
- if array.is_null(i) {
- 0
- } else {
- let s = array.value(i);
- s.chars().next().map_or(0, |c| c as i32)
- }
- })
- .collect();
-
- let array = Int32Array::new(values.into(), array.nulls().cloned());
+ let len = array.len();
+ let nulls = array.nulls().cloned();
+
+ // Split the null-handling out of the hot loop: when there is no null
+ // buffer every index is valid, so we can skip the per-element null check
+ // and use unchecked accessors.
+ let values: Vec<i32> = match nulls {
+ Some(ref n) => (0..len)
+ .map(|i| {
+ if n.is_null(i) {
+ 0
+ } else {
+ // SAFETY: `n.is_null(i)` was false, so `i` is a valid,
+ // non-null index.
+ let s = unsafe { array.value_unchecked(i) };
+ first_char_code(s)
+ }
+ })
+ .collect(),
+ None => (0..len)
+ .map(|i| {
+ // SAFETY: no null buffer means every index in `0..len` is
valid.
+ let s = unsafe { array.value_unchecked(i) };
+ first_char_code(s)
+ })
+ .collect(),
+ };
+
+ let array = Int32Array::new(values.into(), nulls);
Ok(Arc::new(array))
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
