This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new a8d1af6b8a fix: prevent LEAD/LAG IGNORE NULLS panic without null
bitmap (#23706)
a8d1af6b8a is described below
commit a8d1af6b8a82ded56eb547ad698b224c1f8bf7be
Author: xudong.w <[email protected]>
AuthorDate: Mon Jul 20 12:09:26 2026 +0800
fix: prevent LEAD/LAG IGNORE NULLS panic without null bitmap (#23706)
## Which issue does this PR close?
- Closes #23705.
## Rationale for this change
Arrow arrays containing no null values commonly omit the null bitmap.
The whole-partition evaluation path for `LEAD` and `LAG` with `IGNORE
NULLS` unconditionally unwrapped that optional bitmap, causing a panic
for valid non-null input arrays.
## What changes are included in this PR?
- Fall back to the existing regular shift implementation when the input
has no null bitmap.
- Add a regression test covering both `LEAD` and `LAG` with an array
that explicitly has no null bitmap.
## Are these changes tested?
Yes.
## Are there any user-facing changes?
`LEAD` and `LAG` with `IGNORE NULLS` no longer panic when the input
Arrow array has no null bitmap. There are no public API changes.
---
datafusion/functions-window/src/lead_lag.rs | 29 +++++++++++++++++++++++++++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/datafusion/functions-window/src/lead_lag.rs
b/datafusion/functions-window/src/lead_lag.rs
index de4071c0ce..2363a6beed 100644
--- a/datafusion/functions-window/src/lead_lag.rs
+++ b/datafusion/functions-window/src/lead_lag.rs
@@ -433,8 +433,12 @@ fn evaluate_all_with_ignore_null(
default_value: &ScalarValue,
is_lag: bool,
) -> Result<ArrayRef, DataFusionError> {
- let valid_indices: Vec<usize> =
- array.nulls().unwrap().valid_indices().collect::<Vec<_>>();
+ // Arrays without NULLs do not necessarily have a null bitmap.
+ let Some(nulls) = array.nulls() else {
+ return shift_with_default_value(array, offset, default_value);
+ };
+
+ let valid_indices: Vec<usize> = nulls.valid_indices().collect::<Vec<_>>();
let direction = !is_lag;
let new_array_results: Result<Vec<_>, DataFusionError> = (0..array.len())
.map(|id| {
@@ -838,4 +842,25 @@ mod tests {
.collect::<Int32Array>(),
)
}
+
+ #[test]
+ fn test_ignore_nulls_without_null_bitmap() -> Result<()> {
+ let input = Int32Array::from(vec![1, 2, 3]);
+ assert!(input.nulls().is_none());
+ let input: ArrayRef = Arc::new(input);
+
+ for (offset, expected) in [
+ (1, Int32Array::from(vec![None, Some(1), Some(2)])),
+ (-1, Int32Array::from(vec![Some(2), Some(3), None])),
+ ] {
+ let actual = evaluate_all_with_ignore_null(
+ &input,
+ offset,
+ &ScalarValue::Int32(None),
+ offset > 0,
+ )?;
+ assert_eq!(expected, *as_int32_array(&actual)?);
+ }
+ Ok(())
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
