jayzhan211 commented on code in PR #26386:
URL: https://github.com/apache/datafusion/pull/26386#discussion_r4306428009
##########
datafusion/core/tests/fuzz_cases/window_fuzz.rs:
##########
@@ -759,40 +780,65 @@ fn generate_random_string(rng: &mut StdRng, length:
usize) -> String {
/// Return randomly sized record batches with:
/// three sorted int32 columns 'a', 'b', 'c' ranged from 0..DISTINCT as columns
/// one random int32 column x
+///
+/// `sort_options` sets the sort order of 'a', 'b' and 'c', in this sequence.
+/// Each value in 'a', 'b', 'c' and 'x' is NULL with the probability
`null_pct`.
pub(crate) fn make_staggered_batches<const STREAM: bool>(
len: usize,
n_distinct: usize,
+ null_pct: f64,
+ sort_options: [SortOptions; 3],
random_seed: u64,
) -> Vec<RecordBatch> {
// use a random number generator to pick a random sized output
let mut rng = StdRng::seed_from_u64(random_seed);
- let mut input123: Vec<(i32, i32, i32)> = vec![(0, 0, 0); len];
- let mut input4: Vec<i32> = vec![0; len];
+ let mut input123: Vec<(Option<i32>, Option<i32>, Option<i32>)> =
Review Comment:
nit: now that the keys are ordered with `lexsort_to_indices`, the `input123`
tuple vector isn't needed any more. The three key arrays can be built directly:
```diff
- let mut input123: Vec<(Option<i32>, Option<i32>, Option<i32>)> =
- vec![(None, None, None); len];
@@
- for v in &mut input123 {
- *v = (
- random_key(&mut rng),
- random_key(&mut rng),
- random_key(&mut rng),
- )
- }
+ let keys: [ArrayRef; 3] = std::array::from_fn(|_| {
+ Arc::new(Int32Array::from_iter((0..len).map(|_| random_key(&mut
rng)))) as _
+ });
@@
- let keys: [ArrayRef; 3] = [
- Arc::new(Int32Array::from_iter(input123.iter().map(|k| k.0))),
- Arc::new(Int32Array::from_iter(input123.iter().map(|k| k.1))),
- Arc::new(Int32Array::from_iter(input123.iter().map(|k| k.2))),
- ];
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
