Rich-T-kid commented on code in PR #12223:
URL: https://github.com/apache/arrow-rs/pull/12223#discussion_r4561143374
##########
arrow-select/src/coalesce/byte.rs:
##########
@@ -4,14 +4,21 @@
use super::InProgressArray;
use crate::filter::{FilterIndices, FilterPredicate, FilterSelection};
+use crate::take::{TakeOptions, take};
use arrow_array::builder::GenericByteBuilder;
use arrow_array::cast::AsArray;
use arrow_array::types::ByteArrayType;
-use arrow_array::{Array, ArrayRef, GenericByteArray, OffsetSizeTrait};
+use arrow_array::{Array, ArrayRef, GenericByteArray, OffsetSizeTrait,
UInt32Array};
use arrow_buffer::ArrowNativeType;
use arrow_schema::ArrowError;
use std::sync::Arc;
+const INDEXED_COPY_MAX_SELECTIVITY_DENOMINATOR: usize = 1024;
+
+fn should_use_indexed_copy(input_rows: usize, selected_rows: usize) -> bool {
+ selected_rows <= input_rows / INDEXED_COPY_MAX_SELECTIVITY_DENOMINATOR
+}
+
Review Comment:
Fixed in 6adfdee970. The sparse byte-array dispatch now happens before the
original dense helper, so the 0.1/0.8 path is unchanged. The hosted rerun is
flat at 0.1/0.8 (within noise, with several cases modestly faster) while
retaining the 2.0–3.6x wins at 0.001:
https://github.com/apache/arrow-rs/pull/12223#issuecomment-6847806303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
