tustvold commented on code in PR #3989:
URL: https://github.com/apache/arrow-datafusion/pull/3989#discussion_r1007331972
##########
datafusion/core/src/physical_plan/file_format/row_filter.rs:
##########
@@ -98,14 +108,22 @@ impl ArrowPredicate for DatafusionArrowPredicate {
}
fn evaluate(&mut self, batch: RecordBatch) -> ArrowResult<BooleanArray> {
+ // scoped timer updates on drop
+ let mut timer = self.time.timer();
match self
.physical_expr
.evaluate(&batch)
.map(|v| v.into_array(batch.num_rows()))
{
Ok(array) => {
if let Some(mask) =
array.as_any().downcast_ref::<BooleanArray>() {
- Ok(BooleanArray::from(mask.data().clone()))
+ let bool_arr = BooleanArray::from(mask.data().clone());
+ // TODO is there a more efficient way to count the rows
that are filtered?
Review Comment:
If you don't care about nulls
```
bool_arr.values().count_set_bits_offset(self.offset(), self.len())
```
If you do care about nulls it is slightly more complicated, I'll get
something into arrow-rs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
