Jimexist commented on a change in pull request #579:
URL: https://github.com/apache/arrow-datafusion/pull/579#discussion_r656118650
##########
File path: datafusion/src/physical_plan/windows.rs
##########
@@ -248,21 +252,25 @@ impl AggregateWindowExpr {
let values = self.evaluate_args(batch)?;
let results = partition_points
.iter()
- .map(|partition_range| {
+ .map::<Result<_>, _>(|partition_range| {
let sort_partition_points =
find_ranges_in_range(partition_range,
&sort_partition_points);
let mut window_accumulators = self.create_accumulator()?;
- sort_partition_points
+ let result = sort_partition_points
.iter()
.map(|range| window_accumulators.scan_peers(&values,
range))
- .collect::<Result<Vec<_>>>()
+ .collect::<Result<Vec<_>>>()?
+ .into_iter()
+ .flatten();
+ ScalarValue::iter_to_array(result)
Review comment:
@Dandandan i find that changing to use this, even if would be saving
ArrayRef creating, makes the performance much worse, because it changes from
iterating trusted len iterator to this boxed type with unbounded untrusted
iterator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
