alamb commented on code in PR #6621:
URL: https://github.com/apache/arrow-datafusion/pull/6621#discussion_r1227359136
##########
datafusion/physical-expr/src/window/nth_value.rs:
##########
@@ -169,9 +164,35 @@ impl PartitionEvaluator for NthValueEvaluator {
Ok(())
}
- fn set_state(&mut self, state: &BuiltinWindowState) -> Result<()> {
- if let BuiltinWindowState::NthValue(nth_value_state) = state {
- self.state = nth_value_state.clone()
+ /// When the window frame has a fixed beginning (e.g UNBOUNDED
+ /// PRECEDING), some functions such as FIRST_VALUE, LAST_VALUE and
Review Comment:
Done (BTW if you do "CTRL-G" you can get inline suggestions, to get the
github UI to show the suggestion inline):
```suggestion
/// PRECEDING), for some functions such as FIRST_VALUE, LAST_VALUE and
```
<img width="795" alt="Screenshot 2023-06-12 at 8 11 45 PM"
src="https://github.com/apache/arrow-datafusion/assets/490673/12c5c298-e825-4839-b068-cf732f3c09c6";>
##########
datafusion/physical-expr/src/window/nth_value.rs:
##########
@@ -169,9 +164,35 @@ impl PartitionEvaluator for NthValueEvaluator {
Ok(())
}
- fn set_state(&mut self, state: &BuiltinWindowState) -> Result<()> {
- if let BuiltinWindowState::NthValue(nth_value_state) = state {
- self.state = nth_value_state.clone()
+ /// When the window frame has a fixed beginning (e.g UNBOUNDED
+ /// PRECEDING), some functions such as FIRST_VALUE, LAST_VALUE and
+ /// NTH_VALUE we can memoize result. Once result is calculated it
+ /// will always stay same. Hence, we do not need to keep past data
+ /// as we process the entire dataset. This feature enables us to
+ /// prune rows from table. The default implementation does nothing
+ fn memoize(&mut self, state: &mut WindowAggState) -> Result<()> {
+ let out = &state.out_col;
+ let size = out.len();
+ let (is_prunable, new_prunable) = match self.state.kind {
+ NthValueKind::First => {
+ let n_range =
+ state.window_frame_range.end -
state.window_frame_range.start;
+ (n_range > 0 && size > 0, true)
+ }
+ NthValueKind::Last => (true, false),
+ NthValueKind::Nth(n) => {
+ let n_range =
+ state.window_frame_range.end -
state.window_frame_range.start;
+ (n_range >= (n as usize) && size >= (n as usize), true)
+ }
+ };
+ if is_prunable {
+ if self.state.finalized_result.is_none() && new_prunable {
Review Comment:
I agree the intent is clearer using the term `is_last`. Thank you for that
suggestion . Done in 45427999e2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
