alamb commented on issue #6899:
URL:
https://github.com/apache/arrow-datafusion/issues/6899#issuecomment-1631122271
> I think we need an optimization step that transforms the plan you gave to
one that uses a fetching sort and does away with the filter. It seems to me the
window operator would still be used as is.
I agree the window operator probably should remain as is
Maybe we could use a specialized sort operator like
```
Filter(rn < 10)
WindowExec(ROW_NUMBER...)
PartitionedSort(order_by={value1, value2}, prefix={value1}, fetch = 10)
```
Where the `PartitionedSort` semantics are to only output the top 10 values
for some prefix of the sort key (in this case, each distinct value of `value1`)
🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
