nugend opened a new issue #318:
URL: https://github.com/apache/arrow-datafusion/issues/318


   **Is your feature request related to a problem or challenge? Please describe 
what you are trying to do.**
   In many timeseries workloads, the need to join one record to another based 
on recency or ordering is quite prevalent. This can be with two sensors that 
maintain different sampling rates, in handling market data (such as situations 
where you want to find the most recent quote for a given trade), or any other 
scenario where an ordering of events can be applied. In these situations, it is 
quite frequent that one set of data is at a *much* larger scale than the other, 
typically starting at an order of magnitude. It is also quite common that you 
would want to actually store your data *in* the appropriate order so as to 
minimize the effort when performing this kind of join operation (though that is 
not strictly the problem itself).
   
   **Describe the solution you'd like**
   Essentially the solution implemented by [Clickhouse 
](https://clickhouse.tech/docs/en/sql-reference/statements/select/join/#asof-join-usage)
 and [KDB](https://code.kx.com/q/ref/aj/) (please note that the Clickhouse 
solution allows for the full breadth of ordering conditions for closest match).
   
   **Describe alternatives you've considered**
   There is a general [subquery 
solution](https://dba.stackexchange.com/questions/116424/left-join-on-closest-date)
 that can be used to achieve the desired outcome, but it is typically not 
performant and can be fairly awkward to express. Provided implementation of 
#141 and sufficient query optimization of the subquery solution into sort-merge 
join, it may not be necessary as direct syntax.
   
   **Additional context**
   This likely depends on #141


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to