2010YOUY01 commented on PR #23738: URL: https://github.com/apache/datafusion/pull/23738#issuecomment-5053554920
> > I think we should first agree on the high-level direction and then start polishing the implementation. We could ignore the implementation details and test coverage for now. > > Thank you @2010YOUY01 joining the review. I agree we should settle the semantics and execution model first. > > > * What are the precise semantics of an ASOF join? I suspect that different systems, such as DuckDB and Snowflake, may differ slightly, so we should first define the semantics rigorously. > > I propose Snowflake-style semantics: each left row selects the closest eligible right row within an optional equality-key group. `>/>=` selects the greatest right key; `</<=` selects the smallest. Left rows are always preserved, unmatched rows are NULL-padded, NULL keys do not match, and ties are nondeterministic. > > > * What is the high-level algorithm used by the ASOF join operator? > > The initial operator uses an ordered merge: co-partition and order both inputs, then advance the right cursor monotonically while retaining the latest eligible candidate. This is O(L + R) after ordering with bounded join state. Other implementations could later share the same logical semantics. > > Does this direction make sense? Thanks for explaining the rationale! I’m interested in helping further with this feature, now I need some time to think it through. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
