berkaysynnada commented on issue #5037: URL: https://github.com/apache/arrow-rs/issues/5037#issuecomment-1796384939
> > FYI @alamb my memory is hazy as to what forms of aggregate pushdown DF performs, and if we might need to introduce some notion of inexact statistics (if it doesn't already exist). > > I think the recent work by @berkaysynnada to add https://github.com/apache/arrow-datafusion/blob/e95e3f89c97ae27149c1dd8093f91a5574210fe6/datafusion/common/src/stats.rs#L29-L36 might be relevant > > However, I think it is likely we will/should eventually add another variant like > > ``` > enum Precision { > // The value is known to be within the range (it is at at most this large for Max, or at least this large for Min) > // but the actual values may be lower/higher. > Bounded(ScalarValue) > } > ``` > > I believe we have a similar usecase in IOx for when we want to ensure the bound includes the actual range, but could be larger (cc @NGA-TRAN ) I think so too, adding a range-specifying variant will pave the way for many things. While I have other high-priority tasks to address shortly, I'm always available to offer support if someone wishes to take this on. The variant I have in mind is as follows: ``` enum Precision { ... InBetween(Interval) } ``` It will also be easier to use after updating intervals (planning to open the related PR in a few days). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
