alamb commented on issue #5037: URL: https://github.com/apache/arrow-rs/issues/5037#issuecomment-1795909387
> FYI @alamb my memory is hazy as to what forms of aggregate pushdown DF performs, and if we might need to introduce some notion of inexact statistics (if it doesn't already exist). I think the recent work by @berkaysynnada to add https://github.com/apache/arrow-datafusion/blob/e95e3f89c97ae27149c1dd8093f91a5574210fe6/datafusion/common/src/stats.rs#L29-L36 might be relevant However, I think it is likely we will/should eventually add another variant like ``` enum Precision { // The value is known to be within the range (it is at at most this large for Max, or at least this large for Min) // but the actual values may be lower/higher. Bounded(ScalarValue) } ``` I believe we have a similar usecase in IOx for when we want to ensure the bound includes the actual range, but could be larger (cc @NGA-TRAN ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
