tustvold commented on PR #5180: URL: https://github.com/apache/arrow-rs/pull/5180#issuecomment-1845151484
> We are trying to arrive at a partial order that is compatible with equality, and only returns true when the comparison is definite > Do you have any ideas how we can achieve that? I'm not sure such a relation is possible or at least it is not trivially obvious to me. Parquet for example does not produce interval statistics at all - https://github.com/apache/parquet-format/blob/066f9817332da32bdc6dc6dea833b6ee9c269934/LogicalTypes.md?plain=1#L537-L539 A couple of thoughts: * The arrow-ord kernels must be kept consistent with the order used for sorting, which in turn must be a total order, I think this means any partial order would need to be a separate notion * Intervals can only be meaningfully evaluated with respect to a temporal type, it therefore seems surprising to me that we would be pruning on intervals at all, as opposed to using them in the context of range analysis for an arithmetic operation involving them -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
