ozankabak commented on PR #5180: URL: https://github.com/apache/arrow-rs/pull/5180#issuecomment-1845417976
> We cannot change the ordering relation of these kernels to a partial order, as it must match that of sorts, which in turn must be a total order AFAICT the way the proposed ordering relation works will not break any sorts. Since indeterminate cases return false all ways, it will act like a pseudo total order. This is what I'm trying to nail down by discussing with you -- if it breaks sorts, I agree it is a no-go. > We cannot add new array types that aren't part of the arrow standard > Perhaps this logic could be implemented within the range analysis framework of DataFusion, as opposed to here? At this point, we are trying to get the code to compile so we can iterate to find the right semantics. The purpose here is not rush a merge or make a de-facto change to arrow spec. The code may very well move to somewhere else if this is not the right place at this time. My personal take is that returning clearly wrong comparison results for interval datatypes should be fixed at this level. The library claims to support these datatypes after all, performs such comparisons without any error or warning, so returning arbitrary (relative to real-world interval semantics) comparison results silently is simply wrong IMHO. However, *it doesn't really matter at this point* -- we can always revisit such decisions over time. Let's try to find a good solution together first, and we can place the code somewhere else to get things moving. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
