Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-19 Thread Zoltan Ivanfi
Hi, Tim, I added your suggestion to introduce a new ColumnOrder to PARQUET-1222 as the preferred solution. Alex, not writing min/max if there is a NaN is indeed a feasible quick-fix, but I think it would be better to just ignore NaN-s for the p

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-19 Thread Tim Armstrong
We could drop NaNs and require that -0 be normalised to +0 when writing out stats. That would remove any degrees of freedom from the writer and then straightforward comparison with =, <, >, >=, <=, != would work as expected. On Mon, Feb 19, 2018 at 8:04 AM, Zoltan Ivanfi wrote: > Hi, > > Tim, I