[jira] [Comment Edited] (ARROW-401) [Java] Floating point vectors should do an approximate comparison in integration tests

[Jacques Nadeau (JIRA)]

    [ 
https://issues.apache.org/jira/browse/ARROW-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717215#comment-15717215
 ] 

Jacques Nadeau edited comment on ARROW-401 at 12/3/16 2:46 AM:
---------------------------------------------------------------

This is what I've used elsewhere before:
{code}
    boolean evaluateEquality(Float f1, Float f2) {
      if(f1.isNaN()){
        return f2.isNaN();
      }

      if(f1.isInfinite()){
        return f2.isInfinite();
      }

      if ((f1 + f2) / 2 != 0) {
        return Math.abs(f1 - f2) / ((f1 + f2) / 2) < 1.0E-6;
      } else {
        return !(f1 != 0);
      }
    }
{code}

{code}
    boolean evaluateEquality(Double f1, Double f2) {
      if(f1.isNaN()){
        return f2.isNaN();
      }

      if(f1.isInfinite()){
        return f2.isInfinite();
      }

      if ((f1 + f2) / 2 != 0) {
        return Math.abs(f1 - f2) / ((f1 + f2) / 2) < 1.0E-12;
      } else {
        return !(f1 != 0);
      }
    }
  }
{code}


was (Author: jnadeau):
This is what I've used elsewhere before:
{code}
    boolean evaluateEquality(Float f1, Float f2) {
      if(f1.isNaN()){
        return f2.isNaN();
      }

      if(f1.isInfinite()){
        return f2.isInfinite();
      }

      if ((f1 + f2) / 2 != 0) {
        return Math.abs(f1 - f2) / ((f1 + f2) / 2) < 1.0E-6;
      } else {
        return !(f1 != 0);
      }
    }
{code}

{code}
    @Override
    boolean evaluateEquality(Double f1, Double f2) {
      if(f1.isNaN()){
        return f2.isNaN();
      }

      if(f1.isInfinite()){
        return f2.isInfinite();
      }

      if ((f1 + f2) / 2 != 0) {
        return Math.abs(f1 - f2) / ((f1 + f2) / 2) < 1.0E-12;
      } else {
        return !(f1 != 0);
      }
    }
  }
{code}

> [Java] Floating point vectors should do an approximate comparison in 
> integration tests
> --------------------------------------------------------------------------------------
>
>                 Key: ARROW-401
>                 URL: https://issues.apache.org/jira/browse/ARROW-401
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Java - Vectors
>            Reporter: Wes McKinney
>            Assignee: Julien Le Dem
>            Priority: Blocker
>
> Floating point precision rears its ugly head:
> {code}
> Incompatible files
> Different values in column:
> Field{name=float64_nullable, type=FloatingPoint{2}, children=[], 
> layout=TypeLayout{[{width=1,type=VALIDITY}, {width=64,type=DATA}]}} at index 
> 1: 912.4140000000002 != 912.414
> 10:23:45.863 [main] ERROR org.apache.arrow.tools.Integration - Incompatible 
> files
> java.lang.IllegalArgumentException: Different values in column:
> Field{name=float64_nullable, type=FloatingPoint{2}, children=[], 
> layout=TypeLayout{[{width=1,type=VALIDITY}, {width=64,type=DATA}]}} at index 
> 1: 912.4140000000002 != 912.414
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)