Guess what 1f*5/5==1f (Java; 1 AS float multiplied by 5 divided by 5 is equals to 1 as a float) Everyone not studying computer science won't understand this at first glance (and Google has the best explanations) but with Math.abs((1f/5*5)-1)<0.0001 you do not have a workaround, but the way it has to be done.... (Die not read the thread, just want to explain that you cannot = on floats)
Am 08. Juli 2015 20:42:46 MESZ, schrieb Joel Madero <[email protected]>: >Sounds to me like normal floating number behavior. I know someone is >trying >to document this and explain why it happens (it's a technical reason >that >goes beyond my skills) but this seems entirely normal behavior given >the >constraints of programming languages generally and hardware in >particular. > > >Best, >Joel > >P.S. You can see similar issues in other spreadsheet software. This >seems >to give some indication as to the reasons why: >https://support.microsoft.com/en-us/kb/78113 (yes I know it's MSO >link...just explaining the reason why floating number isn't entirely >accurate all the time) > >On Wed, Jul 8, 2015 at 11:33 AM, Rui Pedro Caldeira ><[email protected]> >wrote: > >> Hey and thanks for your fast answer. This is the data set: >> >> 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 >0.000001 >> 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 >> 0.000002 0.000003 0.999997 0.999997 0.999997 0.999997 0.999998 >> 1.000001 1.999997 >> >> Using the formula in the spreadsheet it gives the value: 0.328061419 >(VAR) >> and 0.5727664611 (STDEV) >> >> >> And yes, i do mean CHISQ.INV :) >> >> Thanks, Rui. >> >> Cumprimentos, >> Rui Pedro Caldeira >> >> On Wed, Jul 8, 2015 at 7:24 PM, Brian Barker ><[email protected]> >> wrote: >> >> > At 18:44 08/07/2015 +0100, Rui Pedro Caldeira wrote: >> > >> >> I'm having some problems with the VAR and STDEV function, when >executing >> >> these lines of code. The average is well calculated but the >remaining >> two >> >> return very low, unrealistic and equal values >(6.93018471335974E-310). >> >> >> > >> > That value is 2^-1027 - which is probably the smallest >floating-point >> > value your system can represent without underflow. >> > >> > Have you compared the results produced from your data by equivalent >> normal >> > formulae in a spreadsheet cell? >> > >> > Do you mean CHISQ.INV and not CHISQINV? >> > >> > Brian Barker >> > >> > -- >> > To unsubscribe e-mail to: [email protected] >> > Problems? >> > >http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ >> > Posting guidelines + more: >http://wiki.documentfoundation.org/Netiquette >> > List archive: http://listarchives.libreoffice.org/global/users/ >> > All messages sent to this list will be publicly archived and cannot >be >> > deleted >> > >> > >> >> -- >> To unsubscribe e-mail to: [email protected] >> Problems? >> http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ >> Posting guidelines + more: >http://wiki.documentfoundation.org/Netiquette >> List archive: http://listarchives.libreoffice.org/global/users/ >> All messages sent to this list will be publicly archived and cannot >be >> deleted >> > > > >-- >*Joel Madero* >LibreOffice QA Volunteer >[email protected] > >-- >To unsubscribe e-mail to: [email protected] >Problems? >http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ >Posting guidelines + more: >http://wiki.documentfoundation.org/Netiquette >List archive: http://listarchives.libreoffice.org/global/users/ >All messages sent to this list will be publicly archived and cannot be >deleted -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. -- To unsubscribe e-mail to: [email protected] Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted
