Joel, just read the MS-Excel explanation you are referring to. Great explanation. Thanks for pointing to this.
Rob. On 8 jul. 2015, at 20:42, Joel Madero wrote: > Sounds to me like normal floating number behavior. I know someone is trying > to document this and explain why it happens (it's a technical reason that > goes beyond my skills) but this seems entirely normal behavior given the > constraints of programming languages generally and hardware in particular. > > > Best, > Joel > > P.S. You can see similar issues in other spreadsheet software. This seems > to give some indication as to the reasons why: > https://support.microsoft.com/en-us/kb/78113 (yes I know it's MSO > link...just explaining the reason why floating number isn't entirely > accurate all the time) > > On Wed, Jul 8, 2015 at 11:33 AM, Rui Pedro Caldeira <[email protected]> > wrote: > >> Hey and thanks for your fast answer. This is the data set: >> >> 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 >> 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 >> 0.000002 0.000003 0.999997 0.999997 0.999997 0.999997 0.999998 >> 1.000001 1.999997 >> >> Using the formula in the spreadsheet it gives the value: 0.328061419 (VAR) >> and 0.5727664611 (STDEV) >> >> >> And yes, i do mean CHISQ.INV :) >> >> Thanks, Rui. >> >> Cumprimentos, >> Rui Pedro Caldeira >> >> On Wed, Jul 8, 2015 at 7:24 PM, Brian Barker <[email protected]> >> wrote: >> >>> At 18:44 08/07/2015 +0100, Rui Pedro Caldeira wrote: >>> >>>> I'm having some problems with the VAR and STDEV function, when executing >>>> these lines of code. The average is well calculated but the remaining >> two >>>> return very low, unrealistic and equal values (6.93018471335974E-310). >>>> >>> >>> That value is 2^-1027 - which is probably the smallest floating-point >>> value your system can represent without underflow. >>> >>> Have you compared the results produced from your data by equivalent >> normal >>> formulae in a spreadsheet cell? >>> >>> Do you mean CHISQ.INV and not CHISQINV? >>> >>> Brian Barker >>> >>> -- >>> To unsubscribe e-mail to: [email protected] >>> Problems? >>> http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ >>> Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette >>> List archive: http://listarchives.libreoffice.org/global/users/ >>> All messages sent to this list will be publicly archived and cannot be >>> deleted >>> >>> >> >> -- >> To unsubscribe e-mail to: [email protected] >> Problems? >> http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ >> Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette >> List archive: http://listarchives.libreoffice.org/global/users/ >> All messages sent to this list will be publicly archived and cannot be >> deleted >> > > > > -- > *Joel Madero* > LibreOffice QA Volunteer > [email protected] > > -- > To unsubscribe e-mail to: [email protected] > Problems? > http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ > Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette > List archive: http://listarchives.libreoffice.org/global/users/ > All messages sent to this list will be publicly archived and cannot be deleted -- To unsubscribe e-mail to: [email protected] Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted
