Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-04 Thread josef . pktd
On Fri, Apr 4, 2014 at 8:50 AM, Daπid wrote: > > On 2 April 2014 16:06, Sturla Molden wrote: >> >> wrote: >> >> > pandas came later and thought ddof=1 is worth more than consistency. >> >> Pandas is a data analysis package. NumPy is a numerical array package. >> >> I think ddof=1 is justified fo

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-04 Thread Daπid
On 2 April 2014 16:06, Sturla Molden wrote: > wrote: > > > pandas came later and thought ddof=1 is worth more than consistency. > > Pandas is a data analysis package. NumPy is a numerical array package. > > I think ddof=1 is justified for Pandas, for consistency with statistical > software (SPSS

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-03 Thread josef . pktd
On Thu, Apr 3, 2014 at 2:21 PM, Bago wrote: > > > >> >> Sturla >> >> P.S. Personally I am not convinced "unbiased" is ever a valid argument, as >> the biased estimator has smaller error. This is from experience in >> marksmanship: I'd rather shoot a tight series with small systematic error >> than

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-03 Thread Bago
> Sturla > > P.S. Personally I am not convinced "unbiased" is ever a valid argument, as > the biased estimator has smaller error. This is from experience in > marksmanship: I'd rather shoot a tight series with small systematic error > than scatter my bullets wildly but "unbiased" on the target. It

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-03 Thread josef . pktd
On Wed, Apr 2, 2014 at 10:06 AM, Sturla Molden wrote: > wrote: > >> pandas came later and thought ddof=1 is worth more than consistency. > > Pandas is a data analysis package. NumPy is a numerical array package. > > I think ddof=1 is justified for Pandas, for consistency with statistical > softwa

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-03 Thread Sturla Molden
wrote: > pandas came later and thought ddof=1 is worth more than consistency. Pandas is a data analysis package. NumPy is a numerical array package. I think ddof=1 is justified for Pandas, for consistency with statistical software (SPSS et al.) For NumPy, there are many computational tasks whe

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-03 Thread Sturla Molden
alex wrote: > I don't have any opinion about this debate, but I love the > justification in that thread "Any surprise that is created by the > different default should be mitigated by the fact that it's an > opportunity to learn something about what you are doing." That is so true. Sturla _

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread josef . pktd
On Tue, Apr 1, 2014 at 5:11 PM, Nathaniel Smith wrote: > On Tue, Apr 1, 2014 at 9:51 PM, Ralf Gommers wrote: >> >> >> >> On Tue, Apr 1, 2014 at 10:08 PM, Nathaniel Smith wrote: >>> >>> On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden >>> wrote: >>> > Haslwanter Thomas wrote: >>> > >>> >> Personal

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread alex
On Tue, Apr 1, 2014 at 4:54 PM, Charles R Harris wrote: > > > > On Tue, Apr 1, 2014 at 2:08 PM, Nathaniel Smith wrote: >> >> On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden >> wrote: >> > Haslwanter Thomas wrote: >> > >> >> Personally I cannot think of many applications where it would be >> >> de

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 9:51 PM, Ralf Gommers wrote: > > > > On Tue, Apr 1, 2014 at 10:08 PM, Nathaniel Smith wrote: >> >> On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden >> wrote: >> > Haslwanter Thomas wrote: >> > >> >> Personally I cannot think of many applications where it would be >> >> desir

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Charles R Harris
On Tue, Apr 1, 2014 at 2:08 PM, Nathaniel Smith wrote: > On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden > wrote: > > Haslwanter Thomas wrote: > > > >> Personally I cannot think of many applications where it would be desired > >> to calculate the standard deviation with ddof=0. In addition, I fee

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Ralf Gommers
On Tue, Apr 1, 2014 at 10:08 PM, Nathaniel Smith wrote: > On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden > wrote: > > Haslwanter Thomas wrote: > > > >> Personally I cannot think of many applications where it would be desired > >> to calculate the standard deviation with ddof=0. In addition, I fe

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden wrote: > Haslwanter Thomas wrote: > >> Personally I cannot think of many applications where it would be desired >> to calculate the standard deviation with ddof=0. In addition, I feel that >> there should be consistency between standard modules such a

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Eelco Hoogendoorn
I agree; breaking code over this would be ridiculous. Also, I prefer the zero default, despite the mean/std combo probably being more common. On Tue, Apr 1, 2014 at 10:02 PM, Sturla Molden wrote: > Haslwanter Thomas wrote: > > > Personally I cannot think of many applications where it would be d

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Sturla Molden
Haslwanter Thomas wrote: > Personally I cannot think of many applications where it would be desired > to calculate the standard deviation with ddof=0. In addition, I feel that > there should be consistency between standard modules such as numpy, scipy, > and pandas. ddof=0 is the maxiumum likel

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Benjamin Root
Because np.mean() is ddof=0? (I mean effectively, not that it actually has a parameter for that) There is consistency within the library, and I certainly wouldn't want to have NaN all of the sudden coming from my calls to mean() that I apply to an arbitrary non-empty array of values that happened t

[Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Haslwanter Thomas
While most other Python applications (scipy, pandas) use for the calculation of the standard deviation the default "ddof=1" (i.e. they calculate the sample standard deviation), the Numpy implementation uses the default "ddof=0". Personally I cannot think of many applications where it would be des