On Thu, Apr 3, 2014 at 2:21 PM, Bago <mrb...@gmail.com> wrote: > > > >> >> Sturla >> >> P.S. Personally I am not convinced "unbiased" is ever a valid argument, as >> the biased estimator has smaller error. This is from experience in >> marksmanship: I'd rather shoot a tight series with small systematic error >> than scatter my bullets wildly but "unbiased" on the target. It is the >> total error that counts. The series with smallest total error gets the >> best >> score. It is better to shoot two series and calibrate the sight in between >> than use a calibration-free sight that don't allow us to aim. That's why I >> think classical statistics got this one wrong. Unbiased is never a virtue, >> but the smallest error is. Thus, if we are to repeat an experiment, we >> should calibrate our estimator just like a marksman calibrates his sight. >> But the aim should always be calibrated to give the smallest error, not an >> unbiased scatter. Noone in their right mind would claim a shotgun is more >> precise than a rifle because it has smaller bias. But that is what >> applying >> the Bessel correction implies. >> > > I agree with the point, and what makes it even worse is that ddof=1 does not > even produce an unbiased standard deviation estimate. I produces an unbiased > variance estimate but the sqrt of this variance estimate is a biased > standard deviation estimate, > http://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation.
But ddof=1 still produces a smaller bias than ddof=0 I think the main point in stats is that without ddof, the variance will be too small and t-test or similar will be liberal in small samples, or confidence intervals will be too short. (for statisticians that prefer to have tests that maintain their level and prefer to err on the "conservative" side.) Josef > > Bago > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
