1. Bootstrap will do poorly for small sample sizes (less than 25 or
so). Parametric methods (t) have the advantage of working down to a
sample size of less than 5.
2. You need to have the number of resamples reasonably large, say
10,000, for poorly behaved distributions. Otherwise extreme
percentiles of the resampling distribution used in calculations are
too inaccurate.
3. You examples are pathological. For the normal case, the t C.I.
should be optimal, so no surprise there. For the Student t(df=3)
case, you have a near singular case with only a couple of moments
that exist. Try something skewed (lognormal) or bimodal (mixture) as
a better example.
4. BCA general gives the best results, but only when the sample size
is moderately large ( > 25) and the number of resamples is large
(several thousand).
At 07:09 AM 8/16/2010, Mark Seeto wrote:
Hello, I have a question regarding bootstrap confidence intervals.
Suppose we have a data set consisting of single measurements, and that
the measurements are independent but the distribution is unknown. If
we want a confidence interval for the population mean, when should a
bootstrap confidence interval be preferred over the elementary t
interval?
I was hoping the answer would be "always", but some simple simulations
suggest that this is incorrect. I simulated some data and calculated
95% elementary t intervals and 95% bootstrap BCA intervals (with the
boot package). I calculated the proportion of confidence intervals
lying entirely above the true mean, the proportion entirely below the
true mean, and the proportion containing the true mean. I used a
normal distribution and a t distribution with 3 df.
library(boot)
samplemean <- function(x, ind) mean(x[ind])
ci.norm <- function(sample.size, n.samples, mu=0, sigma=1, boot.reps) {
t.under <- 0; t.over <- 0
bca.under <- 0; bca.over <- 0
for (k in 1:n.samples) {
x <- rnorm(sample.size, mu, sigma)
b <- boot(x, samplemean, R = boot.reps)
bci <- boot.ci(b, type="bca")
if (mu < mean(x) - qt(0.975, sample.size - 1)*sd(x)/sqrt(sample.size))
t.under <- t.under + 1
if (mu > mean(x) + qt(0.975, sample.size - 1)*sd(x)/sqrt(sample.size))
t.over <- t.over + 1
if (mu < bci$bca[4]) bca.under <- bca.under + 1
if (mu > bci$bca[5]) bca.over <- bca.over + 1
}
return(list(t = c(t.under, t.over, n.samples - (t.under +
t.over))/n.samples,
bca = c(bca.under, bca.over, n.samples - (bca.under +
bca.over))/n.samples))
}
ci.t <- function(sample.size, n.samples, df, boot.reps) {
t.under <- 0; t.over <- 0
bca.under <- 0; bca.over <- 0
for (k in 1:n.samples) {
x <- rt(sample.size, df)
b <- boot(x, samplemean, R = boot.reps)
bci <- boot.ci(b, type="bca")
if (0 < mean(x) - qt(0.975, sample.size - 1)*sd(x)/sqrt(sample.size))
t.under <- t.under + 1
if (0 > mean(x) + qt(0.975, sample.size - 1)*sd(x)/sqrt(sample.size))
t.over <- t.over + 1
if (0 < bci$bca[4]) bca.under <- bca.under + 1
if (0 > bci$bca[5]) bca.over <- bca.over + 1
}
return(list(t = c(t.under, t.over, n.samples - (t.under +
t.over))/n.samples,
bca = c(bca.under, bca.over, n.samples - (bca.under +
bca.over))/n.samples))
}
set.seed(1)
ci.norm(sample.size = 10, n.samples = 1000, boot.reps = 1000)
$t
[1] 0.019 0.026 0.955
$bca
[1] 0.049 0.059 0.892
ci.norm(sample.size = 20, n.samples = 1000, boot.reps = 1000)
$t
[1] 0.030 0.024 0.946
$bca
[1] 0.035 0.037 0.928
ci.t(sample.size = 10, n.samples = 1000, df = 3, boot.reps = 1000)
$t
[1] 0.018 0.022 0.960
$bca
[1] 0.055 0.076 0.869
Warning message:
In norm.inter(t, adj.alpha) : extreme order statistics used as endpoints
ci.t(sample.size = 20, n.samples = 1000, df = 3, boot.reps = 1000)
$t
[1] 0.027 0.014 0.959
$bca
[1] 0.054 0.047 0.899
I don't understand the warning message, but for these examples, the
ordinary t interval appears to be better than the bootstrap BCA
interval. I would really appreciate any recommendations anyone can
give on when bootstrap confidence intervals should be used.
Thanks,
Mark
--
Mark Seeto
National Acoustic Laboratories, Australian Hearing
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
================================================================
Robert A. LaBudde, PhD, PAS, Dpl. ACAFS e-mail: r...@lcfltd.com
Least Cost Formulations, Ltd. URL: http://lcfltd.com/
824 Timberlake Drive Tel: 757-467-0954
Virginia Beach, VA 23464-3239 Fax: 757-467-2947
"Vere scire est per causas scire"
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
