On Thu, Oct 16, 2008 at 1:09 AM, Chris MacRaild <[EMAIL PROTECTED]> wrote: >> >> Well, the Jackknife technique >> (http://en.wikipedia.org/wiki/Resampling_(statistics)#Jackknife) does >> something like this. It uses the errors present inside the collected >> data to estimate the parameter errors. It's not great, but is useful >> when errors cannot be measured. You can also use the covariance >> matrix from the optimisation space to estimate errors. Both are rough >> and approximate, and in convoluted spaces (the diffusion tensor space >> and double motion model-free models of Clore et al., 1990) are known >> to have problems. Monte Carlo simulations perform much better in >> complex spaces. >> > > I have used (and extensively tested) Bootstrap resampling for this > problem. In my hands it works very well provided the data quality is > high (which of course it must be if the resulting values are to be of > any use in model-free analysis). In other words it gives errors > indistinguishable from those derived by Monte Carlo based on duplicate > spectra. Bootstraping, like Jacknife, does not depend on an estimate > of peak hight uncertainty. Its success presumably reflects the smooth > and simple optimisation space involved in an exponential fit to good > data - I fully expect it to fail if applied to the complex spaces of > model-free optimisation.
If someone would like bootstrapping for a certain technique, this could added to relax without too much problem by duplicating the Monte Carlo code and making slight modifications. Implementing Jackknife or the covariance matrix for error propagation would be more complex and questionable as to its value. Anyway, if it's not absolutely necessary I will concentrate my efforts on getting Gary Thompson's multi processor code functional (to run relax on clusters, grids, or multi-cpu systems - see https://mail.gna.org/public/relax-devel/2006-04/msg00023.html). And the BMRB and CCPN integration (CCPN at https://mail.gna.org/public/relax-devel/2007-11/msg00037.html continued at https://mail.gna.org/public/relax-devel/2007-12/msg00000.html, and BMRB at https://mail.gna.org/public/relax-devel/2008-07/msg00057.html). One question I have about the bootstrapping you used Chris is, how did you get the errors for the variance of the Gaussian distributions used to generate the bootstrapping samples? The bootstrapping method I know for error analysis is very similar to Monte Carlo simulations. For Monte Carlo simulations you have: 1) Fit the original data set to get the fitted parameter set (this uses the original error set). 2) Generate the back calculated data set from the fitted parameter set. 3) Randomise n times, assuming a Gaussian distribution, the back calculated data set using original error set. 4) Fit the n Monte Carlo data sets as in 1). 5) The values of 1) and standard deviation of 4) give the final parameter values. The bootstrapping technique for error analysis I am familiar with is: 1) Fit the original data set to get the fitted parameter set (this uses the original error set). 2) N/A. 3) Randomise n times, assuming a Gaussian distribution, the original data set using original error set. 4) Fit the n bootstrapped data sets as in 1). 5) The values of 1) and standard deviation of 4) give the final parameter values. Is this how you implemented it? > While on the topic, I can also confirm that baseline RMSD is a good > estimator of peak hight uncertainty. In my hands no sqrt(2) correction > is required. Interestingly, there seems to be no simple relationship > between baseline RMSD and peak volume uncertainty. I never managed to > understand why that is, but perhaps it is related to the behaviour of > noise under apodisation? This is quite useful to know. If you are using peak volumes for an intensity measure and you don't have duplicate spectra, you could be in trouble. I have extensively played with noise and peak position uncertainty in experiments and simulation and I would guess that for volumes the problem is more than just the apodisation. I think that total spectral power; truncation; phasing; apodisation - which affects error smoothing, truncation, peak intensity, spectral power, etc.; zero filling (again related to spectral power); and window position and size all have an effect here. Well they do for the chemical shift uncertainty in my simulations anyway - so much for Bax's LW/SN formula for 2 coupled peaks! This might require a full PhD project in spectral processing to solve. Btw, as I mentioned earlier in this thread there is still the bug in relax's relaxation curve fitting where the standard deviations are averaged rather than the variances! Cheers, Edward _______________________________________________ relax (http://nmr-relax.com) This is the relax-users mailing list [email protected] To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
