Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Contrary to what I understood at the first reading of Antoine's respons and what can do Bootstrapping, it seems that this method is well suited to do the job. http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29#Resampling_residuals If I have well understood this link, after applying this algorithm you obtain a large number of occurrences (Nboot) of each one of your parameter (C1 C2 and C3 in my case) and then you can determine a value and a confidence interval: C1 = mean(C1boot)+/- std(C1boot)*1.95996 C2 = mean(C2boot)+/- std(C2boot)*1.95996 C3 = mean(C3boot)+/- std(C3boot)*1.95996 I think 1.95996 is for a confidence interval of 97.5% (from Student law), but not sure... If it is good, it looks easy to implement and understand. -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028795.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
This makes me also thinking about /Design Of Experiment/. You may get useful ressources about DOE through the module http://atoms.scilab.org/toolboxes/scidoe ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Le 19/02/2014 14:31, Yohann a écrit : Hi Antoine, thank you for your answer but what I need is a confidence interval on each parameter ! A raw empirical approach: Let R be the RMS residue for the best set P of fitting parameters. * change the value of P(1) by -- say -- +1%, for the parameter #1. * calculate the new residue R1p, and the change dR1p = abs(R-R1p) you may symmetrize the process : change P(1) => P(1)*0.99 (-1%) calculate R1m with this biased fitting model, and get dR1=dR1m+dR1p * These dR somewhat represent sensitivities of the fit w.r.t. to each parameter. * Assume that the confidence interval for parameter # is proportional to 1/dR# For "stiff" parameters, dR# will be big (easily divergent), and 1/dR# small. DISCLAIMER : You may use this approach under your own responsability ;) Its main (may be unique) merit is to be simple. Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Le 21/02/2014 16:06, Antoine Monmayrant a écrit : On 02/19/2014 08:55 PM, Samuel Gougeon wrote: Le 19/02/2014 17:32, Claus Futtrup a écrit : Hi Yohan I see that I'm not alone wondering a bit about what you'd like to do. A mathematically solid and readily available technique to corner mathematical rounding errors and other errors of operation is called Interval Analysis. For matlab there's INTLAB. See: http://www.ti3.tu-harburg.de/~rump/intlab/ I'm sure it could be converted to Scilab without big trouble. Maybe I even have an old one laying around (??) somewhere ... maybe. Yes indeed, here : http://www-sop.inria.fr/coprin/logiciels/Int4Sci/ The thing is I don't see how to use these tools to do what Yoahn wants to do. Neither do i. I just got the opportunity to say that it would be nice to have this external toolbox in ATOMS... Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Hi Antoine Initial message from Yohan was not sufficient to determine whether Interval Analysis could do it or not. As I said in my previous answer - IntLab is for determining mathematical rounding errors and other errors of operation ... for example if you make almost division-by-zero, then you have a big gap of potentially correct solutions. This is what IntLab is for, robust mathematics, not curve fitting or analyzing measurement data. Best regards, Claus On 2/21/2014 16:06, Antoine Monmayrant wrote: On 02/19/2014 08:55 PM, Samuel Gougeon wrote: Le 19/02/2014 17:32, Claus Futtrup a écrit : Hi Yohan I see that I'm not alone wondering a bit about what you'd like to do. A mathematically solid and readily available technique to corner mathematical rounding errors and other errors of operation is called Interval Analysis. For matlab there's INTLAB. See: http://www.ti3.tu-harburg.de/~rump/intlab/ I'm sure it could be converted to Scilab without big trouble. Maybe I even have an old one laying around (??) somewhere ... maybe. Yes indeed, here : http://www-sop.inria.fr/coprin/logiciels/Int4Sci/ The thing is I don't see how to use these tools to do what Yoahn wants to do. I think I have the same kind of issues than Yoahn (I've done a fit, how good is it as compare to another one, and what kind of "confidence" or "error bar" should I associate to each parameter). Could you tell us how we can use interval analysis to adress this (just curious, it's a bit far from my field of expertise)? Cheers, Antoine ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users . --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Hello, If you can make the hypothesis that your data is corrupted by gaussian noise, then you can approximate the covariance matrix of your estimated parameters. Let p be the vector of parameters and r(p) the residual vector given by r(p)=sigma^(-1)*(y-Y(p)) where y is your measuement vector, Y(p) the "simulated" measurement, sigma a diagonal matrix with the std error for each measurement. If we denote by drdp(p) the derivative (or jacobian matrix) of r with respect to p then the covariance matrice C of parameters can be estimated by C=F^(-1) where F=drdp(p)' * drdp(p) is the Fisher information matrix. The diagonal terms of V give you the variance of the parameters. Of course, even in the gaussian case, this is a crude approximation Here is (attached) a simple example where the derivative is approximated with the "derivative" macro of Scilab. S. 2014-02-19 14:31 GMT+01:00 Yohann : > Hi Antoine, > thank you for your answer but > what I need is a confidence interval on each parameter ! > Cheers > Yohann > > > > > -- > View this message in context: > http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028742.html > Sent from the Scilab users - Mailing Lists Archives mailing list archive > at Nabble.com. > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users > premier_ordre_stats.sce Description: Binary data premier_ordre_data.dat Description: Binary data ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
On 02/19/2014 08:55 PM, Samuel Gougeon wrote: Le 19/02/2014 17:32, Claus Futtrup a écrit : Hi Yohan I see that I'm not alone wondering a bit about what you'd like to do. A mathematically solid and readily available technique to corner mathematical rounding errors and other errors of operation is called Interval Analysis. For matlab there's INTLAB. See: http://www.ti3.tu-harburg.de/~rump/intlab/ I'm sure it could be converted to Scilab without big trouble. Maybe I even have an old one laying around (??) somewhere ... maybe. Yes indeed, here : http://www-sop.inria.fr/coprin/logiciels/Int4Sci/ The thing is I don't see how to use these tools to do what Yoahn wants to do. I think I have the same kind of issues than Yoahn (I've done a fit, how good is it as compare to another one, and what kind of "confidence" or "error bar" should I associate to each parameter). Could you tell us how we can use interval analysis to adress this (just curious, it's a bit far from my field of expertise)? Cheers, Antoine ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Hello, If you can make the hypothesis that your data is corrupted by gaussian noise, then you can approximate the covariance matrix of your estimated parameters. Let p be the vector of parameters and r(p) the residual vector given by r(p)=sigma^(-1)*(y-Y(p)) where y is your measuement vector, Y(p) the "simulated" measurement, sigma a diagonal matrix with the std error for each measurement. If we denote by drdp(p) the derivative (or jacobian matrix) of r with respect to p then the covariance matrice C of parameters can be estimated by C=F^(-1) where F=drdp(p)' * drdp(p) is the Fisher information matrix. The diagonal terms of V give you the variance of the parameters. Of course, even in the gaussian case, this is a crude approximation S. 2014-02-19 17:10 GMT+01:00 Yohann : > yes I know Denis, It was just an example to illustrate my question. > My real dataset and function to fit are completely different and more > complex. > Thank you > > > > -- > View this message in context: > http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028748.html > Sent from the Scilab users - Mailing Lists Archives mailing list archive > at Nabble.com. > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users > ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Hello. You could have à look at the nls function in my toolbox Grocer, avaolable undet Atoms ( see also my web page dubois.ensae.net/grocer.html). Êric Le 20 févr. 2014 08:32, "Yohann" a écrit : > Hi Antoine, > thank you for your answer but > what I need is a confidence interval on each parameter ! > Cheers > Yohann > > > > > -- > View this message in context: > http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028742.html > Sent from the Scilab users - Mailing Lists Archives mailing list archive > at Nabble.com. > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users > ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Le 19/02/2014 17:32, Claus Futtrup a écrit : Hi Yohan I see that I'm not alone wondering a bit about what you'd like to do. A mathematically solid and readily available technique to corner mathematical rounding errors and other errors of operation is called Interval Analysis. For matlab there's INTLAB. See: http://www.ti3.tu-harburg.de/~rump/intlab/ I'm sure it could be converted to Scilab without big trouble. Maybe I even have an old one laying around (??) somewhere ... maybe. Yes indeed, here : http://www-sop.inria.fr/coprin/logiciels/Int4Sci/ Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Here is a simple example where the derivative is approximated with the "derivative" macro of Scilab. S. 2014-02-20 9:43 GMT+01:00 Stéphane Mottelet : > Hello, > > If you can make the hypothesis that your data is corrupted by gaussian > noise, then you can approximate the covariance matrix of your estimated > parameters. Let p be the vector of parameters and r(p) the residual vector > given by > > r(p)=sigma^(-1)*(y-Y(p)) > > where y is your measuement vector, Y(p) the "simulated" measurement, sigma > a diagonal matrix with the std error for each measurement. If we denote by > > drdp(p) the derivative (or jacobian matrix) of r with respect to p then > the covariance matrice C of parameters can be estimated by > > > C=F^(-1) > > where > > F=drdp(p)' * drdp(p) > > is the Fisher information matrix. The diagonal terms of V give you the > variance of the parameters. Of course, even in the gaussian case, this is a > crude approximation > > S. > > > 2014-02-19 17:10 GMT+01:00 Yohann : > > yes I know Denis, It was just an example to illustrate my question. >> My real dataset and function to fit are completely different and more >> complex. >> Thank you >> >> >> >> -- >> View this message in context: >> http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028748.html >> Sent from the Scilab users - Mailing Lists Archives mailing list archive >> at Nabble.com. >> ___ >> users mailing list >> users@lists.scilab.org >> http://lists.scilab.org/mailman/listinfo/users >> > > premier_ordre_stats.sce Description: Binary data premier_ordre_data.dat Description: Binary data ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
On 02/19/2014 02:31 PM, Yohann wrote: Hi Antoine, thank you for your answer but what I need is a confidence interval on each parameter ! Well, so resampling can do the trick! Antoine Cheers Yohann -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028742.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users -- +++ Antoine Monmayrant LAAS - CNRS 7 avenue du Colonel Roche BP 54200 31031 TOULOUSE Cedex 4 FRANCE Tel:+33 5 61 33 64 59 email : antoine.monmayr...@laas.fr permanent email : antoine.monmayr...@polytechnique.org +++ ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Le 19/02/2014 14:31, Yohann a écrit : Hi Antoine, thank you for your answer but what I need is a confidence interval on each parameter ! The confidence interval on the computed parameters depends on the confidence interval on your x and y data. In your case where you are looking for polynomial fitting then perturbations on y produces perturbation on the computed parameters roughly proportionnal to the conditionning of the matrix A=[x*x x ones(x)] if c is the mean square solution of the probleme for y and c1 is the mean square solution of the problem of y1 then norm(c-c1)/norm(c)<=cond(A)*norm(y-y1)/norm(y) x = [ 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6]'; y = [5.02 6.08 3.33 -0.93 -0.22 7.83 16.52 15.55 2.67 -11.42 -11.78 5.09 25.25]'; A=[x.^2 x ones(x)]; y1=y;y1=y1.*(1+0.01*rand(y1));ey=norm(y-y1)/norm(y) c1=A\y1;ec=norm(c-c1)/norm(c) ec<=cond(A)*ey the dependance with respect to perturbation on x is more difficult. Cheers Yohann -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028742.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
yes I know Denis, It was just an example to illustrate my question. My real dataset and function to fit are completely different and more complex. Thank you -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028748.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Hi Antoine, thank you for your answer but what I need is a confidence interval on each parameter ! Cheers Yohann -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696p4028742.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Hi Yohan I see that I'm not alone wondering a bit about what you'd like to do. A mathematically solid and readily available technique to corner mathematical rounding errors and other errors of operation is called Interval Analysis. For matlab there's INTLAB. See: http://www.ti3.tu-harburg.de/~rump/intlab/ I'm sure it could be converted to Scilab without big trouble. Maybe I even have an old one laying around (??) somewhere ... maybe. Another approach would be in a specific situation to apply the techniques of interval analysis, only centered around your concern ... and leave the rest of your script normal. Best regards, Claus On 2/18/2014 15:06, Yohann wrote: Hi all !! First of all, thank you to all contributors of Scilab. However, I have a problem to calculate the estimated error on each parameter calculated with leastsq. For example if I have the simple following function to fit: !!! function y = fun2fit(x, c) y = c(1)*x*x + c(2)*x + c(3); endfunction function e = myerror(c, x, y) e = fun2fit(x, c) - y; endfunction // Data x = [ 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6]'; y = [5.02 6.08 3.33 -0.93 -0.22 7.83 16.52 15.55 2.67 -11.42 -11.78 5.09 25.25]'; // First guess c0 = [1 1 1]'; y0 = fun2fit(x, c0); // least square [f, copt] = leastsq(list(myerror, x, y), c0) !!! I'd like to have an estimated error on each of the 3 element of copt. Any idea ? Thanks ! Y -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users . --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
Dear Yohann The (x,y) dataset does not look like a parabola... (it resembles more a type of x*sin(x) function). Trying to fit this dataset with a parabola, will not only give unreliable result, but also will put a high challenge to error estimation algorithms... Denis -Message d'origine- De : users [mailto:users-boun...@lists.scilab.org] De la part de Yohann Envoyé : mardi 18 février 2014 15:07 À : users@lists.scilab.org Objet : [Scilab-users] evaluate error on each parameter calculated with leastsq Hi all !! First of all, thank you to all contributors of Scilab. However, I have a problem to calculate the estimated error on each parameter calculated with leastsq. For example if I have the simple following function to fit: !!! function y = fun2fit(x, c) y = c(1)*x*x + c(2)*x + c(3); endfunction function e = myerror(c, x, y) e = fun2fit(x, c) - y; endfunction // Data x = [ 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6]'; y = [5.02 6.08 3.33 -0.93 -0.22 7.83 16.52 15.55 2.67 -11.42 -11.78 5.09 25.25]'; // First guess c0 = [1 1 1]'; y0 = fun2fit(x, c0); // least square [f, copt] = leastsq(list(myerror, x, y), c0) !!! I'd like to have an estimated error on each of the 3 element of copt. Any idea ? Thanks ! Y -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] evaluate error on each parameter calculated with leastsq
On 02/18/2014 03:06 PM, Yohann wrote: Hi all !! First of all, thank you to all contributors of Scilab. However, I have a problem to calculate the estimated error on each parameter calculated with leastsq. For example if I have the simple following function to fit: !!! function y = fun2fit(x, c) y = c(1)*x*x + c(2)*x + c(3); endfunction function e = myerror(c, x, y) e = fun2fit(x, c) - y; endfunction // Data x = [ 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6]'; y = [5.02 6.08 3.33 -0.93 -0.22 7.83 16.52 15.55 2.67 -11.42 -11.78 5.09 25.25]'; // First guess c0 = [1 1 1]'; y0 = fun2fit(x, c0); // least square [f, copt] = leastsq(list(myerror, x, y), c0) !!! I'd like to have an estimated error on each of the 3 element of copt. Any idea ? What do you mean? Do you want a confidence interval or any kind of "error bar" for each parameter? Or any way to see how your fit is dependent on each parameter? If it's the case, you might be interested in bootstraping or jackknifing: http://en.wikipedia.org/wiki/Resampling_(statistics) These methods are quite ressource hungry but work quite well to measure the quality and robustness of your fit. Cheers, Antoine Thanks ! Y -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users -- +++ Antoine Monmayrant LAAS - CNRS 7 avenue du Colonel Roche BP 54200 31031 TOULOUSE Cedex 4 FRANCE Tel:+33 5 61 33 64 59 email : antoine.monmayr...@laas.fr permanent email : antoine.monmayr...@polytechnique.org +++ ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
[Scilab-users] evaluate error on each parameter calculated with leastsq
Hi all !! First of all, thank you to all contributors of Scilab. However, I have a problem to calculate the estimated error on each parameter calculated with leastsq. For example if I have the simple following function to fit: !!! function y = fun2fit(x, c) y = c(1)*x*x + c(2)*x + c(3); endfunction function e = myerror(c, x, y) e = fun2fit(x, c) - y; endfunction // Data x = [ 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6]'; y = [5.02 6.08 3.33 -0.93 -0.22 7.83 16.52 15.55 2.67 -11.42 -11.78 5.09 25.25]'; // First guess c0 = [1 1 1]'; y0 = fun2fit(x, c0); // least square [f, copt] = leastsq(list(myerror, x, y), c0) !!! I'd like to have an estimated error on each of the 3 element of copt. Any idea ? Thanks ! Y -- View this message in context: http://mailinglists.scilab.org/evaluate-error-on-each-parameter-calculated-with-leastsq-tp4028696.html Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
