Re: [R] Curve fitting
Generally nlsr package has better reliability in getting parameter estimates because it tries to use automatic derivatives rather than a rather poor numerical estimate, and also uses a Levenberg-Marquardt stabilization of the linearized model. However, nls() can sometimes be a bit more flexible. JN On 2020-04-05 3:20 p.m., Jeff Newmiller wrote: > err... stats::nls... > > On April 5, 2020 12:14:15 PM PDT, Jeff Newmiller > wrote: >> stats::nlm? >> >> On April 5, 2020 11:53:10 AM PDT, Bernard Comcast >> wrote: >>> Any recommendations on an R package to fit data to a nonlinear model >>> Y=f(x) with a single x and y variable? >>> >>> I want to be able to generate parameter uncertainty estimates and >>> prediction uncertainties if possible. >>> >>> Bernard >>> Sent from my iPhone so please excuse the spelling!" >>> __ >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide >>> http://www.R-project.org/posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. > __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting
Thanks Jeff Bernard Sent from my iPhone so please excuse the spelling!" > On Apr 5, 2020, at 3:14 PM, Jeff Newmiller wrote: > > stats::nlm? > >> On April 5, 2020 11:53:10 AM PDT, Bernard Comcast >> wrote: >> Any recommendations on an R package to fit data to a nonlinear model >> Y=f(x) with a single x and y variable? >> >> I want to be able to generate parameter uncertainty estimates and >> prediction uncertainties if possible. >> >> Bernard >> Sent from my iPhone so please excuse the spelling!" >> __ >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > -- > Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting
err... stats::nls... On April 5, 2020 12:14:15 PM PDT, Jeff Newmiller wrote: >stats::nlm? > >On April 5, 2020 11:53:10 AM PDT, Bernard Comcast > wrote: >>Any recommendations on an R package to fit data to a nonlinear model >>Y=f(x) with a single x and y variable? >> >>I want to be able to generate parameter uncertainty estimates and >>prediction uncertainties if possible. >> >>Bernard >>Sent from my iPhone so please excuse the spelling!" >>__ >>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>https://stat.ethz.ch/mailman/listinfo/r-help >>PLEASE do read the posting guide >>http://www.R-project.org/posting-guide.html >>and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting
stats::nlm? On April 5, 2020 11:53:10 AM PDT, Bernard Comcast wrote: >Any recommendations on an R package to fit data to a nonlinear model >Y=f(x) with a single x and y variable? > >I want to be able to generate parameter uncertainty estimates and >prediction uncertainties if possible. > >Bernard >Sent from my iPhone so please excuse the spelling!" >__ >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Curve fitting
Any recommendations on an R package to fit data to a nonlinear model Y=f(x) with a single x and y variable? I want to be able to generate parameter uncertainty estimates and prediction uncertainties if possible. Bernard Sent from my iPhone so please excuse the spelling!" __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting an exponential function
Hi Waltenegus, you should consider to show us your code and the data you used to fit the curve. If you can't or if you prefer you can simply show the code and some sample data on which the code can be run. You'll find nice and useful tool in nlstools package. Hope this help Giuseppe - Giuseppe Calamita PhD at CNR-IMAA Italian National Council of Research - Institute of Methodologies for Environmental Analysis, Tito Scalo -Potenza ITALY -- View this message in context: http://r.789695.n4.nabble.com/curve-fitting-an-exponential-function-tp4644765p4645439.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting, probably splines
You could use the summation convention built into mgcv:gam for this. See ?linear.functional.terms for details, but here is some example code, both for the exact match, you describe, and a noisy version. best, Simon library(mgcv) f2 - function(x) 0.2*x^11*(10*(1-x))^6+10*(10*x)^3*(1-x)^10 ## test func x - (1:1000-.5)/1000 ## quadrature points f - f2(x) ## function values at x X - matrix(x,20,1000,byrow=TRUE) L - X*0 for (i in 1:20) L[i,1:50+(i-1)*50] - 1/50 y - L%*%f ## average function values in 20 intervals ## estimate f from average values, using summation convention ## in gam. Basically smooth is evaluated at each value in X ## to give a matrix s(X) and then rowSums(s(X)*L) is predictor ## for y ... b - gam(y~s(X,by=L,fx=TRUE,k=20)) plot(b,shift=coef(b)[1]) ## estimate lines(x,f,col=2) ## truth ## noisy version... y - y + rnorm(10)*.1 ## add noise to averages b - gam(y~s(X,by=L),method=REML) plot(b,shift=coef(b)[1]) ## estimate lines(x,f,col=2) ## truth On 04/12/2012 02:45 PM, Michael Haenlein wrote: Dear all, This is probably more related to statistics than to [R] but I hope someone can give me an idea how to solve it nevertheless: Assume I have a variable y that is a function of x: y=f(x). I know the average value of y for different intervals of x. For example, I know that in the interval[0;x1] the average y is y1, in the interval [x1;x2] the average y is y2 and so forth. I would like to find a line of minimum curvature so that the average values of y in each interval correspond to y1, y2, ... My idea was to use (cubic) splines. But the problem I have seems somewhat different to what is usually done with splines. As far as I understand it, splines help to find a curve that passes a set of given points. But I don't have any points, I only have average values of y per interval. If you have any suggestions on how to solve this, I'd love to hear them. Thanks very much in advance, Michael [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting, probably splines
This sounds like possibly using logsplines may be what you want. See the 'oldlogspline' function in the 'logspline' package. On Thu, Apr 12, 2012 at 7:45 AM, Michael Haenlein haenl...@escpeurope.eu wrote: Dear all, This is probably more related to statistics than to [R] but I hope someone can give me an idea how to solve it nevertheless: Assume I have a variable y that is a function of x: y=f(x). I know the average value of y for different intervals of x. For example, I know that in the interval[0;x1] the average y is y1, in the interval [x1;x2] the average y is y2 and so forth. I would like to find a line of minimum curvature so that the average values of y in each interval correspond to y1, y2, ... My idea was to use (cubic) splines. But the problem I have seems somewhat different to what is usually done with splines. As far as I understand it, splines help to find a curve that passes a set of given points. But I don't have any points, I only have average values of y per interval. If you have any suggestions on how to solve this, I'd love to hear them. Thanks very much in advance, Michael [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Gregory (Greg) L. Snow Ph.D. 538...@gmail.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Curve fitting, probably splines
Dear all, This is probably more related to statistics than to [R] but I hope someone can give me an idea how to solve it nevertheless: Assume I have a variable y that is a function of x: y=f(x). I know the average value of y for different intervals of x. For example, I know that in the interval[0;x1] the average y is y1, in the interval [x1;x2] the average y is y2 and so forth. I would like to find a line of minimum curvature so that the average values of y in each interval correspond to y1, y2, ... My idea was to use (cubic) splines. But the problem I have seems somewhat different to what is usually done with splines. As far as I understand it, splines help to find a curve that passes a set of given points. But I don't have any points, I only have average values of y per interval. If you have any suggestions on how to solve this, I'd love to hear them. Thanks very much in advance, Michael [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
Many thanks for the suggestion! That may reduce the computational time needed to find x value given the y one (for hundreds of pairs). Certainly, I will look into manuals for approx() and approxfun() in this regard. Again, thanks for your taking time to read my previous posts and make this valuable suggestion. Regards, Joseph On Sat, May 1, 2010 at 12:41 AM, Greg Snow greg.s...@imail.org wrote: I did not understand enough of the rest of your question to give any better response than others have given. Looking back at your previous posts, there is one suggestion that I can make that may help. You can use the approx or approxfun functions to approximate an inverse, just generate a bunch of x,y pairs from your function, then feed them to approx while switching x and y. Not an exact inverse, but if you give it enough points then it will be close. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Friday, April 30, 2010 5:24 PM To: Greg Snow Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations I have already learned a lot from the list, both technical and not, and cannot thank enough for those valuable suggestions. In fact, as said in my previous posts, I got really critical help and advices, which really addresses the issues I have. By the way, there is one point or two in your post I agree on, but I am not sure why you just pointed out side issues (by snipping a part of my saying) without touching the main topic of this thread at all. I can go on but won't because arguing for the sake of argument is of no value to anyone in this thread. It would have been better if you could have focused on the topic and provided some technical and practical information which I could learn from and be very thankful for. Regards, Joseph On Fri, Apr 30, 2010 at 11:35 PM, Greg Snow greg.s...@imail.org wrote: -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- project.org] On Behalf Of Kyeong Soo (Joseph) Kim Sent: Friday, April 30, 2010 4:10 AM To: kMan Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations [snip] By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. In my experience with this list, and others, the perceived level of cynical/skeptical/arrogant answers has more to do with the reader than with the writer. If you want to be offended, you will find things to be offended about even when none was intended. If you look for help and useful responses (follow the posting guide) and are thankful for what you learn, you will learn more and be bothered less. R-help is a mixture of different levels and cultures. In framing responses it is hard to know what the other person may find offensive (I was once yelled at and chewed out quite thoroughly for truthfully answering no when asked if I drink coffee). Most responders on this list (actually I would say all, but there might be an exception that I have not noticed) are trying to be helpful, there is just a large variability in the tones of the responses. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
Dear Keith, Thanks for the suggestion and taking your time to respond to it. But, you misunderstand something and seems that you do not read all my previous e-mails. For instance, can a hand-drawing curve give you an inverse function (analytically or numerically) so that you can find an x value given the y value (not just for one, but for hundreds of points)? As for the statistical inferences, I admit that my communications were not that very clear. My intention is to get a smoothed curve from the simulation data in a statistically meaningful way as much as possible for my intended use of the resulting curve. As said before, I don't know all the thorough theoretical details behind regression and curve fitting functions available in R (know the basics though as one with PhD in Elec. Eng. unlike someone's assessment), but am doing my best to catch up reading textbooks and manuals, and posting this question to this list is definitely a way to learn from many experts and advanced users of R. By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. Again, what I want to hear from the list is the proper use of regression/curve fitting functions of R for my simulation data with replications: Applying after taking means or directly on them? So far I haven't heard anyone even specifically touching my question, although there were several seemingly related suggestions. Regards, Joseph On Fri, Apr 30, 2010 at 4:25 AM, kMan kchambe...@gmail.com wrote: Dear Joseph, If you do not need to make any inferences, that is, you just want it to look pretty, then drawing a curve by hand is as good a solution as any. Plus, there is no reason for expert testimony to say that the curve does not mean anything. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Tuesday, April 27, 2010 2:33 PM To: Gabor Grothendieck Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication network (like M/M/1 queue, but way too complicated to mathematically analyze it using classical queueing theory) and the conclusion I want to make is qualitative rather than quantatitive -- a high-level comparative study of various network architectures based on the equivalence principle (a concept specific to netwokring, not in the general sense). What l want in this regard is a smooth, non-decreasing (hence one-to-one) function built out of simulation data because later in my processing, I need an inverse function of the said curve to find out an x value given the y value. That was, in fact, the reason I used the exponential (i.e., non-decreasing function) curve fiting. Even though I don't need a statistical inference framework for my work, I want to make sure that my use of regression/curve fitting techniques with my simulation data (as a tool for getting the mentioned curve) is proper and a usual practice among experts like you. To get answer to my question, I digged a lot through the Internet but found no clear explanation so far. Your suggestions and providing examples (always!) are much appreciated, but I am still not sure the use of those regression procedures with the kind of data I described is a right way to do. Again, many thanks for your prompt and kind answers, Joseph On Tue, Apr 27, 2010 at 8:46 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has a book associated with it if you need more info. e.g. library(mgcv) fm - gam(dist ~ s(speed), data = cars) summary(fm) plot(dist ~ speed, cars, pch = 20) fm.ci - with(predict(fm, se = TRUE), cbind(0, -2*se.fit, 2*se.fit) + c(fit)) matlines(cars$speed, fm.ci, lty = c(1, 2, 2), col = c(1, 2, 2)) On Tue, Apr 27, 2010 at 3:07 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: Hello Gabor, Many thanks for providing actual examples for the problem! In fact I know how to apply and generate plots using various R functions including loess, lowess, and smooth.spline procedures. My question, however, is whether applying those procedures directly on the data with multiple observations/duplicate points(?) is on the sound basis or not. Before asking
Re: [R] Curve Fitting/Regression with Multiple Observations
You may want to run RSiteSearch(monotone splines) at the R prompt. The 3rd hit looks quite promising. However, if I understand your data, you have multiple y values for the same x values. If so, can you justify inverting the regression function? The traffic on this mailing list is very high, and the signal to noise ratio is rather low. This has the tendency of burning out those who started with good intentions to help. Andy From: Kyeong Soo (Joseph) Kim Dear Keith, Thanks for the suggestion and taking your time to respond to it. But, you misunderstand something and seems that you do not read all my previous e-mails. For instance, can a hand-drawing curve give you an inverse function (analytically or numerically) so that you can find an x value given the y value (not just for one, but for hundreds of points)? As for the statistical inferences, I admit that my communications were not that very clear. My intention is to get a smoothed curve from the simulation data in a statistically meaningful way as much as possible for my intended use of the resulting curve. As said before, I don't know all the thorough theoretical details behind regression and curve fitting functions available in R (know the basics though as one with PhD in Elec. Eng. unlike someone's assessment), but am doing my best to catch up reading textbooks and manuals, and posting this question to this list is definitely a way to learn from many experts and advanced users of R. By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. Again, what I want to hear from the list is the proper use of regression/curve fitting functions of R for my simulation data with replications: Applying after taking means or directly on them? So far I haven't heard anyone even specifically touching my question, although there were several seemingly related suggestions. Regards, Joseph On Fri, Apr 30, 2010 at 4:25 AM, kMan kchambe...@gmail.com wrote: Dear Joseph, If you do not need to make any inferences, that is, you just want it to look pretty, then drawing a curve by hand is as good a solution as any. Plus, there is no reason for expert testimony to say that the curve does not mean anything. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Tuesday, April 27, 2010 2:33 PM To: Gabor Grothendieck Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication network (like M/M/1 queue, but way too complicated to mathematically analyze it using classical queueing theory) and the conclusion I want to make is qualitative rather than quantatitive -- a high-level comparative study of various network architectures based on the equivalence principle (a concept specific to netwokring, not in the general sense). What l want in this regard is a smooth, non-decreasing (hence one-to-one) function built out of simulation data because later in my processing, I need an inverse function of the said curve to find out an x value given the y value. That was, in fact, the reason I used the exponential (i.e., non-decreasing function) curve fiting. Even though I don't need a statistical inference framework for my work, I want to make sure that my use of regression/curve fitting techniques with my simulation data (as a tool for getting the mentioned curve) is proper and a usual practice among experts like you. To get answer to my question, I digged a lot through the Internet but found no clear explanation so far. Your suggestions and providing examples (always!) are much appreciated, but I am still not sure the use of those regression procedures with the kind of data I described is a right way to do. Again, many thanks for your prompt and kind answers, Joseph On Tue, Apr 27, 2010 at 8:46 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has a book associated with it if you need more info. e.g. library(mgcv) fm - gam(dist ~ s(speed), data = cars) summary(fm) plot(dist ~ speed, cars, pch = 20) fm.ci - with(predict(fm, se = TRUE), cbind(0, -2*se.fit
Re: [R] Curve Fitting/Regression with Multiple Observations
Dear Joseph, I have had a similar experience to replies. Andy's assessment about signal to noise on the list is, I believe, quite accurate, and quite elegant. My experience has generally been that R-replies get better with age. I welcome the feedback you just provided. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Friday, April 30, 2010 4:10 AM To: kMan Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Dear Keith, Thanks for the suggestion and taking your time to respond to it. But, you misunderstand something and seems that you do not read all my previous e-mails. For instance, can a hand-drawing curve give you an inverse function (analytically or numerically) so that you can find an x value given the y value (not just for one, but for hundreds of points)? As for the statistical inferences, I admit that my communications were not that very clear. My intention is to get a smoothed curve from the simulation data in a statistically meaningful way as much as possible for my intended use of the resulting curve. As said before, I don't know all the thorough theoretical details behind regression and curve fitting functions available in R (know the basics though as one with PhD in Elec. Eng. unlike someone's assessment), but am doing my best to catch up reading textbooks and manuals, and posting this question to this list is definitely a way to learn from many experts and advanced users of R. By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. Again, what I want to hear from the list is the proper use of regression/curve fitting functions of R for my simulation data with replications: Applying after taking means or directly on them? So far I haven't heard anyone even specifically touching my question, although there were several seemingly related suggestions. Regards, Joseph On Fri, Apr 30, 2010 at 4:25 AM, kMan kchambe...@gmail.com wrote: Dear Joseph, If you do not need to make any inferences, that is, you just want it to look pretty, then drawing a curve by hand is as good a solution as any. Plus, there is no reason for expert testimony to say that the curve does not mean anything. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Tuesday, April 27, 2010 2:33 PM To: Gabor Grothendieck Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication network (like M/M/1 queue, but way too complicated to mathematically analyze it using classical queueing theory) and the conclusion I want to make is qualitative rather than quantatitive -- a high-level comparative study of various network architectures based on the equivalence principle (a concept specific to netwokring, not in the general sense). What l want in this regard is a smooth, non-decreasing (hence one-to-one) function built out of simulation data because later in my processing, I need an inverse function of the said curve to find out an x value given the y value. That was, in fact, the reason I used the exponential (i.e., non-decreasing function) curve fiting. Even though I don't need a statistical inference framework for my work, I want to make sure that my use of regression/curve fitting techniques with my simulation data (as a tool for getting the mentioned curve) is proper and a usual practice among experts like you. To get answer to my question, I digged a lot through the Internet but found no clear explanation so far. Your suggestions and providing examples (always!) are much appreciated, but I am still not sure the use of those regression procedures with the kind of data I described is a right way to do. Again, many thanks for your prompt and kind answers, Joseph On Tue, Apr 27, 2010 at 8:46 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has a book associated with it if you need more info. e.g. library(mgcv) fm - gam(dist ~ s(speed), data = cars) summary(fm) plot(dist ~ speed, cars, pch = 20) fm.ci - with(predict(fm, se = TRUE), cbind(0, -2*se.fit, 2*se.fit
Re: [R] Curve Fitting/Regression with Multiple Observations
Dear Andy, You're the kind soul I mentioned in my previous e-mail! Certainly yours is the kind of response I've been looking for, and now I can start with that, especially splinefun() with monoH.FC method. As for my simulation data, your understanding is correct; there are multiple y values from different replications for the same x values. Even though there are multiple y values for a given x value, this could be interpreted as the combination of multiple, different random components (inherent in any monte carlo simulation) + one fixed, unknown deterministic component. So underlying assumption is that there is a one-to-one (monotone) function between x and y. This is typical in many computer simulation in networking. As said before, for instance, you can get a nice, closed-form (monotone) function of utilization (i.e., \rho) for the average delay of customers in the queueing system in M/M/1 queue. The simulation with different random seeds, however, gives slightly different average delays for a given utilization per run. Still, we know from the underlying model that there is one-to-one correspondence between the utilization and the average delay. Of course, unlike the simple M/M/1 queue, for most of actual networking systems to analyze, we don't know the exact models, but it is well accepted and assumed in nearly all existing work in this area that there is still one-to-one correspondence between the utilization (or system load) and performance measures like delay, throughput, and packet loss. I do appreciate your suggestion and this would be of tremendous help for my current research. Also, thanks for the assessment on this list, which I take as a valuable advice in the future. With Regards, Joseph On Fri, Apr 30, 2010 at 12:52 PM, Liaw, Andy andy_l...@merck.com wrote: You may want to run RSiteSearch(monotone splines) at the R prompt. The 3rd hit looks quite promising. However, if I understand your data, you have multiple y values for the same x values. If so, can you justify inverting the regression function? The traffic on this mailing list is very high, and the signal to noise ratio is rather low. This has the tendency of burning out those who started with good intentions to help. Andy From: Kyeong Soo (Joseph) Kim Dear Keith, Thanks for the suggestion and taking your time to respond to it. But, you misunderstand something and seems that you do not read all my previous e-mails. For instance, can a hand-drawing curve give you an inverse function (analytically or numerically) so that you can find an x value given the y value (not just for one, but for hundreds of points)? As for the statistical inferences, I admit that my communications were not that very clear. My intention is to get a smoothed curve from the simulation data in a statistically meaningful way as much as possible for my intended use of the resulting curve. As said before, I don't know all the thorough theoretical details behind regression and curve fitting functions available in R (know the basics though as one with PhD in Elec. Eng. unlike someone's assessment), but am doing my best to catch up reading textbooks and manuals, and posting this question to this list is definitely a way to learn from many experts and advanced users of R. By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. Again, what I want to hear from the list is the proper use of regression/curve fitting functions of R for my simulation data with replications: Applying after taking means or directly on them? So far I haven't heard anyone even specifically touching my question, although there were several seemingly related suggestions. Regards, Joseph On Fri, Apr 30, 2010 at 4:25 AM, kMan kchambe...@gmail.com wrote: Dear Joseph, If you do not need to make any inferences, that is, you just want it to look pretty, then drawing a curve by hand is as good a solution as any. Plus, there is no reason for expert testimony to say that the curve does not mean anything. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Tuesday, April 27, 2010 2:33 PM To: Gabor Grothendieck Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication
Re: [R] Curve Fitting/Regression with Multiple Observations
Dear Keith, I will keep that in mind in my future posting. Again, thanks for your time and advice! Regards, Joseph On Fri, Apr 30, 2010 at 3:54 PM, kMan kchambe...@gmail.com wrote: Dear Joseph, I have had a similar experience to replies. Andy's assessment about signal to noise on the list is, I believe, quite accurate, and quite elegant. My experience has generally been that R-replies get better with age. I welcome the feedback you just provided. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Friday, April 30, 2010 4:10 AM To: kMan Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Dear Keith, Thanks for the suggestion and taking your time to respond to it. But, you misunderstand something and seems that you do not read all my previous e-mails. For instance, can a hand-drawing curve give you an inverse function (analytically or numerically) so that you can find an x value given the y value (not just for one, but for hundreds of points)? As for the statistical inferences, I admit that my communications were not that very clear. My intention is to get a smoothed curve from the simulation data in a statistically meaningful way as much as possible for my intended use of the resulting curve. As said before, I don't know all the thorough theoretical details behind regression and curve fitting functions available in R (know the basics though as one with PhD in Elec. Eng. unlike someone's assessment), but am doing my best to catch up reading textbooks and manuals, and posting this question to this list is definitely a way to learn from many experts and advanced users of R. By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. Again, what I want to hear from the list is the proper use of regression/curve fitting functions of R for my simulation data with replications: Applying after taking means or directly on them? So far I haven't heard anyone even specifically touching my question, although there were several seemingly related suggestions. Regards, Joseph On Fri, Apr 30, 2010 at 4:25 AM, kMan kchambe...@gmail.com wrote: Dear Joseph, If you do not need to make any inferences, that is, you just want it to look pretty, then drawing a curve by hand is as good a solution as any. Plus, there is no reason for expert testimony to say that the curve does not mean anything. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Tuesday, April 27, 2010 2:33 PM To: Gabor Grothendieck Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication network (like M/M/1 queue, but way too complicated to mathematically analyze it using classical queueing theory) and the conclusion I want to make is qualitative rather than quantatitive -- a high-level comparative study of various network architectures based on the equivalence principle (a concept specific to netwokring, not in the general sense). What l want in this regard is a smooth, non-decreasing (hence one-to-one) function built out of simulation data because later in my processing, I need an inverse function of the said curve to find out an x value given the y value. That was, in fact, the reason I used the exponential (i.e., non-decreasing function) curve fiting. Even though I don't need a statistical inference framework for my work, I want to make sure that my use of regression/curve fitting techniques with my simulation data (as a tool for getting the mentioned curve) is proper and a usual practice among experts like you. To get answer to my question, I digged a lot through the Internet but found no clear explanation so far. Your suggestions and providing examples (always!) are much appreciated, but I am still not sure the use of those regression procedures with the kind of data I described is a right way to do. Again, many thanks for your prompt and kind answers, Joseph On Tue, Apr 27, 2010 at 8:46 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has
[R] Curve Fitting
I am having troubles in fitting functions of the form y~a*x^b+c to data, for example x-c(0.1,0.36,0.63,0.90,1.166,1.43, 1.70, 1.96, 2.23) y-c(8.09,9.0,9.62,10.11,10.53,10.9, 11.25, 11.56, 11.86) I tried for example with nls, which did only work with really good initial guessed values. Any suggestion, what I should use? Thanks a lot Thomas [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting
Thomas, I think the issue of having reasonable starting values is inherent in all nonlinear optimization problems (unless they have some additional properties like convexity, for example). Using a different algorithm may or may not help. In fact, a vast majority of existing algorithms guarantees only local convergence provided that the starting point is within a certain distance from the solution. If you are interested in this particular model, the following procedure seems to give reasonable starting point. 1. Take a guess at c. This is relatively easy since c is the value of y when a=0. For your data a crude guess could be c=8. 2. For a fixed value of c your model can be linearized by the following transformations y1 - log(y-8) x1 - log(x) 3. Do summary(lm(y1~x1)) and obtain values for intecept and slope 4. Your starting values for nls are (a=exp(Intercept), b=slope, c=8) Hope this helps, Andy __ Andy Jaworski 518-1-01 Process Laboratory 3M Corporate Research Laboratory - E-mail: apjawor...@mmm.com Tel: (651) 733-6092 Fax: (651) 736-3122 From: Thomas Bschorr bsch...@phys.ethz.ch To: r-help@r-project.org Date: 04/30/2010 03:33 PM Subject: [R] Curve Fitting Sent by: r-help-boun...@r-project.org I am having troubles in fitting functions of the form y~a*x^b+c to data, for example x-c(0.1,0.36,0.63,0.90,1.166,1.43, 1.70, 1.96, 2.23) y-c(8.09,9.0,9.62,10.11,10.53,10.9, 11.25, 11.56, 11.86) I tried for example with nls, which did only work with really good initial guessed values. Any suggestion, what I should use? Thanks a lot Thomas [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting
You can use nls2 to try many starting values. It works just like nls but: - if you give it a two row data frame as the start value it will create a grid between the upper and lower values of each parameter and then run an optimization starting at each such point on the grid returning the best - it has all the algorithms that nls has (because it calls nls) plus an additional algorithm called brute-force which simply evaluates the formula at the start value Thus you can use it in two ways: 1. perform a brute force run and take the best to get a good starting value and using that starting value run nls, or 2. run an nls from each starting value Obviously the second approach takes more run time but has some advantages in certain cases. In many cases either approach will work. library(nls2) # 1. First approach fo - y ~ a * x ^ b + c start - data.frame(a = c(-10, 10), b = c(-10, 10), c = c(-10, 10)) fm.brute - nls2(fo, start = start, alg = brute) fm - nls2(fo, start = fm.brute) fm Nonlinear regression model model: y ~ a * x^b + c data: environment a b c 3.1865 0.5025 7.0900 residual sum-of-squares: 5.677e-05 Number of iterations to convergence: 7 Achieved convergence tolerance: 4.437e-06 # 2. Second approach fm2 - nls2(fo, start = start, control = nls.control(warnOnly = TRUE)) There were 32 warnings (use warnings() to see them) fm2 Nonlinear regression model model: y ~ a * x^b + c data: environment a b c 3.1865 0.5025 7.0900 residual sum-of-squares: 5.677e-05 Number of iterations to convergence: 9 Achieved convergence tolerance: 1.149e-07 On Fri, Apr 30, 2010 at 4:30 PM, Thomas Bschorr bsch...@phys.ethz.ch wrote: I am having troubles in fitting functions of the form y~a*x^b+c to data, for example x-c(0.1,0.36,0.63,0.90,1.166,1.43, 1.70, 1.96, 2.23) y-c(8.09,9.0,9.62,10.11,10.53,10.9, 11.25, 11.56, 11.86) I tried for example with nls, which did only work with really good initial guessed values. Any suggestion, what I should use? Thanks a lot Thomas [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
-Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- project.org] On Behalf Of Kyeong Soo (Joseph) Kim Sent: Friday, April 30, 2010 4:10 AM To: kMan Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations [snip] By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. In my experience with this list, and others, the perceived level of cynical/skeptical/arrogant answers has more to do with the reader than with the writer. If you want to be offended, you will find things to be offended about even when none was intended. If you look for help and useful responses (follow the posting guide) and are thankful for what you learn, you will learn more and be bothered less. R-help is a mixture of different levels and cultures. In framing responses it is hard to know what the other person may find offensive (I was once yelled at and chewed out quite thoroughly for truthfully answering no when asked if I drink coffee). Most responders on this list (actually I would say all, but there might be an exception that I have not noticed) are trying to be helpful, there is just a large variability in the tones of the responses. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
I have already learned a lot from the list, both technical and not, and cannot thank enough for those valuable suggestions. In fact, as said in my previous posts, I got really critical help and advices, which really addresses the issues I have. By the way, there is one point or two in your post I agree on, but I am not sure why you just pointed out side issues (by snipping a part of my saying) without touching the main topic of this thread at all. I can go on but won't because arguing for the sake of argument is of no value to anyone in this thread. It would have been better if you could have focused on the topic and provided some technical and practical information which I could learn from and be very thankful for. Regards, Joseph On Fri, Apr 30, 2010 at 11:35 PM, Greg Snow greg.s...@imail.org wrote: -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- project.org] On Behalf Of Kyeong Soo (Joseph) Kim Sent: Friday, April 30, 2010 4:10 AM To: kMan Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations [snip] By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. In my experience with this list, and others, the perceived level of cynical/skeptical/arrogant answers has more to do with the reader than with the writer. If you want to be offended, you will find things to be offended about even when none was intended. If you look for help and useful responses (follow the posting guide) and are thankful for what you learn, you will learn more and be bothered less. R-help is a mixture of different levels and cultures. In framing responses it is hard to know what the other person may find offensive (I was once yelled at and chewed out quite thoroughly for truthfully answering no when asked if I drink coffee). Most responders on this list (actually I would say all, but there might be an exception that I have not noticed) are trying to be helpful, there is just a large variability in the tones of the responses. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
I did not understand enough of the rest of your question to give any better response than others have given. Looking back at your previous posts, there is one suggestion that I can make that may help. You can use the approx or approxfun functions to approximate an inverse, just generate a bunch of x,y pairs from your function, then feed them to approx while switching x and y. Not an exact inverse, but if you give it enough points then it will be close. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Friday, April 30, 2010 5:24 PM To: Greg Snow Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations I have already learned a lot from the list, both technical and not, and cannot thank enough for those valuable suggestions. In fact, as said in my previous posts, I got really critical help and advices, which really addresses the issues I have. By the way, there is one point or two in your post I agree on, but I am not sure why you just pointed out side issues (by snipping a part of my saying) without touching the main topic of this thread at all. I can go on but won't because arguing for the sake of argument is of no value to anyone in this thread. It would have been better if you could have focused on the topic and provided some technical and practical information which I could learn from and be very thankful for. Regards, Joseph On Fri, Apr 30, 2010 at 11:35 PM, Greg Snow greg.s...@imail.org wrote: -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- project.org] On Behalf Of Kyeong Soo (Joseph) Kim Sent: Friday, April 30, 2010 4:10 AM To: kMan Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations [snip] By the way, I wonder why most of the responses I've received from this list are so cynical (or skeptical?) and in some sense done in a quite arrogant way. It's very hard to imagine that one would receive such responses in my own areas of computer simulation and optical communications/networking. If a newbie asks a question to the list not making much sense or another FAQ, that is usually ignored (i.e., no response) because all we are too busy to deal with that. Sometimes, though, a kind soul (like Gabor) takes his/her own valuable time and doesn't mind explaining all the details from simple basics. In my experience with this list, and others, the perceived level of cynical/skeptical/arrogant answers has more to do with the reader than with the writer. If you want to be offended, you will find things to be offended about even when none was intended. If you look for help and useful responses (follow the posting guide) and are thankful for what you learn, you will learn more and be bothered less. R-help is a mixture of different levels and cultures. In framing responses it is hard to know what the other person may find offensive (I was once yelled at and chewed out quite thoroughly for truthfully answering no when asked if I drink coffee). Most responders on this list (actually I would say all, but there might be an exception that I have not noticed) are trying to be helpful, there is just a large variability in the tones of the responses. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
Dear Joseph, If you do not need to make any inferences, that is, you just want it to look pretty, then drawing a curve by hand is as good a solution as any. Plus, there is no reason for expert testimony to say that the curve does not mean anything. Sincerely, KeithC. -Original Message- From: Kyeong Soo (Joseph) Kim [mailto:kyeongsoo@gmail.com] Sent: Tuesday, April 27, 2010 2:33 PM To: Gabor Grothendieck Cc: r-help@r-project.org Subject: Re: [R] Curve Fitting/Regression with Multiple Observations Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication network (like M/M/1 queue, but way too complicated to mathematically analyze it using classical queueing theory) and the conclusion I want to make is qualitative rather than quantatitive -- a high-level comparative study of various network architectures based on the equivalence principle (a concept specific to netwokring, not in the general sense). What l want in this regard is a smooth, non-decreasing (hence one-to-one) function built out of simulation data because later in my processing, I need an inverse function of the said curve to find out an x value given the y value. That was, in fact, the reason I used the exponential (i.e., non-decreasing function) curve fiting. Even though I don't need a statistical inference framework for my work, I want to make sure that my use of regression/curve fitting techniques with my simulation data (as a tool for getting the mentioned curve) is proper and a usual practice among experts like you. To get answer to my question, I digged a lot through the Internet but found no clear explanation so far. Your suggestions and providing examples (always!) are much appreciated, but I am still not sure the use of those regression procedures with the kind of data I described is a right way to do. Again, many thanks for your prompt and kind answers, Joseph On Tue, Apr 27, 2010 at 8:46 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has a book associated with it if you need more info. e.g. library(mgcv) fm - gam(dist ~ s(speed), data = cars) summary(fm) plot(dist ~ speed, cars, pch = 20) fm.ci - with(predict(fm, se = TRUE), cbind(0, -2*se.fit, 2*se.fit) + c(fit)) matlines(cars$speed, fm.ci, lty = c(1, 2, 2), col = c(1, 2, 2)) On Tue, Apr 27, 2010 at 3:07 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: Hello Gabor, Many thanks for providing actual examples for the problem! In fact I know how to apply and generate plots using various R functions including loess, lowess, and smooth.spline procedures. My question, however, is whether applying those procedures directly on the data with multiple observations/duplicate points(?) is on the sound basis or not. Before asking my question to the list, I checked smooth.spline manual pages and found the mentioning of cv option related with duplicate points, but I'm not sure duplicate points in the manual has the same meaning as multiple observations in my case. To me, the manual seems a bit unclear in this regard. Looking at car data, I found it has multiple points with the same speed but different dist, which is exactly what I mean by multiple observations, but am still not sure. Regards, Joseph On Tue, Apr 27, 2010 at 7:35 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: This will compute a loess curve and plot it: example(loess) plot(dist ~ speed, cars, pch = 20) lines(cars$speed, fitted(cars.lo)) Also this directly plots it but does not give you the values of the curve separately: library(lattice) xyplot(dist ~ speed, cars, type = c(p, smooth)) On Tue, Apr 27, 2010 at 1:30 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So
[R] Curve Fitting/Regression with Multiple Observations
I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So I'm thinking right now of directly applying spline (or whatever regression procedures for this purpose) to the simulation data with repetitions rather than means. The simulation data in this case looks like this (assuming three repetitions): # xy 1 1.2 1 0.9 1 1.3 2 2.2 2 1.7 2 2.0 ... So my idea is to let spline procedure handle the fluctuations in the data (i.e., in repetitions) by itself. But I wonder whether this direct application of spline procedures for data with multiple observations makes sense from the statistical analysis (i.e., theoretical) point of view. It may be a stupid question and quite obvious to many, but personally I don't know where to start. It would be greatly appreciated if anyone can shed a light on this in this regard. Many thanks in advance, Joseph __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
Joseph: I believe you need to stop inventing your own statistical methods and consult a professional statistician. I do not think this list is the proper place to look for a statistics tutorial when your statistical background appears to be so inadequate for the task. Sorry to be so direct -- perhaps I am wrong in my assessment. But if I am even close, would you like an accountant to fix your car or an auto mechanic to do your taxes? Cheers, Bert Bert Gunter Genentech Nonclinical Biostatistics -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Kyeong Soo (Joseph) Kim Sent: Tuesday, April 27, 2010 10:31 AM To: r-help@r-project.org Subject: [R] Curve Fitting/Regression with Multiple Observations I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So I'm thinking right now of directly applying spline (or whatever regression procedures for this purpose) to the simulation data with repetitions rather than means. The simulation data in this case looks like this (assuming three repetitions): # xy 1 1.2 1 0.9 1 1.3 2 2.2 2 1.7 2 2.0 ... So my idea is to let spline procedure handle the fluctuations in the data (i.e., in repetitions) by itself. But I wonder whether this direct application of spline procedures for data with multiple observations makes sense from the statistical analysis (i.e., theoretical) point of view. It may be a stupid question and quite obvious to many, but personally I don't know where to start. It would be greatly appreciated if anyone can shed a light on this in this regard. Many thanks in advance, Joseph __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
This will compute a loess curve and plot it: example(loess) plot(dist ~ speed, cars, pch = 20) lines(cars$speed, fitted(cars.lo)) Also this directly plots it but does not give you the values of the curve separately: library(lattice) xyplot(dist ~ speed, cars, type = c(p, smooth)) On Tue, Apr 27, 2010 at 1:30 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So I'm thinking right now of directly applying spline (or whatever regression procedures for this purpose) to the simulation data with repetitions rather than means. The simulation data in this case looks like this (assuming three repetitions): # x y 1 1.2 1 0.9 1 1.3 2 2.2 2 1.7 2 2.0 ... So my idea is to let spline procedure handle the fluctuations in the data (i.e., in repetitions) by itself. But I wonder whether this direct application of spline procedures for data with multiple observations makes sense from the statistical analysis (i.e., theoretical) point of view. It may be a stupid question and quite obvious to many, but personally I don't know where to start. It would be greatly appreciated if anyone can shed a light on this in this regard. Many thanks in advance, Joseph __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
Hello Gabor, Many thanks for providing actual examples for the problem! In fact I know how to apply and generate plots using various R functions including loess, lowess, and smooth.spline procedures. My question, however, is whether applying those procedures directly on the data with multiple observations/duplicate points(?) is on the sound basis or not. Before asking my question to the list, I checked smooth.spline manual pages and found the mentioning of cv option related with duplicate points, but I'm not sure duplicate points in the manual has the same meaning as multiple observations in my case. To me, the manual seems a bit unclear in this regard. Looking at car data, I found it has multiple points with the same speed but different dist, which is exactly what I mean by multiple observations, but am still not sure. Regards, Joseph On Tue, Apr 27, 2010 at 7:35 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: This will compute a loess curve and plot it: example(loess) plot(dist ~ speed, cars, pch = 20) lines(cars$speed, fitted(cars.lo)) Also this directly plots it but does not give you the values of the curve separately: library(lattice) xyplot(dist ~ speed, cars, type = c(p, smooth)) On Tue, Apr 27, 2010 at 1:30 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So I'm thinking right now of directly applying spline (or whatever regression procedures for this purpose) to the simulation data with repetitions rather than means. The simulation data in this case looks like this (assuming three repetitions): # x y 1 1.2 1 0.9 1 1.3 2 2.2 2 1.7 2 2.0 ... So my idea is to let spline procedure handle the fluctuations in the data (i.e., in repetitions) by itself. But I wonder whether this direct application of spline procedures for data with multiple observations makes sense from the statistical analysis (i.e., theoretical) point of view. It may be a stupid question and quite obvious to many, but personally I don't know where to start. It would be greatly appreciated if anyone can shed a light on this in this regard. Many thanks in advance, Joseph __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has a book associated with it if you need more info. e.g. library(mgcv) fm - gam(dist ~ s(speed), data = cars) summary(fm) plot(dist ~ speed, cars, pch = 20) fm.ci - with(predict(fm, se = TRUE), cbind(0, -2*se.fit, 2*se.fit) + c(fit)) matlines(cars$speed, fm.ci, lty = c(1, 2, 2), col = c(1, 2, 2)) On Tue, Apr 27, 2010 at 3:07 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: Hello Gabor, Many thanks for providing actual examples for the problem! In fact I know how to apply and generate plots using various R functions including loess, lowess, and smooth.spline procedures. My question, however, is whether applying those procedures directly on the data with multiple observations/duplicate points(?) is on the sound basis or not. Before asking my question to the list, I checked smooth.spline manual pages and found the mentioning of cv option related with duplicate points, but I'm not sure duplicate points in the manual has the same meaning as multiple observations in my case. To me, the manual seems a bit unclear in this regard. Looking at car data, I found it has multiple points with the same speed but different dist, which is exactly what I mean by multiple observations, but am still not sure. Regards, Joseph On Tue, Apr 27, 2010 at 7:35 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: This will compute a loess curve and plot it: example(loess) plot(dist ~ speed, cars, pch = 20) lines(cars$speed, fitted(cars.lo)) Also this directly plots it but does not give you the values of the curve separately: library(lattice) xyplot(dist ~ speed, cars, type = c(p, smooth)) On Tue, Apr 27, 2010 at 1:30 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So I'm thinking right now of directly applying spline (or whatever regression procedures for this purpose) to the simulation data with repetitions rather than means. The simulation data in this case looks like this (assuming three repetitions): # x y 1 1.2 1 0.9 1 1.3 2 2.2 2 1.7 2 2.0 ... So my idea is to let spline procedure handle the fluctuations in the data (i.e., in repetitions) by itself. But I wonder whether this direct application of spline procedures for data with multiple observations makes sense from the statistical analysis (i.e., theoretical) point of view. It may be a stupid question and quite obvious to many, but personally I don't know where to start. It would be greatly appreciated if anyone can shed a light on this in this regard. Many thanks in advance, Joseph __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve Fitting/Regression with Multiple Observations
Frankly speaking, I am not looking for such a framework. The system I'm studying is a communication network (like M/M/1 queue, but way too complicated to mathematically analyze it using classical queueing theory) and the conclusion I want to make is qualitative rather than quantatitive -- a high-level comparative study of various network architectures based on the equivalence principle (a concept specific to netwokring, not in the general sense). What l want in this regard is a smooth, non-decreasing (hence one-to-one) function built out of simulation data because later in my processing, I need an inverse function of the said curve to find out an x value given the y value. That was, in fact, the reason I used the exponential (i.e., non-decreasing function) curve fiting. Even though I don't need a statistical inference framework for my work, I want to make sure that my use of regression/curve fitting techniques with my simulation data (as a tool for getting the mentioned curve) is proper and a usual practice among experts like you. To get answer to my question, I digged a lot through the Internet but found no clear explanation so far. Your suggestions and providing examples (always!) are much appreciated, but I am still not sure the use of those regression procedures with the kind of data I described is a right way to do. Again, many thanks for your prompt and kind answers, Joseph On Tue, Apr 27, 2010 at 8:46 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If you are looking for a framework for statistical inference you could look at additive models as in the mgcv package which has a book associated with it if you need more info. e.g. library(mgcv) fm - gam(dist ~ s(speed), data = cars) summary(fm) plot(dist ~ speed, cars, pch = 20) fm.ci - with(predict(fm, se = TRUE), cbind(0, -2*se.fit, 2*se.fit) + c(fit)) matlines(cars$speed, fm.ci, lty = c(1, 2, 2), col = c(1, 2, 2)) On Tue, Apr 27, 2010 at 3:07 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: Hello Gabor, Many thanks for providing actual examples for the problem! In fact I know how to apply and generate plots using various R functions including loess, lowess, and smooth.spline procedures. My question, however, is whether applying those procedures directly on the data with multiple observations/duplicate points(?) is on the sound basis or not. Before asking my question to the list, I checked smooth.spline manual pages and found the mentioning of cv option related with duplicate points, but I'm not sure duplicate points in the manual has the same meaning as multiple observations in my case. To me, the manual seems a bit unclear in this regard. Looking at car data, I found it has multiple points with the same speed but different dist, which is exactly what I mean by multiple observations, but am still not sure. Regards, Joseph On Tue, Apr 27, 2010 at 7:35 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: This will compute a loess curve and plot it: example(loess) plot(dist ~ speed, cars, pch = 20) lines(cars$speed, fitted(cars.lo)) Also this directly plots it but does not give you the values of the curve separately: library(lattice) xyplot(dist ~ speed, cars, type = c(p, smooth)) On Tue, Apr 27, 2010 at 1:30 PM, Kyeong Soo (Joseph) Kim kyeongsoo@gmail.com wrote: I recently came to realize the true power of R for statistical analysis -- mainly for post-processing of data from large-scale simulations -- and have been converting many of existing Python(SciPy) scripts to those based on R and/or Perl. In the middle of this conversion, I revisited the problem of curve fitting for simulation data with multiple observations resulting from repetitions. In the past, I first processed simulation data (i.e., multiple y's from repetitions) to get a mean with a confidence interval for a given value of x (independent variable) and then applied spline procedure for those mean values only (i.e., unique pairs of (x_i, y_i) for i=1, 2, ...) to get a smoothed curve. Because of rather large confidence intervals, however, the resulting curves were hardly smooth enough for my purpose, I had to fix the function to exponential and used least square methods to fit its parameters for data. From a plot with confidence intervals, it's rather easy for one to visually and manually(?) figure out a smoothed curve for it. So I'm thinking right now of directly applying spline (or whatever regression procedures for this purpose) to the simulation data with repetitions rather than means. The simulation data in this case looks like this (assuming three repetitions): # x y 1 1.2 1 0.9 1 1.3 2 2.2 2 1.7 2 2.0 ... So my idea is to let spline procedure handle the fluctuations in the data (i.e., in repetitions) by itself. But I wonder whether this direct application of spline procedures for data with multiple observations makes
[R] curve fitting to data
Hi to all This is the first time I am quoting a question and I hope, my question is not too basic... For the following data, I wish to draw a fitted curve. x - c(123,129,141,144,144,145,149,150,158,159,163,174,183,187,242,248) y - c(14.42,26.96,31.3,19.95,36.36,15.4,24.76,35.39,28.07,40.97,26.23,42.83,46.53,14.79,49.18,48.08) If I plot the data, it looks somehow that a logistic function would render good results. My questions are: How do I use nls and/or SSlogis (or other) to fit the curve? How can I see the summary statistics of the fit? How do I finally draw the line to my x,y (untransformed data) plot? Any help would be highly appreciated. Thank you and cheers Pascale -- ..___ Dr. Pascale Weber Swiss Federal Research Institute WSL Zuercherstrasse 111 CH-8903 Birmensdorf Switzerland __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting to data
A simple y vs log(x) fit seems to work pretty well here: fit - lm(y ~ log(x)) summary(fit) plot(y ~ log(x)) abline(fit) On Fri, Dec 4, 2009 at 9:06 AM, Pascale Weber pascale.we...@wsl.ch wrote: Hi to all This is the first time I am quoting a question and I hope, my question is not too basic... For the following data, I wish to draw a fitted curve. x - c(123,129,141,144,144,145,149,150,158,159,163,174,183,187,242,248) y - c(14.42,26.96,31.3,19.95,36.36,15.4,24.76,35.39,28.07,40.97,26.23,42.83,46.53,14.79,49.18,48.08) If I plot the data, it looks somehow that a logistic function would render good results. My questions are: How do I use nls and/or SSlogis (or other) to fit the curve? How can I see the summary statistics of the fit? How do I finally draw the line to my x,y (untransformed data) plot? Any help would be highly appreciated. Thank you and cheers Pascale -- ..___ Dr. Pascale Weber Swiss Federal Research Institute WSL Zuercherstrasse 111 CH-8903 Birmensdorf Switzerland __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting to data
Pascale, If you do want an nls fit with the associated error structure assumptions, check ?SSlogis. fm - nls(y ~ SSlogis(x, Asy, xmid, scal)) summary(fm) xx - seq(123, 248, length = 101) yy - predict(fm, list(x = xx)) plot(x, y) lines(xx, yy) -Peter Ehlers Gabor Grothendieck wrote: A simple y vs log(x) fit seems to work pretty well here: fit - lm(y ~ log(x)) summary(fit) plot(y ~ log(x)) abline(fit) On Fri, Dec 4, 2009 at 9:06 AM, Pascale Weber pascale.we...@wsl.ch wrote: Hi to all This is the first time I am quoting a question and I hope, my question is not too basic... For the following data, I wish to draw a fitted curve. x - c(123,129,141,144,144,145,149,150,158,159,163,174,183,187,242,248) y - c(14.42,26.96,31.3,19.95,36.36,15.4,24.76,35.39,28.07,40.97,26.23,42.83,46.53,14.79,49.18,48.08) If I plot the data, it looks somehow that a logistic function would render good results. My questions are: How do I use nls and/or SSlogis (or other) to fit the curve? How can I see the summary statistics of the fit? How do I finally draw the line to my x,y (untransformed data) plot? Any help would be highly appreciated. Thank you and cheers Pascale -- ..___ Dr. Pascale Weber Swiss Federal Research Institute WSL Zuercherstrasse 111 CH-8903 Birmensdorf Switzerland __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] curve fitting
I have the data: for x: 2, 8, 14, 20, 26, 32, 38, 44, 50, 56, 62, 68, 74, for y: 100, 99, 99, 98, 97, 94, 82, 66, 48, 38, 22, 10, 1. y depends on x by equation: y = 100*exp(b*(1-exp(c*x))/c), where b and c are coefficients. I need to find coefficients in this equation for given data. How can I do that by means of R? Thank you for advance. With regard, Dmitry. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting
Dear Dmitry, Take a look at ?nls and its examples. HTH, Jorge On Tue, May 12, 2009 at 5:44 PM, Dmitry Gospodaryov gospodar...@rambler.ruwrote: I have the data: for x: 2, 8, 14, 20, 26, 32, 38, 44, 50, 56, 62, 68, 74, for y: 100, 99, 99, 98, 97, 94, 82, 66, 48, 38, 22, 10, 1. y depends on x by equation: y = 100*exp(b*(1-exp(c*x))/c), where b and c are coefficients. I need to find coefficients in this equation for given data. How can I do that by means of R? Thank you for advance. With regard, Dmitry. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting
or use nls.lm as in install.packages(minpack.lm) library(minpack.lm) x - c(2, 8, 14, 20, 26, 32, 38, 44, 50, 56, 62, 68, 74) y - c(100, 99, 99, 98, 97, 94, 82, 66, 48, 38, 22, 10, 1) res - function(p, x, y) y - ff(p,x) ff - function(p, x) 100*exp(p[1]*(1-exp(p[2]*x))/p[2]) aa - nls.lm(par=c(.0001,.0001), fn=res, x=x, y=y) plot(x,y) lines(x, ff(coef(aa), x)) On Tue, 12 May 2009, Jorge Ivan Velez wrote: Dear Dmitry, Take a look at ?nls and its examples. HTH, Jorge On Tue, May 12, 2009 at 5:44 PM, Dmitry Gospodaryov gospodar...@rambler.ruwrote: I have the data: for x: 2, 8, 14, 20, 26, 32, 38, 44, 50, 56, 62, 68, 74, for y: 100, 99, 99, 98, 97, 94, 82, 66, 48, 38, 22, 10, 1. y depends on x by equation: y = 100*exp(b*(1-exp(c*x))/c), where b and c are coefficients. I need to find coefficients in this equation for given data. How can I do that by means of R? Thank you for advance. With regard, Dmitry. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting,FDA for biological data
Hi Thanks a lot, I think you have covered the things I want to do for now so I will try to implement them as soon I can. A finite Fourier series could be the best tool IF the the multiple periodicities are all integer fractions of a common scale. This is certainly true for my repetitive unit (the smallest data peak) so I hope this makes things easier. Thanks spencerg wrote: Dear Dr Gkikopoulos: 1. Have you looked at bioconductor.org? They have substantive extensions to R specifically for genomic data, which I assume would include chromosome. 2. To identify periodicities at different timescales, I agree with Stephen that spectrum would likely help. 3. The best software to fit data into discrete number of curves depends on the particular discrete number of curves you want to consider and how you want to fit data into them. A finite Fourier series could be the best tool IF the the multiple periodicities are all integer fractions of a common scale. In that case, using a fourier base in the fda package could be your method of choice. Otherwise, you might consider Bayesian Model Averaging. RSiteSearch(Bayesian Model Averaging) produced 80 hits for me just now, and RSiteSearch(Bayesian Model Averaging, function) produced 60. RSiteSearch.function in the RSiteSearch package [available via install.packages(RSiteSearch,repos=http://r-forge.r-project.org;)] told me that 27 of the 60 were in the ensembleBMA package, and another 14 were in the BMA package. 4. The best way to compare data from different experiments depends on your evaluation of 3 above. The fda package includes an fRegress function that might be useful. Hope this helps. Spencer Graves trias wrote: There are a couple of different goals for this projects *identify periodicities at different timescales (ie different dT) *fit data into discrete number of curves, ie 6 different basic functions should be enough to describe the basic repeating elements in this data (ie 6 different categories of peaks) *comapre data from different experiments of the same time reference (in my case this is location on chromosome) for changes in the underlying basic elements (ie changes of the basic funtions,periodicity etc) I think if I can find a strategy to answer some of these question I be in a good position to explore this data analysis further if needed. Thanks a lot stephen sefick wrote: What is your end goal? If it is to try and account for the variability of the timeseries you may want to look at ?spectrum If it is to model the periodicity... Stephen Sefick On Fri, Apr 3, 2009 at 11:30 AM, trias t.gkikopou...@dundee.ac.uk wrote: Here is the gif that didn't come through earlier http://www.nabble.com/file/p22870832/signal.gif signal.gif -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p22870832.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Stephen Sefick Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p23024818.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting,FDA for biological data
Dear Dr Gkikopoulos: 1. Have you looked at bioconductor.org? They have substantive extensions to R specifically for genomic data, which I assume would include chromosome. 2. To identify periodicities at different timescales, I agree with Stephen that spectrum would likely help. 3. The best software to fit data into discrete number of curves depends on the particular discrete number of curves you want to consider and how you want to fit data into them. A finite Fourier series could be the best tool IF the the multiple periodicities are all integer fractions of a common scale. In that case, using a fourier base in the fda package could be your method of choice. Otherwise, you might consider Bayesian Model Averaging. RSiteSearch(Bayesian Model Averaging) produced 80 hits for me just now, and RSiteSearch(Bayesian Model Averaging, function) produced 60. RSiteSearch.function in the RSiteSearch package [available via install.packages(RSiteSearch,repos=http://r-forge.r-project.org;)] told me that 27 of the 60 were in the ensembleBMA package, and another 14 were in the BMA package. 4. The best way to compare data from different experiments depends on your evaluation of 3 above. The fda package includes an fRegress function that might be useful. Hope this helps. Spencer Graves trias wrote: There are a couple of different goals for this projects *identify periodicities at different timescales (ie different dT) *fit data into discrete number of curves, ie 6 different basic functions should be enough to describe the basic repeating elements in this data (ie 6 different categories of peaks) *comapre data from different experiments of the same time reference (in my case this is location on chromosome) for changes in the underlying basic elements (ie changes of the basic funtions,periodicity etc) I think if I can find a strategy to answer some of these question I be in a good position to explore this data analysis further if needed. Thanks a lot stephen sefick wrote: What is your end goal? If it is to try and account for the variability of the timeseries you may want to look at ?spectrum If it is to model the periodicity... Stephen Sefick On Fri, Apr 3, 2009 at 11:30 AM, trias t.gkikopou...@dundee.ac.uk wrote: Here is the gif that didn't come through earlier http://www.nabble.com/file/p22870832/signal.gif signal.gif -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p22870832.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Stephen Sefick Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting,FDA for biological data
There are a couple of different goals for this projects *identify periodicities at different timescales (ie different dT) *fit data into discrete number of curves, ie 6 different basic functions should be enough to describe the basic repeating elements in this data (ie 6 different categories of peaks) *comapre data from different experiments of the same time reference (in my case this is location on chromosome) for changes in the underlying basic elements (ie changes of the basic funtions,periodicity etc) I think if I can find a strategy to answer some of these question I be in a good position to explore this data analysis further if needed. Thanks a lot stephen sefick wrote: What is your end goal? If it is to try and account for the variability of the timeseries you may want to look at ?spectrum If it is to model the periodicity... Stephen Sefick On Fri, Apr 3, 2009 at 11:30 AM, trias t.gkikopou...@dundee.ac.uk wrote: Here is the gif that didn't come through earlier http://www.nabble.com/file/p22870832/signal.gif signal.gif -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p22870832.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Stephen Sefick Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p22906065.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Curve fitting,FDA for biological data
Dear all, Another newbie just got attracted to this mailing list. I am a biologist currently working my way through R, had sort play around with python earlier this year. I have some data exhibiting periodicity ** my data consists of peaks and valleys, with peaks arising due to the presence of a repetitive structural unit,** with x being a reference grid (position along a chromosome) and y being strength of signal (this y signal fluctuates to give rise to the peaks and valleys). ie presence of a structural unit along a chromosome gives rise to a peak in my data. I would like to use a curve fitting algorithm (I guess something like a fourier analysis and/or splines). Due to the nature of the data I would like to look for periodicities at different scales (along the x grid). So say 2-4 different splines/curves are probably enough to describe the 40,000 occourences of the repetitive structural unit in my data, while say 4-6 of these units could exhibit certain patterns in the way they group together. I assume in my case, I can consider my x axis (position) to be equivalent to a time x axis as in signal processing. I considered using the FDA package (silverman and ramsey I think). Does anyone have an ideas if this is the right way to go or suggestions etc PS I have highlighted in the attached gif with red, the occourence of the repetitive signal (differences in the wavelength for example could be important but not more than 4 would be required to fit all data), and in yellow a hypothetical occourence of a periodicity in a different scale Thanks a lot Dr Triantafyllos Gkikopoulos The University of Dundee is a registered Scottish charity, No: SC015096 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting,FDA for biological data
Here is the gif that didn't come through earlier http://www.nabble.com/file/p22870832/signal.gif signal.gif -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p22870832.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Curve fitting,FDA for biological data
What is your end goal? If it is to try and account for the variability of the timeseries you may want to look at ?spectrum If it is to model the periodicity... Stephen Sefick On Fri, Apr 3, 2009 at 11:30 AM, trias t.gkikopou...@dundee.ac.uk wrote: Here is the gif that didn't come through earlier http://www.nabble.com/file/p22870832/signal.gif signal.gif -- View this message in context: http://www.nabble.com/Curve-fitting%2CFDA-for-biological-data-tp22868069p22870832.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Stephen Sefick Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] curve fitting with given term
hello, I want to fit a curve to a simple x,y dataset - my problem is, that I want to fit it for the following term: n(1-e^x/y) - so I get the n constant for my data... can anyone help/comment on that? cheers, gregor __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting with given term
gregor rolshausen wrote: hello, I want to fit a curve to a simple x,y dataset - my problem is, that I want to fit it for the following term: n(1-e^x/y) - so I get the n constant for my data... Not an R problem in the first place, but the question arises what n(1-e^x/y) means, its is just some scalar value so far. I am looking for some equation ... Uwe Ligges can anyone help/comment on that? cheers, gregor __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting with given term
ok. sorry for being blurry. I have x,y data, that probably fits a asymptotic curve (asymptote at N). now I want to fit a curve onto the data, that gives me the N. therefore I thought to fit an e-function, namely N(1-e^(y/x)) onto the data and get the N from the fitted curves' equation. in the course of this, I was looking for a R-function to fit a given function to data. (I believe there is some implementation in MatLab for this kind of question, anyhow, I wanted to look in R as well...) I am not an expert, so excuse my misuse of terms. I hope my problem graspable...? cheers, gregor Uwe Ligges wrote: gregor rolshausen wrote: hello, I want to fit a curve to a simple x,y dataset - my problem is, that I want to fit it for the following term: n(1-e^x/y) - so I get the n constant for my data... Not an R problem in the first place, but the question arises what n(1-e^x/y) means, its is just some scalar value so far. I am looking for some equation ... Uwe Ligges can anyone help/comment on that? cheers, gregor __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting with given term
gregor rolshausen gregor.rolshausen at biologie.uni-freiburg.de writes: ok. sorry for being blurry. I have x,y data, that probably fits a asymptotic curve (asymptote at N). now I want to fit a curve onto the data, that gives me the N. therefore I thought to fit an e-function, namely N(1-e^(y/x)) onto the data and get the N from the fitted curves' equation. in the course of this, I was looking for a R-function to fit a given function to data. (I believe there is some implementation in MatLab for this kind of question, anyhow, I wanted to look in R as well...) I am not an expert, so excuse my misuse of terms. I hope my problem graspable...? Perhaps you mean y = N*(1-exp(-x/a)) assuming x is the predictor and y the response variable. (This further assumes that the errors are independent, normally distributed [if you want to make inferences on the parameters etc.], etc..) Check out ?nls (for nonlinear least squares). good luck, Ben Bolker __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] curve fitting problem
I'm trying to fit a function y=k*l^(m*x) to some data points, with reasonable starting value estimates (I think). I keep getting singular matrix 'a' in solve. This is the code: ox - c(-600,-300,-200,1,100,200) ir - c(1,2.5,4,9,14,20) model - nls(ir ~ k*l^(m*ox),start=list(k=10,l=3,m=0.004),algorithm=plinear) summary(model) plot(ox,ir) testox - seq(-600,200,length=100) k - 10 l - 3 m - 0.004 testir - k*l^(m*testox) lines(testox,testir) Any thoughts? Thanks! -- This message was sent on behalf of [EMAIL PROTECTED] at openSubscriber.com http://www.opensubscriber.com/messages/[EMAIL PROTECTED]/topic.html __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] curve fitting problem
Your model is singular. Varying m and log(l) have the same effect: log(ir) = log(k) + m * log(l) * ox Also with plinear you don't specify the linear coefficients but rather an X matrix whose coefficients represent them: If we use this model instead: ir = k * exp(m * ox) Then: mod0 - lm(log(ir) ~ ox) mod0 Call: lm(formula = log(ir) ~ ox) Coefficients: (Intercept) ox 2.199743 0.003835 nls(ir ~ exp(m * ox), start = list(m = coef(mod0)[2]), algorithm = plinear) Nonlinear regression model model: ir ~ exp(m * ox) data: parent.frame() m .lin 0.003991 9.091758 residual sum-of-squares: 0.3551 Number of iterations to convergence: 3 Achieved convergence tolerance: 5.289e-07 On Dec 24, 2007 9:04 AM, [EMAIL PROTECTED] wrote: I'm trying to fit a function y=k*l^(m*x) to some data points, with reasonable starting value estimates (I think). I keep getting singular matrix 'a' in solve. This is the code: ox - c(-600,-300,-200,1,100,200) ir - c(1,2.5,4,9,14,20) model - nls(ir ~ k*l^(m*ox),start=list(k=10,l=3,m=0.004),algorithm=plinear) summary(model) plot(ox,ir) testox - seq(-600,200,length=100) k - 10 l - 3 m - 0.004 testir - k*l^(m*testox) lines(testox,testir) Any thoughts? Thanks! -- This message was sent on behalf of [EMAIL PROTECTED] at openSubscriber.com http://www.opensubscriber.com/messages/[EMAIL PROTECTED]/topic.html __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
