Re: [R] Optim() and Instability
Since some questioned the scaling idea, here are runs first with
scaling and then without scaling. Note how much better the solution
is in the first run (see arrows). It is also evident from the data
> head(data, 3)
y x1 x2 x3
1 0.660 20 7.0 1680
2 0.165 5 1.7 350
3 0.660 20 7.0 1680
> # run 1 - scaling
> str(optim(par = c(1,1, 1), min.perc_error, data = data,
+ control = list(parscale = c(1, 1, 0.0001
List of 5
$ par: num [1:3] 0.030232 0.024411 -0.000113
$ value : num 0.653 <=
$ counts : Named int [1:2] 180 NA
..- attr(*, "names")= chr [1:2] "function" "gradient"
$ convergence: int 0
$ message: NULL
> # run 2 - no scaling
> str(optim(par = c(1,1, 1), min.perc_error, data = data))
List of 5
$ par: num [1:3] 0.6305 -0.1247 -0.0032
$ value : num 473 <=
$ counts : Named int [1:2] 182 NA
..- attr(*, "names")= chr [1:2] "function" "gradient"
$ convergence: int 0
$ message: NULL
On Sat, Nov 14, 2015 at 10:32 AM, Gabor Grothendieck
wrote:
> I meant the parscale parameter.
>
> On Sat, Nov 14, 2015 at 10:30 AM, Gabor Grothendieck
> wrote:
>> Tyipcally the parameters being optimized should be the same order of
>> magnitude or else you can expect numerical problems. That is what the
>> fnscale control parameter is for.
>>
>> On Sat, Nov 14, 2015 at 10:15 AM, Lorenzo Isella
>> wrote:
>>> Dear All,
>>> I am using optim() for a relatively simple task: a linear model where
>>> instead of minimizing the sum of the squared errors, I minimize the sum
>>> of the squared relative errors.
>>> However, I notice that the default algorithm is very sensitive to the
>>> choice of the initial fit parameters, whereas I get much more stable
>>> (and therefore better?) results with the BFGS algorithm.
>>> I would like to have some feedback on this (perhaps I made a mistake
>>> somewhere).
>>> I provide a small self-contained example.
>>> You can download a tiny data set from the link
>>>
>>> https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
>>>
>>> whereas I paste the script I am using at the end of the email.
>>> Any feedback is really appreciated.
>>> Many thanks
>>>
>>> Lorenzo
>>>
>>>
>>>
>>> min.perc_error <- function(data, par) {
>>> with(data, sum(((par[1]*x1 + par[2]*x2+par[3]*x3 -
>>> y)/y)^2))
>>>}
>>>
>>> par_ini1 <- c(.3,.1, 1e-3)
>>>
>>> par_ini2 <- c(1,1, 1)
>>>
>>>
>>> data <- read.csv("data-instability.csv")
>>>
>>> mm_def1 <-optim(par = par_ini1
>>>, min.perc_error, data = data)
>>>
>>> mm_bfgs1 <-optim(par = par_ini1
>>>, min.perc_error, data = data, method="BFGS")
>>>
>>> print("fit parameters with the default algorithms and the first seed
>>> ")
>>> print(mm_def1$par)
>>>
>>> print("fit parameters with the BFGS algorithms and the first seed ")
>>> print(mm_bfgs1$par)
>>>
>>>
>>>
>>> mm_def2 <-optim(par = par_ini2
>>>, min.perc_error, data = data)
>>>
>>> mm_bfgs2 <-optim(par = par_ini2
>>>, min.perc_error, data = data, method="BFGS")
>>>
>>>
>>>
>>>
>>> print("fit parameters with the default algorithms and the second seed
>>> ")
>>> print(mm_def2$par)
>>>
>>> print("fit parameters with the BFGS algorithms and the second seed ")
>>> print(mm_bfgs2$par)
>>>
>>> __
>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>>
>> --
>> Statistics & Software Consulting
>> GKX Group, GKX Associates Inc.
>> tel: 1-877-GKX-GROUP
>> email: ggrothendieck at gmail.com
>
>
>
> --
> Statistics & Software Consulting
> GKX Group, GKX Associates Inc.
> tel: 1-877-GKX-GROUP
> email: ggrothendieck at gmail.com
--
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Optim() and Instability
> On 14 Nov 2015, at 17:02, Berend Hasselman wrote:
>
>>
>> On 14 Nov 2015, at 16:15, Lorenzo Isella wrote:
>>
>> Dear All,
>> I am using optim() for a relatively simple task: a linear model where
>> instead of minimizing the sum of the squared errors, I minimize the sum
>> of the squared relative errors.
>> However, I notice that the default algorithm is very sensitive to the
>> choice of the initial fit parameters, whereas I get much more stable
>> (and therefore better?) results with the BFGS algorithm.
>> I would like to have some feedback on this (perhaps I made a mistake
>> somewhere).
>> I provide a small self-contained example.
>> You can download a tiny data set from the link
>>
>> https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
>>
>> whereas I paste the script I am using at the end of the email.
>> Any feedback is really appreciated.
>> Many thanks
>>
>
> The initial parameter values for the percentage error variant are not very
> good.
> If you print min.perc_error(data,par_ini2) you can see that.
>
> Try
>
> par_ini2 <- c(1e-4,1e-4,1e-4)
>
> and you'll get results that are closer to each other.
> The rest is up to you.
Try this at the end of your script:
# Original
min.perc_error(data,par_ini2)
# Much better
par_ini3 <- c(1e-4,1e-4,1e-4)
min.perc_error(data,par_ini3)
mm_def3 <-optim(par = par_ini3
, min.perc_error, data = data)
mm_bfgs3 <-optim(par = par_ini3
, min.perc_error, data = data, method="BFGS")
print("fit parameters with the default algorithms and the second seed
")
print(mm_def3$par)
min.perc_error(data,mm_def3$par)
print("fit parameters with the BFGS algorithms and the second seed ")
print(mm_bfgs3$par)
min.perc_error(data,mm_bfgs3$par)
and rejoice!
Berend
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Optim() and Instability
> On 14 Nov 2015, at 16:15, Lorenzo Isella wrote:
>
> Dear All,
> I am using optim() for a relatively simple task: a linear model where
> instead of minimizing the sum of the squared errors, I minimize the sum
> of the squared relative errors.
> However, I notice that the default algorithm is very sensitive to the
> choice of the initial fit parameters, whereas I get much more stable
> (and therefore better?) results with the BFGS algorithm.
> I would like to have some feedback on this (perhaps I made a mistake
> somewhere).
> I provide a small self-contained example.
> You can download a tiny data set from the link
>
> https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
>
> whereas I paste the script I am using at the end of the email.
> Any feedback is really appreciated.
> Many thanks
>
The initial parameter values for the percentage error variant are not very good.
If you print min.perc_error(data,par_ini2) you can see that.
Try
par_ini2 <- c(1e-4,1e-4,1e-4)
and you'll get results that are closer to each other.
The rest is up to you.
Berend
> Lorenzo
>
>
>
> min.perc_error <- function(data, par) {
> with(data, sum(((par[1]*x1 + par[2]*x2+par[3]*x3 -
> y)/y)^2))
> }
>
> par_ini1 <- c(.3,.1, 1e-3)
>
> par_ini2 <- c(1,1, 1)
>
>
> data <- read.csv("data-instability.csv")
>
> mm_def1 <-optim(par = par_ini1
> , min.perc_error, data = data)
>
> mm_bfgs1 <-optim(par = par_ini1
> , min.perc_error, data = data, method="BFGS")
>
> print("fit parameters with the default algorithms and the first seed
> ")
> print(mm_def1$par)
>
> print("fit parameters with the BFGS algorithms and the first seed ")
> print(mm_bfgs1$par)
>
>
>
> mm_def2 <-optim(par = par_ini2
> , min.perc_error, data = data)
>
> mm_bfgs2 <-optim(par = par_ini2
> , min.perc_error, data = data, method="BFGS")
>
>
>
>
> print("fit parameters with the default algorithms and the second seed
> ")
> print(mm_def2$par)
>
> print("fit parameters with the BFGS algorithms and the second seed ")
> print(mm_bfgs2$par)
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Optim() and Instability
I meant the parscale parameter.
On Sat, Nov 14, 2015 at 10:30 AM, Gabor Grothendieck
wrote:
> Tyipcally the parameters being optimized should be the same order of
> magnitude or else you can expect numerical problems. That is what the
> fnscale control parameter is for.
>
> On Sat, Nov 14, 2015 at 10:15 AM, Lorenzo Isella
> wrote:
>> Dear All,
>> I am using optim() for a relatively simple task: a linear model where
>> instead of minimizing the sum of the squared errors, I minimize the sum
>> of the squared relative errors.
>> However, I notice that the default algorithm is very sensitive to the
>> choice of the initial fit parameters, whereas I get much more stable
>> (and therefore better?) results with the BFGS algorithm.
>> I would like to have some feedback on this (perhaps I made a mistake
>> somewhere).
>> I provide a small self-contained example.
>> You can download a tiny data set from the link
>>
>> https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
>>
>> whereas I paste the script I am using at the end of the email.
>> Any feedback is really appreciated.
>> Many thanks
>>
>> Lorenzo
>>
>>
>>
>> min.perc_error <- function(data, par) {
>> with(data, sum(((par[1]*x1 + par[2]*x2+par[3]*x3 -
>> y)/y)^2))
>>}
>>
>> par_ini1 <- c(.3,.1, 1e-3)
>>
>> par_ini2 <- c(1,1, 1)
>>
>>
>> data <- read.csv("data-instability.csv")
>>
>> mm_def1 <-optim(par = par_ini1
>>, min.perc_error, data = data)
>>
>> mm_bfgs1 <-optim(par = par_ini1
>>, min.perc_error, data = data, method="BFGS")
>>
>> print("fit parameters with the default algorithms and the first seed
>> ")
>> print(mm_def1$par)
>>
>> print("fit parameters with the BFGS algorithms and the first seed ")
>> print(mm_bfgs1$par)
>>
>>
>>
>> mm_def2 <-optim(par = par_ini2
>>, min.perc_error, data = data)
>>
>> mm_bfgs2 <-optim(par = par_ini2
>>, min.perc_error, data = data, method="BFGS")
>>
>>
>>
>>
>> print("fit parameters with the default algorithms and the second seed
>> ")
>> print(mm_def2$par)
>>
>> print("fit parameters with the BFGS algorithms and the second seed ")
>> print(mm_bfgs2$par)
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>
>
> --
> Statistics & Software Consulting
> GKX Group, GKX Associates Inc.
> tel: 1-877-GKX-GROUP
> email: ggrothendieck at gmail.com
--
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Optim() and Instability
Tyipcally the parameters being optimized should be the same order of
magnitude or else you can expect numerical problems. That is what the
fnscale control parameter is for.
On Sat, Nov 14, 2015 at 10:15 AM, Lorenzo Isella
wrote:
> Dear All,
> I am using optim() for a relatively simple task: a linear model where
> instead of minimizing the sum of the squared errors, I minimize the sum
> of the squared relative errors.
> However, I notice that the default algorithm is very sensitive to the
> choice of the initial fit parameters, whereas I get much more stable
> (and therefore better?) results with the BFGS algorithm.
> I would like to have some feedback on this (perhaps I made a mistake
> somewhere).
> I provide a small self-contained example.
> You can download a tiny data set from the link
>
> https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
>
> whereas I paste the script I am using at the end of the email.
> Any feedback is really appreciated.
> Many thanks
>
> Lorenzo
>
>
>
> min.perc_error <- function(data, par) {
> with(data, sum(((par[1]*x1 + par[2]*x2+par[3]*x3 -
> y)/y)^2))
>}
>
> par_ini1 <- c(.3,.1, 1e-3)
>
> par_ini2 <- c(1,1, 1)
>
>
> data <- read.csv("data-instability.csv")
>
> mm_def1 <-optim(par = par_ini1
>, min.perc_error, data = data)
>
> mm_bfgs1 <-optim(par = par_ini1
>, min.perc_error, data = data, method="BFGS")
>
> print("fit parameters with the default algorithms and the first seed
> ")
> print(mm_def1$par)
>
> print("fit parameters with the BFGS algorithms and the first seed ")
> print(mm_bfgs1$par)
>
>
>
> mm_def2 <-optim(par = par_ini2
>, min.perc_error, data = data)
>
> mm_bfgs2 <-optim(par = par_ini2
>, min.perc_error, data = data, method="BFGS")
>
>
>
>
> print("fit parameters with the default algorithms and the second seed
> ")
> print(mm_def2$par)
>
> print("fit parameters with the BFGS algorithms and the second seed ")
> print(mm_bfgs2$par)
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
--
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Optim() and Instability
Dear All,
I am using optim() for a relatively simple task: a linear model where
instead of minimizing the sum of the squared errors, I minimize the sum
of the squared relative errors.
However, I notice that the default algorithm is very sensitive to the
choice of the initial fit parameters, whereas I get much more stable
(and therefore better?) results with the BFGS algorithm.
I would like to have some feedback on this (perhaps I made a mistake
somewhere).
I provide a small self-contained example.
You can download a tiny data set from the link
https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
whereas I paste the script I am using at the end of the email.
Any feedback is really appreciated.
Many thanks
Lorenzo
min.perc_error <- function(data, par) {
with(data, sum(((par[1]*x1 + par[2]*x2+par[3]*x3 -
y)/y)^2))
}
par_ini1 <- c(.3,.1, 1e-3)
par_ini2 <- c(1,1, 1)
data <- read.csv("data-instability.csv")
mm_def1 <-optim(par = par_ini1
, min.perc_error, data = data)
mm_bfgs1 <-optim(par = par_ini1
, min.perc_error, data = data, method="BFGS")
print("fit parameters with the default algorithms and the first seed
")
print(mm_def1$par)
print("fit parameters with the BFGS algorithms and the first seed ")
print(mm_bfgs1$par)
mm_def2 <-optim(par = par_ini2
, min.perc_error, data = data)
mm_bfgs2 <-optim(par = par_ini2
, min.perc_error, data = data, method="BFGS")
print("fit parameters with the default algorithms and the second seed
")
print(mm_def2$par)
print("fit parameters with the BFGS algorithms and the second seed ")
print(mm_bfgs2$par)
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
