[R] Can I monitor the iterative/convergence process while using Optim or MaxLik?

2010-09-14 Thread Sally Luo 
Hi R-helpers,

Is it possible that I have the estimates from each step/iteration shown on
the computer screen in order to monitor the process while I am using Optim
or MaxLik?

Thanks for your help.

Maomao

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?

2010-09-14 Thread Greg Snow 
The simple way to do this is just to rewrite your function to be optimized to 
print out the current values that it was called with, this way you will see 
where it is and what it is doing.  Look at the cat function for how to print 
the values, also look at the flush.console function.

For example you could take the 1st example on the optim help page and insert 
the following 2 lines as the 1st 2 lines of the f function:

cat(x, '\n')
flush.console()

and then at each iteration you can see the values tried.

-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.s...@imail.org
801.408.8111


 -Original Message-
 From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-
 project.org] On Behalf Of Sally Luo
 Sent: Tuesday, September 14, 2010 1:50 PM
 To: r-help@r-project.org
 Subject: [R] Can I monitor the iterative/convergence process while
 using Optim or MaxLik?
 
 Hi R-helpers,
 
 Is it possible that I have the estimates from each step/iteration shown
 on
 the computer screen in order to monitor the process while I am using
 Optim
 or MaxLik?
 
 Thanks for your help.
 
 Maomao
 
   [[alternative HTML version deleted]]
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-
 guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?

2010-09-14 Thread Arne Henningsen 
On 14 September 2010 21:49, Sally Luo shali...@gmail.com wrote:

 Is it possible that I have the estimates from each step/iteration shown on
 the computer screen in order to monitor the process while I am using Optim
 or MaxLik?


You can call maxLik() with argument print.level set to a value larger than
0.

/Arne

-- 
Arne Henningsen
http://www.arne-henningsen.name

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?

2010-09-14 Thread Ravi Varadhan 
This is generally not the best way to do it for the following reasons.   

The objective function may be evaluated multiple times within a single 
iteration, for various reasons including (i) gradient evaluation when analytic 
gradient is not specified, and (ii) during line-search stretgy for steplength 
determination.  Therefore, embedding a `cat' or `print' statement inside the 
objective function may generate a lot of noise.  You can run your suggested 
`optim' example to verify this!

A better approach is to embed the `cat' statement inside the gradient function, 
in addition to using the `trace' and `REPORT' arguments, which are integers in 
`optim'.  Here is the `optim' example that you suggested:

fr - function(x) { ## Rosenbrock Banana function 
x1 - x[1] 
x2 - x[2] 
100 * (x2 - x1 * x1)^2 + (1 - x1)^2 
} 

grr.print - function(x) { ## Gradient of 'fr' 
cat(pars: , x, '\n')
flush.console()
x1 - x[1] 
x2 - x[2] 
c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) 
} 

 optim(c(-1.2,1), fr, grr.print, method = BFGS, control=list(trace=1, 
 REPORT=1)) 

This prints out both parameter values and the function value at each iteration.

This approach cannot, however, be used in `optim' when analytic gradient is not 
available (or for derivative free methods, e.g. Nelder-Mead).  


Hope this helps,
Ravi.



Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University

Ph. (410) 502-2619
email: rvarad...@jhmi.edu


- Original Message -
From: Greg Snow greg.s...@imail.org
Date: Tuesday, September 14, 2010 4:27 pm
Subject: Re: [R] Can I monitor the iterative/convergence process while  using 
Optim or MaxLik?
To: Sally Luo shali...@gmail.com, r-help@r-project.org 
r-help@r-project.org


 The simple way to do this is just to rewrite your function to be 
 optimized to print out the current values that it was called with, 
 this way you will see where it is and what it is doing.  Look at the 
 cat function for how to print the values, also look at the 
 flush.console function.
  
  For example you could take the 1st example on the optim help page and 
 insert the following 2 lines as the 1st 2 lines of the f function:
  
  cat(x, '\n')
  flush.console()
  
  and then at each iteration you can see the values tried.
  
  -- 
  Gregory (Greg) L. Snow Ph.D.
  Statistical Data Center
  Intermountain Healthcare
  greg.s...@imail.org
  801.408.8111
  
  
   -Original Message-
   From: r-help-boun...@r-project.org [
   project.org] On Behalf Of Sally Luo
   Sent: Tuesday, September 14, 2010 1:50 PM
   To: r-help@r-project.org
   Subject: [R] Can I monitor the iterative/convergence process while
   using Optim or MaxLik?
   
   Hi R-helpers,
   
   Is it possible that I have the estimates from each step/iteration shown
   on
   the computer screen in order to monitor the process while I am using
   Optim
   or MaxLik?
   
   Thanks for your help.
   
   Maomao
   
  [[alternative HTML version deleted]]
   
   __
   R-help@r-project.org mailing list
   
   PLEASE do read the posting guide 
   guide.html
   and provide commented, minimal, self-contained, reproducible code.
  
  __
  R-help@r-project.org mailing list
  
  PLEASE do read the posting guide 
  and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?

2010-09-14 Thread William Dunlap 
You can also save the objective (or gradient) function
arguments and values in datasets so you can later plot
or print them.  E.g., the following lets you avoid hand
editing the objective function to do this.  If you wanted
to track the gradient, replace th c(VALUE,val) with
rbind(VALUE,val).  (Either would be ok with the objective.)

trackFn - function (fn) {
# return function like fn that stashes its
# inputs and outputs in local datasets.
X - NULL
VALUE - NULL
force(fn)
function(x) {
X - rbind(X, x) # stash arguments
val - fn(x)
VALUE - c(VALUE, val) # stash value
val
}
}

Typical usage would be

 fr - function(x) { ## Rosenbrock Banana function 
+ x1 - x[1] 
+ x2 - x[2] 
+ 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 
+ } 
 o - optim(c(-1.2, 1), tfr - trackFn(fr))
 with(environment(tfr), matplot(X, VALUE, type=l))
 with(environment(tfr), plot(X, type=l))

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com  

 -Original Message-
 From: r-help-boun...@r-project.org 
 [mailto:r-help-boun...@r-project.org] On Behalf Of Ravi Varadhan
 Sent: Tuesday, September 14, 2010 2:19 PM
 To: Greg Snow
 Cc: r-help@r-project.org
 Subject: Re: [R] Can I monitor the iterative/convergence 
 process while using Optim or MaxLik?
 
 This is generally not the best way to do it for the following 
 reasons.   
 
 The objective function may be evaluated multiple times within 
 a single iteration, for various reasons including (i) 
 gradient evaluation when analytic gradient is not specified, 
 and (ii) during line-search stretgy for steplength 
 determination.  Therefore, embedding a `cat' or `print' 
 statement inside the objective function may generate a lot of 
 noise.  You can run your suggested `optim' example to verify this!
 
 A better approach is to embed the `cat' statement inside the 
 gradient function, in addition to using the `trace' and 
 `REPORT' arguments, which are integers in `optim'.  Here is 
 the `optim' example that you suggested:
 
 fr - function(x) { ## Rosenbrock Banana function 
 x1 - x[1] 
 x2 - x[2] 
 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 
 } 
 
 grr.print - function(x) { ## Gradient of 'fr' 
 cat(pars: , x, '\n')
 flush.console()
 x1 - x[1] 
 x2 - x[2] 
 c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) 
 } 
 
  optim(c(-1.2,1), fr, grr.print, method = BFGS, 
 control=list(trace=1, REPORT=1)) 
 
 This prints out both parameter values and the function value 
 at each iteration.
 
 This approach cannot, however, be used in `optim' when 
 analytic gradient is not available (or for derivative free 
 methods, e.g. Nelder-Mead).  
 
 
 Hope this helps,
 Ravi.
 
 
 
 Ravi Varadhan, Ph.D.
 Assistant Professor,
 Division of Geriatric Medicine and Gerontology
 School of Medicine
 Johns Hopkins University
 
 Ph. (410) 502-2619
 email: rvarad...@jhmi.edu
 
 
 - Original Message -
 From: Greg Snow greg.s...@imail.org
 Date: Tuesday, September 14, 2010 4:27 pm
 Subject: Re: [R] Can I monitor the iterative/convergence 
 process while using Optim or MaxLik?
 To: Sally Luo shali...@gmail.com, r-help@r-project.org 
 r-help@r-project.org
 
 
  The simple way to do this is just to rewrite your function to be 
  optimized to print out the current values that it was called with, 
  this way you will see where it is and what it is doing.  
 Look at the 
  cat function for how to print the values, also look at the 
  flush.console function.
   
   For example you could take the 1st example on the optim 
 help page and 
  insert the following 2 lines as the 1st 2 lines of the f function:
   
   cat(x, '\n')
   flush.console()
   
   and then at each iteration you can see the values tried.
   
   -- 
   Gregory (Greg) L. Snow Ph.D.
   Statistical Data Center
   Intermountain Healthcare
   greg.s...@imail.org
   801.408.8111
   
   
-Original Message-
From: r-help-boun...@r-project.org [
project.org] On Behalf Of Sally Luo
Sent: Tuesday, September 14, 2010 1:50 PM
To: r-help@r-project.org
Subject: [R] Can I monitor the iterative/convergence 
 process while
using Optim or MaxLik?

Hi R-helpers,

Is it possible that I have the estimates from each 
 step/iteration shown
on
the computer screen in order to monitor the process 
 while I am using
Optim
or MaxLik?

Thanks for your help.

Maomao

 [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list

PLEASE do read the posting guide 
guide.html
and provide commented, minimal, self-contained, 
 reproducible code.
   
   __
   R-help@r-project.org mailing list
   
   PLEASE do read the posting guide 
   and provide commented, minimal, self-contained, reproducible code

Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?

2010-09-14 Thread Ravi Varadhan 
Bill,

I am not able to get this to work for tracking inside the gradient function:

trackFn - function (fn) {
# return function like fn that stashes its
# inputs and outputs in local datasets.
X - NULL
VALUE - NULL
force(fn)
function(x) {
X - rbind(X, x) # stash arguments
val - fn(x)
VALUE - rbind(VALUE, val) # stash value
val
}
}

 fr - function(x) { ## Rosenbrock Banana function 
 x1 - x[1] 
x2 - x[2] 
100 * (x2 - x1 * x1)^2 + (1 - x1)^2 
} 

grr - function(x) { ## Gradient of 'fr' 
x1 - x[1] 
x2 - x[2] 
c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) 
} 

o - optim(c(-1.2, 1), fr, tfr - trackFn(grr))
#with(environment(tfr), matplot(X, VALUE, type=l))
with(environment(tfr), plot(X, type=l))

What is wrong here?

Ravi.



Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University

Ph. (410) 502-2619
email: rvarad...@jhmi.edu


- Original Message -
From: William Dunlap wdun...@tibco.com
Date: Tuesday, September 14, 2010 5:38 pm
Subject: RE: [R] Can I monitor the iterative/convergence process while  using 
Optim or MaxLik?
To: Ravi Varadhan rvarad...@jhmi.edu, Greg Snow greg.s...@imail.org
Cc: r-help@r-project.org


 You can also save the objective (or gradient) function
  arguments and values in datasets so you can later plot
  or print them.  E.g., the following lets you avoid hand
  editing the objective function to do this.  If you wanted
  to track the gradient, replace th c(VALUE,val) with
  rbind(VALUE,val).  (Either would be ok with the objective.)
  
  trackFn - function (fn) {
  # return function like fn that stashes its
  # inputs and outputs in local datasets.
  X - NULL
  VALUE - NULL
  force(fn)
  function(x) {
  X - rbind(X, x) # stash arguments
  val - fn(x)
  VALUE - c(VALUE, val) # stash value
  val
  }
  }
  
  Typical usage would be
  
   fr - function(x) { ## Rosenbrock Banana function 
  + x1 - x[1] 
  + x2 - x[2] 
  + 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 
  + } 
   o - optim(c(-1.2, 1), tfr - trackFn(fr))
   with(environment(tfr), matplot(X, VALUE, type=l))
   with(environment(tfr), plot(X, type=l))
  
  Bill Dunlap
  Spotfire, TIBCO Software
  wdunlap tibco.com  
  
   -Original Message-
   From: r-help-boun...@r-project.org 
   [ On Behalf Of Ravi Varadhan
   Sent: Tuesday, September 14, 2010 2:19 PM
   To: Greg Snow
   Cc: r-help@r-project.org
   Subject: Re: [R] Can I monitor the iterative/convergence 
   process while using Optim or MaxLik?
   
   This is generally not the best way to do it for the following 
   reasons.   
   
   The objective function may be evaluated multiple times within 
   a single iteration, for various reasons including (i) 
   gradient evaluation when analytic gradient is not specified, 
   and (ii) during line-search stretgy for steplength 
   determination.  Therefore, embedding a `cat' or `print' 
   statement inside the objective function may generate a lot of 
   noise.  You can run your suggested `optim' example to verify this!
   
   A better approach is to embed the `cat' statement inside the 
   gradient function, in addition to using the `trace' and 
   `REPORT' arguments, which are integers in `optim'.  Here is 
   the `optim' example that you suggested:
   
   fr - function(x) { ## Rosenbrock Banana function 
   x1 - x[1] 
   x2 - x[2] 
   100 * (x2 - x1 * x1)^2 + (1 - x1)^2 
   } 
   
   grr.print - function(x) { ## Gradient of 'fr' 
   cat(pars: , x, '\n')
   flush.console()
   x1 - x[1] 
   x2 - x[2] 
   c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) 
 
   } 
   
optim(c(-1.2,1), fr, grr.print, method = BFGS, 
   control=list(trace=1, REPORT=1)) 
   
   This prints out both parameter values and the function value 
   at each iteration.
   
   This approach cannot, however, be used in `optim' when 
   analytic gradient is not available (or for derivative free 
   methods, e.g. Nelder-Mead).  
   
   
   Hope this helps,
   Ravi.
   
   
   
   Ravi Varadhan, Ph.D.
   Assistant Professor,
   Division of Geriatric Medicine and Gerontology
   School of Medicine
   Johns Hopkins University
   
   Ph. (410) 502-2619
   email: rvarad...@jhmi.edu
   
   
   - Original Message -
   From: Greg Snow greg.s...@imail.org
   Date: Tuesday, September 14, 2010 4:27 pm
   Subject: Re: [R] Can I monitor the iterative/convergence 
   process while  using Optim or MaxLik?
   To: Sally Luo shali...@gmail.com, r-help@r-project.org 
   r-help@r-project.org
   
   
The simple way to do this is just to rewrite your function to be 
 
optimized to print out the current values that it was called 
 with, 
this way you will see where it is and what it is doing.  
   Look at the 
cat function for how to print