Once again many thanks, Alexios!
It was not clear to me that the solvers are contained in separate
packages with separate manuals.
I still have got three questions
QUESTION 1: I`ve experimented a bit with the outer.iter and inner.iter
values. Can you recommend any minimal values for them? Should they be
kept in a certain relationship when being modified (default is
outer.iter=400 and inner.iter=800)?
QUESTION 2: Moreover, there seems to be NO direct relationship between
lowering the values and shortening the estimation time (until abortion).
Is this as expected or is it a bug?
E.g. holding outer.iter fixed and lowering the inner.iter parameter from
130 to 115 decreases time until "no convergence"-message, but lowering
it from 130 to only 120 strongly INCREASES time (actually I`m not sure
if the solver in this case would finish at all one time or if it`s
somehow stuck).
I tried 3 times. The results are:
#################
require('rugarch')
ptm <- proc.time()
spec <- ugarchspec(
variance.model =
list(model = "eGARCH", garchOrder = c(0,7), submodel = NULL
, external.regressors = NULL, variance.targeting = FALSE),
mean.model = list(armaOrder = c(0,0), external.regressors = NULL),
distribution.model = "sged")
tempgarch <- ugarchfit(spec=spec, data=tempdata , solver="solnp" ,
solver.control=list( outer.iter = 70, inner.iter=130 ) )
tempgarch
(proc.time()-ptm)[3]
# trial 1:
# outer.iter = 70, inner.iter=115 elapsed 79.87 ugarchfit-->warning:
solver FAILER to converge. (there`s a typo in the error message)
# outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
# outer.iter = 70, inner.iter=130 elapsed 135.8 ugarchfit-->warning:
solver failer to converge.
# trial 2:
# outer.iter = 70, inner.iter=115 elapsed 79.2 ugarchfit-->warning:
solver failer to converge.
# outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
# outer.iter = 70, inner.iter=130 elapsed 135.09 ugarchfit-->warning:
solver failer to converge.
# trial 3:
# outer.iter = 70, inner.iter=115 elapsed 79.3 ugarchfit-->warning:
solver failer to converge.
# outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
# outer.iter = 70, inner.iter=130 elapsed 136.13 ugarchfit-->warning:
solver failer to converge.
#################
QUESTION 3: I`m a bit confused about the parallel computing
implementation in ugarchfit.
In the vignette it says (on p. 44) "Since version 1.0-14, rugarch makes
exclusive use of the parallel package for all parallel computations."
But in the manual on p. 70 only the packages "snowfall" or "multicore"
seem to be supported for the solver used in ugarchfit.
So either EXCLUSIVELY parallel or not?
As I could not find any example about how to set up snowfall in this
context I simply tried this way (without speed improvement):
##################
require('rugarch')
require('snowfall')
spec <- ugarchspec(
variance.model=list( model = "eGARCH", garchOrder=c(1,3),
submodel=NULL, external.regressors=NULL, variance.targeting=FALSE ),
mean.model=list( armaOrder=c(0,0), external.regressors=NULL ),
distribution.model = "std")
ptm <- proc.time()
tempgarch <- ugarchfit( spec=spec, data=tempdata ,solver="solnp" ,
solver.control=list( parallel=TRUE, pkg="snowfall", cores=4 ) )
(proc.time()-ptm)[3]
tempgarch
##################
What am I doing wrong? Do I have to wrap the ugarchfit function into
something like the following?
##################
sfInit( parallel=TRUE, cpus=4 )
...
sfStop()
##################
(I`d expect a "no" as an answer assuming that the solver itself will do
something alike if the relevant solver.control options are set)
Best, Johannes
Am 10.05.2014 13:25, schrieb alexios ghalanos:
Hi Johannes,
On 10 May 2014, at 12:19, Johannes Moser <jzmo...@gmail.com> wrote:
Many thanks Alexios!!
1. In my TGARCH setup nlminb doesn`t converge even at smaller GARCH order. So I
will stick to solnp then.
2. Unfortunately I couldn`t find the correct command which limits the number of
iterations via the solver.control options.
The details from the manual mention n.sim and n.restarts, but these seem to
control other parameters.
For the nloptr solver the option maxeval is mentioned. But I don`t work with
this solver and trial-and-error-implementation of this option to sonnp leaded
to no success.
Other packages inspired me to try "maxiter" , "iter.max" , "n.iter" , but they
didn`t work either.
E.g. ugarchfit( spec=spec, data=tempdata , solver="solnp",
solver.control=list( maxeval=20, rseed=9876 ) )
See the documentation for solnp (?solnp). The ‘inner.iter’ and ‘outer.iter’ are
what you are likely looking for. The n.sim and n.restarts if for the gosolnp
solver (multi-start solnp).
3. You`re surely right. The whole study should actually investigate this issue
empirically.
E.g. in one case there was a surprising result in a sample of size 1200:
An ARMA(0,0) eGARCH(5,5) model with a skewed normal for the innovations yielded
very good results.
No sign biases, nice gof, no autocorrelation in standardized and squared
standardized residuals up to order p+q+10, nice AIC and BIC as well as only
highly significant coefficients (6 out of 18 were not significant as to the
robust SE, though). I will compare this model to a more parsimonious one and
also investigate parameter uncertainty.
Yes, in-sample of course….but I was referring to forecast performance (out of
sample).
Best, Johannes
Regards,
Alexios
Am 10.05.2014 11:34, schrieb alexios ghalanos:
Johannes,
I suggest the following:
1. Don't use hybrid, use instead solnp or nlminb.
2. You can control a number of solver convergence criteria (e.g. number
of iterations) using the solver.control argument.
3. Before running the code, do consider a little more how reasonable it
is to be modelling a TGARCH(7,8) model. Investigate the model first
(don't just return the AIC or BIC). Are any of the higher order
ARCH/GARCH parameters different from zero or even significant? I have
not seen a single study which shows that such very high order GARCH
models have better performance than more parsimonious alternatives.
4. At the best of times it takes a considerable amount of data to
estimate the GARCH persistence. Try running a simulation exercise using
for example the ugarchdistribution function to obtain some insight into
higher order GARCH models.
5. Finally, as mentioned numerous times on this forum, the fGARCH model
is a highly parameterized omnibus model. Imposing stationarity during
the optimization, particularly for non-symmetric distributions such as
the ged, is a costly exercise. Consider using the GJR instead and a
distribution which is a little faster to evaluate such as the JSU.
Alternatively consider using the normal distribution to estimate the
GARCH parameters for the purpose of model comparison.
-Alexios
On 10/05/2014 08:23, Johannes Moser wrote:
I guess that the problem is due to the processing in C as part of the
ugarchfit routine.
Is there any way to timeout a ugarchfit command or to constrain the
number if iterations?
At one time the loop seems to be stuck completely.
I waited for several hours for a single ugarchfit step which just didn`t
complete. Then I manually stopped the process.
Separate calculation of the respective model also seems to be "stuck"
(CPU is still working, the "hybrid" algorithms seem to find no solution
though and just keep running).
As I want to set up a GARCH model-preselection battery there hopefully
is a way to handle such problems?
Best, Johannes
Am 09.05.2014 13:58, schrieb Johannes Moser:
Dear all,
I`ve set up a double loop which loops through different GARCH-orders
and ARMA-orders in a rugarch estimation (of several models and error
distributions) and each time writes the AIC and other information into
a data frame.
The resulting data frame should be used for the pre-selection of a
model, which then will be examined manually.
A small part of the model estimation steps using "ugarchfit" take very
long time. So I implemented a timeout function using "evalWithTimeout"
which stops the current estimation step and proceeds with the next
step in the loop and estimates the next model.
The timeout function is wrapped into a "tryCatch" function which
assures thet the loop keeps running after e.g. convergence problems.
A small toy model works fine:
#######################################################################
require('R.utils')
abc <- matrix(NA,10,3)
foo <- function() {
print("Tic");
for (kk in 1:50) {
print(kk);
Sys.sleep(0.1);
}
print("Tac");
}
for (i in 1:10){
ptm <- proc.time()
tryCatch( { abc[i,1] <- evalWithTimeout({foo()} ,timeout=(4+i*0.2)
,onTimeout="silent" )
abc[i,2] <- 1
}
, error = function(x) x)
tt<- proc.time() - ptm
abc[i,3]<-tt[3]
}
abc
#####################################################################
However, in the rugarch setup the "evalWithTimeout" doesn't seem to
stop the "ugarchfit" estimation reliably. E.g. in one instance the
recorded time for a step was 1388.03 seconds even though the limit was
set to be 300 seconds. The next example illustrates my setup in a
simplified version (unfortunately my results depend on the data I have
used, so you will not be able to reproduce them):
#####################################################################
require('rugarch')
quiet1 <- read.table( "dax_quiet1.txt" , header=T)
tempdata <- quiet1$logreturns
g_order <- matrix(NA,5,2)
g_order[1,]<-c(1,1)
g_order[2,]<-c(1,8)
g_order[3,]<-c(9,6)
g_order[4,]<-c(9,8)
g_order[5,]<-c(3,10)
overview <- data.frame(matrix(NA,5,2))
for(i in 1:5){
ptm <- proc.time()
spec <- ugarchspec(
variance.model = list(model = "fGARCH", garchOrder =
g_order[i,], submodel = "TGARCH", external.regressors = NULL,
variance.targeting = FALSE),
mean.model = list(armaOrder = c(0,0), external.regressors =
NULL), distribution.model = "sged")
tryCatch( {tempgarch <- evalWithTimeout({ugarchfit(spec=spec,
data=tempdata ,solver="hybrid")} ,timeout=20 ,onTimeout="silent" )
overview[i,1]<-infocriteria(tempgarch)[1]
}
, error = function(x) x)
tt<- proc.time() - ptm
overview[i,2]<-tt[3]
}
overview
# If the timeout is set set to 20, this setup leads to:
# 2.87 sec.
# 6.95 sec.
# 125 sec. ... here, tryCatch interrupted the process
# 51.73 sec.
# 27.11 sec.
# for the 5 different estimation steps.
# timeout set to 300:
# 2.81 sec.
# 6.85 sec.
# 743.58 sec.
# 41.70 sec.
# 26.85 sec.
# no process was interrupted by tryCatch
#######################################################################
As can be seen even from this simplified example, when the timeout was
set to be 20 there still was a process that took 125 seconds (which is
more than 5 times longer!).
I would be very thankful for any ideas or comments!
Best, Johannes
--
_______________________________________________
R-SIG-Finance@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions
should go.
--
--
_______________________________________________
R-SIG-Finance@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should
go.
