Hi Ken & Kyun-Seop, I agree it should be taught, since it is prevalent in the industry, and should be looked at as something to investigate further, but no hard and fast rule should be applied to if the model is reasonable and fit for purpose. That should be done in conjunction with other diagnostic plots.
One thing that has always bothered me about the condition number is that it is calculated based on the final parameter estimates, but not the scaled parameter estimates. Truly the scaling is supposed to help make the gradient on a comparable scale and fix many numerical problems here. Hence, if the scaling works as it is supposed to, small changes may not affect the colinearity as strongly as the calculated condition number suggests. This is mainly why I see it as a number to keep in mind instead of a hard and fast rule. Matt On Tue, Nov 29, 2022 at 5:09 PM Ken Kowalski <kgkowalsk...@gmail.com> wrote: > Hi Kyun-Seop, > > > > I would state things a little differently rather than say “devalue > condition number and multi-collinearity” we should treat CN as a diagnostic > and rules such as CN>1000 should NOT be used as a hard and fast rule to > reject a model. I agree with Jeroen that we should understand the > implications of a high CN and the impact multi-collinearity may have on the > model estimation and that there are other diagnostics such as correlations, > variance inflation factors (VIF), standard errors, CIs, etc. that can also > help with our understanding of the effects of multi-collinearity and its > implications for model development. > > > > That being said, if you have a model with a high CN and the model > converges with realistic point estimates and reasonable standard errors > then it may still be reasonable to accept that model. However, in this > setting I would probably still want to re-run the model with different > starting values and make sure it converges to the same OFV and set of point > estimates. > > > > As the smallest eigenvalue goes to 0 and the CN goes to infinity we end up > with a singular Hessian matrix (R matrix) so we know that at some point a > high enough CN will result in convergence and COV step failures. Thus, you > shouldn’t simply dismiss CN as not having any diagnostic value, just don’t > apply it in a rule such as CN>1000 to blindly reject a model. The CN>1000 > rule should only be used to call your attention to the potential for an > issue that warrants further investigation before accepting the model or > deciding how to alter the model to improve stability in the estimation. > > > > Best, > > > > Ken > > > > Kenneth G. Kowalski > > Kowalski PMetrics Consulting, LLC > > Email: kgkowalsk...@gmail.com > > Cell: 248-207-5082 > > > > > > > > *From:* owner-nmus...@globomaxnm.com [mailto:owner-nmus...@globomaxnm.com] > *On Behalf Of *Kyun-Seop Bae > *Sent:* Tuesday, November 29, 2022 5:10 PM > *To:* nmusers@globomaxnm.com > *Subject:* Fwd: [NMusers] Condition numbera > > > > Dear All, > > > > I would like to devalue condition number and multi-collinearity in > nonlinear regression. > > The reason we consider condition number (or multi-collinearity) is that > this may cause the following fitting (estimation) problems; > > 1. Fitting failure (fail to converge, fail to minimize) > 2. Unrealistic point estimates > 3. Too wide standard errors > > > > If you do not see the above problems (i.e., no estimation problem with > modest standard error), you do not need to give attention to the condition > number. > > > > I think I saw 10^(n – parameters) criterion in an old version of > Gabrielsson’s book many years ago (but not in the latest version). > > > > Best regards, > > > > Kyun-Seop Bae > > > > On Tue, 29 Nov 2022 at 22:59, Ayyappa Chaturvedula <ayyapp...@gmail.com> > wrote: > > Dear all, > I am wondering if someone can provide references for the condition number > thresholds we are seeing (<1000) etc. Also, the other way I have seen when > I was in graduate school that condition number <10^n (n- number of > parameters) is OK. Personally, I am depending on correlation matrix rather > than condition number and have seen cases where condition number is large > (according to 1000 rule but less than 10^n rule) but correlation matrix is > fine. > > I want to provide these for my teaching purposes and any help is greatly > appreciated. > > Regards, > Ayyappa > > > > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> > Virus-free.www.avast.com > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> > <#m_-1496757161961929954_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >
