Re: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
On 16/01/2013, at 13:50 PM, Duarte Viana wrote:
> Hi Jesse and Gavin,
>
> I remember that post and I think that turning the argument LINPACK of
> the function "svd" on solved the problem. I did so by:
>
> 1. pasting the code of function svd (that can be found in the source code
> of the base package) on the console
> 2. changing the argument LINPACK=FALSE to LINPACK=TRUE
> 3. re-assigning the function to the base package with the code:
> assignInNamespace("svd", svd, "base")
>
> However I do not know whether this is the best solution. Perhaps Gavin
> can find out what the real problem is. I would appreciate if you both
> could keep me informed about any progress.
We have had this problem sometimes in svd in vegan functions (rda, cca), and
then LINPACK = TRUE has helped. However, this is not a good solution as LINPACK
is phasing out from R. In the current development version (and probably in the
next R release later this year), LINPACK is defunct, and this solution won't
work.
It seems that the basic problem is in the svd function of the LAPACK that is
used as the default now and that will be the only choice later this year.
LAPACK is an external library that is beyond the control of package developers
and R core team so that these problems may be unsolvable. It seems that the
problems with the LAPACK code are so common that even the help page of svd()
function tells about them:
Unsuccessful results from the underlying LAPACK code will result
in an error giving a positive error code (most often ‘1’): these
can only be interpreted by detailed study of the FORTRAN code but
mean that the algorithm failed to converge.
Sometimes rescaling of the data (= multiplying with a suitable constant) has
helped, too, and that may be the only thing we can try once LINPACK = TRUE
option disappears within a few months.
Best wishes, Jari Oksanen
>
> On Wed, Jan 16, 2013 at 11:35 AM, Gavin Simpson
> wrote:
>> Hi Jesse,
>>
>> Can you send me the data *and* *exact* code you used so I can look into
>> this further? I promise to delete the data once I have gotten to the
>> bottom of the problem.
>>
>> If you can, please do so *off list*. If you can't then it might help to
>> scale the numbers a bit as range of 5 orders of magnitude may be causing
>> some numerical issues with your data.
>>
>> Note this has nothing to do with vegan; cocorresp is a separate package.
>>
>> Re the last question; it is possible and IIRC there is some Matlab code
>> to do some of this in the supplementary materials for the ter braak &
>> schaffers paper. I got some way to implementing this in R but finishing
>> it off went to the back burner and I never get back to it since.
>>
>> All the best,
>>
>> Gavin
>>
>> On Tue, 2013-01-15 at 19:09 -0800, Jesse_B wrote:
>>> Question 1 - It's been a while, so I don't know who will see this, but I am
>>> having the same issue. I have count data from two species matrices (fish
>>> and inverts) and I am trying to run CoCA through cocorresp. Symmetric CoCA
>>> works fine, and is the main thing that I need, but I would like to be able
>>> to switch predictor-response species matrices in a predictive CoCA, to see
>>> if there are differing patterns of top-down/bottom-up concordance. I have
>>> substantial skew in the data, so I have log+1 transformed both sets of data.
>>> I can run crossval on the raw data (not transformed, 99 samples [33 sites
>>> over 3 seasons], 72 fish species, 226 invert species, individual numbers are
>>> in the same ranges, between 0 and 10K for both fish and inverts), but on the
>>> transformed data, I get the "Error in La.svd(x, nu, nv) : error code 1 from
>>> Lapack routine 'dgesdd' " message consistently on the 5th site. I am
>>> comfortable _using_ R and the vegan package in particular, but I am not
>>> experienced in more deep coding, so I don't have a handle on how to turn
>>> LINPACK on. R version 2.15.2, vegan 2.0-4, cocorresp 0.2-0
>>>
crossval(fishlog,invertlog)
>>> LOO - Site: 1 - Complete
>>> LOO - Site: 2 - Complete
>>> LOO - Site: 3 - Complete
>>> LOO - Site: 4 - Complete
>>> LOO - Site: 5Error in La.svd(x, nu, nv) : error code 1 from Lapack routine
>>> 'dgesdd'
>>>
>>> Question 2 - Is it possible to run crossval on matricies for a CCA? to make
>>> it a PLS-CCA (as in Schaffers et al. 2008) or am I misunderstanding the
>>> process that they used?
>>>
>>> Thanks in advance!
>>>
>>> Jesse
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://r-sig-ecology.471788.n2.nabble.com/Error-in-La-svd-x-nu-nv-error-code-1-from-Lapack-routine-dgesdd-tp7556369p7577802.html
>>> Sent from the r-sig-ecology mailing list archive at Nabble.com.
>>>
>>> ___
>>> R-sig-ecology mailing list
>>> R-sig-ecology@r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
>>>
>>
>> --
>> %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
>> Dr. G
Re: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
Hi Jesse and Gavin,
I remember that post and I think that turning the argument LINPACK of
the function "svd" on solved the problem. I did so by:
1. pasting the code of function svd (that can be found in the source code
of the base package) on the console
2. changing the argument LINPACK=FALSE to LINPACK=TRUE
3. re-assigning the function to the base package with the code:
assignInNamespace("svd", svd, "base")
However I do not know whether this is the best solution. Perhaps Gavin
can find out what the real problem is. I would appreciate if you both
could keep me informed about any progress.
Kind regards,
Duarte
On Wed, Jan 16, 2013 at 11:35 AM, Gavin Simpson wrote:
> Hi Jesse,
>
> Can you send me the data *and* *exact* code you used so I can look into
> this further? I promise to delete the data once I have gotten to the
> bottom of the problem.
>
> If you can, please do so *off list*. If you can't then it might help to
> scale the numbers a bit as range of 5 orders of magnitude may be causing
> some numerical issues with your data.
>
> Note this has nothing to do with vegan; cocorresp is a separate package.
>
> Re the last question; it is possible and IIRC there is some Matlab code
> to do some of this in the supplementary materials for the ter braak &
> schaffers paper. I got some way to implementing this in R but finishing
> it off went to the back burner and I never get back to it since.
>
> All the best,
>
> Gavin
>
> On Tue, 2013-01-15 at 19:09 -0800, Jesse_B wrote:
>> Question 1 - It's been a while, so I don't know who will see this, but I am
>> having the same issue. I have count data from two species matrices (fish
>> and inverts) and I am trying to run CoCA through cocorresp. Symmetric CoCA
>> works fine, and is the main thing that I need, but I would like to be able
>> to switch predictor-response species matrices in a predictive CoCA, to see
>> if there are differing patterns of top-down/bottom-up concordance. I have
>> substantial skew in the data, so I have log+1 transformed both sets of data.
>> I can run crossval on the raw data (not transformed, 99 samples [33 sites
>> over 3 seasons], 72 fish species, 226 invert species, individual numbers are
>> in the same ranges, between 0 and 10K for both fish and inverts), but on the
>> transformed data, I get the "Error in La.svd(x, nu, nv) : error code 1 from
>> Lapack routine 'dgesdd' " message consistently on the 5th site. I am
>> comfortable _using_ R and the vegan package in particular, but I am not
>> experienced in more deep coding, so I don't have a handle on how to turn
>> LINPACK on. R version 2.15.2, vegan 2.0-4, cocorresp 0.2-0
>>
>> > crossval(fishlog,invertlog)
>> LOO - Site: 1 - Complete
>> LOO - Site: 2 - Complete
>> LOO - Site: 3 - Complete
>> LOO - Site: 4 - Complete
>> LOO - Site: 5Error in La.svd(x, nu, nv) : error code 1 from Lapack routine
>> 'dgesdd'
>>
>> Question 2 - Is it possible to run crossval on matricies for a CCA? to make
>> it a PLS-CCA (as in Schaffers et al. 2008) or am I misunderstanding the
>> process that they used?
>>
>> Thanks in advance!
>>
>> Jesse
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://r-sig-ecology.471788.n2.nabble.com/Error-in-La-svd-x-nu-nv-error-code-1-from-Lapack-routine-dgesdd-tp7556369p7577802.html
>> Sent from the r-sig-ecology mailing list archive at Nabble.com.
>>
>> ___
>> R-sig-ecology mailing list
>> R-sig-ecology@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
>>
>
> --
> %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
> Dr. Gavin Simpson [t] +44 (0)20 7679 0522
> ECRC, UCL Geography, [f] +44 (0)20 7679 0565
> Pearson Building, [e] gavin.simpsonATNOSPAMucl.ac.uk
> Gower Street, London [w] http://www.ucl.ac.uk/~ucfagls/
> UK. WC1E 6BT. [w] http://www.freshwaters.org.uk
> %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
>
> ___
> R-sig-ecology mailing list
> R-sig-ecology@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
___
R-sig-ecology mailing list
R-sig-ecology@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
Hi Jesse, Can you send me the data *and* *exact* code you used so I can look into this further? I promise to delete the data once I have gotten to the bottom of the problem. If you can, please do so *off list*. If you can't then it might help to scale the numbers a bit as range of 5 orders of magnitude may be causing some numerical issues with your data. Note this has nothing to do with vegan; cocorresp is a separate package. Re the last question; it is possible and IIRC there is some Matlab code to do some of this in the supplementary materials for the ter braak & schaffers paper. I got some way to implementing this in R but finishing it off went to the back burner and I never get back to it since. All the best, Gavin On Tue, 2013-01-15 at 19:09 -0800, Jesse_B wrote: > Question 1 - It's been a while, so I don't know who will see this, but I am > having the same issue. I have count data from two species matrices (fish > and inverts) and I am trying to run CoCA through cocorresp. Symmetric CoCA > works fine, and is the main thing that I need, but I would like to be able > to switch predictor-response species matrices in a predictive CoCA, to see > if there are differing patterns of top-down/bottom-up concordance. I have > substantial skew in the data, so I have log+1 transformed both sets of data. > I can run crossval on the raw data (not transformed, 99 samples [33 sites > over 3 seasons], 72 fish species, 226 invert species, individual numbers are > in the same ranges, between 0 and 10K for both fish and inverts), but on the > transformed data, I get the "Error in La.svd(x, nu, nv) : error code 1 from > Lapack routine 'dgesdd' " message consistently on the 5th site. I am > comfortable _using_ R and the vegan package in particular, but I am not > experienced in more deep coding, so I don't have a handle on how to turn > LINPACK on. R version 2.15.2, vegan 2.0-4, cocorresp 0.2-0 > > > crossval(fishlog,invertlog) > LOO - Site: 1 - Complete > LOO - Site: 2 - Complete > LOO - Site: 3 - Complete > LOO - Site: 4 - Complete > LOO - Site: 5Error in La.svd(x, nu, nv) : error code 1 from Lapack routine > 'dgesdd' > > Question 2 - Is it possible to run crossval on matricies for a CCA? to make > it a PLS-CCA (as in Schaffers et al. 2008) or am I misunderstanding the > process that they used? > > Thanks in advance! > > Jesse > > > > > -- > View this message in context: > http://r-sig-ecology.471788.n2.nabble.com/Error-in-La-svd-x-nu-nv-error-code-1-from-Lapack-routine-dgesdd-tp7556369p7577802.html > Sent from the r-sig-ecology mailing list archive at Nabble.com. > > ___ > R-sig-ecology mailing list > R-sig-ecology@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-ecology > -- %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% Dr. Gavin Simpson [t] +44 (0)20 7679 0522 ECRC, UCL Geography, [f] +44 (0)20 7679 0565 Pearson Building, [e] gavin.simpsonATNOSPAMucl.ac.uk Gower Street, London [w] http://www.ucl.ac.uk/~ucfagls/ UK. WC1E 6BT. [w] http://www.freshwaters.org.uk %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
Question 1 - It's been a while, so I don't know who will see this, but I am having the same issue. I have count data from two species matrices (fish and inverts) and I am trying to run CoCA through cocorresp. Symmetric CoCA works fine, and is the main thing that I need, but I would like to be able to switch predictor-response species matrices in a predictive CoCA, to see if there are differing patterns of top-down/bottom-up concordance. I have substantial skew in the data, so I have log+1 transformed both sets of data. I can run crossval on the raw data (not transformed, 99 samples [33 sites over 3 seasons], 72 fish species, 226 invert species, individual numbers are in the same ranges, between 0 and 10K for both fish and inverts), but on the transformed data, I get the "Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd' " message consistently on the 5th site. I am comfortable _using_ R and the vegan package in particular, but I am not experienced in more deep coding, so I don't have a handle on how to turn LINPACK on. R version 2.15.2, vegan 2.0-4, cocorresp 0.2-0 > crossval(fishlog,invertlog) LOO - Site: 1 - Complete LOO - Site: 2 - Complete LOO - Site: 3 - Complete LOO - Site: 4 - Complete LOO - Site: 5Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd' Question 2 - Is it possible to run crossval on matricies for a CCA? to make it a PLS-CCA (as in Schaffers et al. 2008) or am I misunderstanding the process that they used? Thanks in advance! Jesse -- View this message in context: http://r-sig-ecology.471788.n2.nabble.com/Error-in-La-svd-x-nu-nv-error-code-1-from-Lapack-routine-dgesdd-tp7556369p7577802.html Sent from the r-sig-ecology mailing list archive at Nabble.com. ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
Thanks Jari for your help.
Changing the option of the svd function to LIMPACK=TRUE indeed
resolved the problem.
I just copy-paste the function svd to the console (maintaining the
name svd), changed LIMPACK=FALSE to LIMPACK=TRUE, and then assigned
the function to the namespace "base":
assignInNamespace("svd", svd, "base")
It worked!
Thanks again,
Duarte
On Mon, May 14, 2012 at 3:25 PM, Jari Oksanen wrote:
> We have had similar reports from many cases (for instance, from cca/rda
> functions in vegan). The problem is difficult to track. Cases that we have
> seen so far probably are problems in LAPACK library, and the kind of LAPACK
> used can depend on the version of R, version of LAPACK, compilers used,
> operating system and its version, computer hardware, phase of moon etc. Error
> code 1 from dgesdd should be a convergence problem.
>
> With presence/absence data there may not be much you can with the data. With
> quantitative data, rescaling of data has been helpful, but hardly with p/a
> (Hellinger transformation will also change data properties and is not data
> neutral: division of subtraction with some constant is data neutral and helps
> sometimes, but hardly with p/a data).
>
> The svd() function in R provides a R<1.7.0 compatibility option LINPACK =
> TRUE which has helped sometimes. We haven't added that as an option for svd()
> in cocorresp or vegan as it is documented only as a compatibility option for
> old R. However, you can try editing the files to use LINPACK = TRUE to see
> if the problem vanishes (but NAMESPACE add some pain in using edited files).
>
> HTH, Jari Oksanen
>
> From: r-sig-ecology-boun...@r-project.org
> [r-sig-ecology-boun...@r-project.org] on behalf of Duarte Viana
> [viana.s...@gmail.com]
> Sent: 14 May 2012 16:09
> To: r-sig-ecology@r-project.org
> Subject: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from
> Lapack routine 'dgesdd'
>
> Hello all,
>
> When I perform a cross validation with the function "crossval" of the
> package cocorresp (for a co-correspondence analysis), I get the
> following error:
>
> Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
>
> I think the problem is not in the function, as other people has had
> the same error for other types of model (see, for example,
> http://comments.gmane.org/gmane.comp.lang.r.ecology/2608). From this
> and other posts, I suppose it is a convergence problem.
>
> I am working with presence-absence species matrices, and the problem
> persists even if I transform the matrices with Hellinger
> transformation (although not at the same site).
>
> Does anyone know how to solve, or go around, this problem?
>
> Any advice would be most welcome.
> Thanks,
>
> Duarte
>
> ___
> R-sig-ecology mailing list
> R-sig-ecology@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
___
R-sig-ecology mailing list
R-sig-ecology@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
We have had similar reports from many cases (for instance, from cca/rda functions in vegan). The problem is difficult to track. Cases that we have seen so far probably are problems in LAPACK library, and the kind of LAPACK used can depend on the version of R, version of LAPACK, compilers used, operating system and its version, computer hardware, phase of moon etc. Error code 1 from dgesdd should be a convergence problem. With presence/absence data there may not be much you can with the data. With quantitative data, rescaling of data has been helpful, but hardly with p/a (Hellinger transformation will also change data properties and is not data neutral: division of subtraction with some constant is data neutral and helps sometimes, but hardly with p/a data). The svd() function in R provides a R<1.7.0 compatibility option LINPACK = TRUE which has helped sometimes. We haven't added that as an option for svd() in cocorresp or vegan as it is documented only as a compatibility option for old R. However, you can try editing the files to use LINPACK = TRUE to see if the problem vanishes (but NAMESPACE add some pain in using edited files). HTH, Jari Oksanen From: r-sig-ecology-boun...@r-project.org [r-sig-ecology-boun...@r-project.org] on behalf of Duarte Viana [viana.s...@gmail.com] Sent: 14 May 2012 16:09 To: r-sig-ecology@r-project.org Subject: [R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd' Hello all, When I perform a cross validation with the function "crossval" of the package cocorresp (for a co-correspondence analysis), I get the following error: Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd' I think the problem is not in the function, as other people has had the same error for other types of model (see, for example, http://comments.gmane.org/gmane.comp.lang.r.ecology/2608). From this and other posts, I suppose it is a convergence problem. I am working with presence-absence species matrices, and the problem persists even if I transform the matrices with Hellinger transformation (although not at the same site). Does anyone know how to solve, or go around, this problem? Any advice would be most welcome. Thanks, Duarte ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[R-sig-eco] Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
Hello all, When I perform a cross validation with the function "crossval" of the package cocorresp (for a co-correspondence analysis), I get the following error: Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd' I think the problem is not in the function, as other people has had the same error for other types of model (see, for example, http://comments.gmane.org/gmane.comp.lang.r.ecology/2608). From this and other posts, I suppose it is a convergence problem. I am working with presence-absence species matrices, and the problem persists even if I transform the matrices with Hellinger transformation (although not at the same site). Does anyone know how to solve, or go around, this problem? Any advice would be most welcome. Thanks, Duarte ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
