Thank you very much for your reply. I am a summer trainee, and I talked to my guide about this, and now what I want might be a little different.
I really do want all the eigenvalues. There cannot be any compromise on that. But please note that time is not much of a concern here, the much bigger concern is of memory. Even if I can distribute the memory somehow for the computation of all eigenvalues, that is fine. (doesnt matter if it takes a huge amount of time, he is willing to wait). Can any slepc Eigensolver distribute memory this way while solving for all eigenvalues ? I am sorry if what I am asking is obvious, I dont know much about distributed memory, etc. also. I am stupid at this moment, but I am learning. Also, when I ran the slepc program on a small cluster, and then ran the top command, I saw that while solving for eigenvalues, the program was already using upto 500% CPU. Does this mean that the eigensolver was "distributing memory" automatically among different nodes ? I understand that it must be distributing computational effort, but how do I know if it was distributing memory also ? It simply showed me the RES memory usage to be 1.3 gb, and the %MEM to be about 17%. Also, I did not use any MPIXXX data structures. I simply used SeqAIJ. Will using MPIAIJ "distribute memory" while solving for all eigenvalues ? (I understand that it will distribute memory to store the matrix, but the memory bottleneck is eigenvalue solver, so its more important for memory to be distributed then ) My guide said that one node will not have enough memory, but all nodes combined will, and hence the requirement of memory to be distributed. I read about ScaLAPACK, and I have already asked on their forums to confirm if it is suitable for me. I am waiting for an answer. Thank you once again. Shitij On 3 August 2011 14:31, Jose E. Roman <jroman at dsic.upv.es> wrote: > > El 03/08/2011, a las 08:30, Shitij Bhargava escribi?: > > > Hi all !! > > > > > > > > The reason I have started using SLEPc/PETSc is that I want to reduce the > memory requirements of a program that I already have in Intel MKL. The > program's purpose is to calculate ALL eigenvalues and eigenvectors of a real > symmetric sparse matrix. Intel MKL doesnt have a sparse solver for this > (cant solve for eigenvalues of a matrix represented in sparse format), so I > am using SLEPc, as it can do this. > > (NOTE: by "sparsity" I mean ZEROES*100/(ZEROES+NONZEROES). Also the > tolerance I am using is 10^-6) > > If you really want to compute ALL eigenvalues, then SLEPc is not the way to > go. SLEPc is intended for computing a few eigenpairs, and in some cases a > few percentage, e.g. 20% at most. Computing more than that with SLEPc is > probably going to be less efficient than other alternatives. > > > > > Till now I have been using the LAPACK method (EPSLAPACK) for the slepc > eigensolver, as I found out that it was the fastest (I configured PETSc to > use Intel MKL BLAS/LAPACK routines). But I just found out that it is using a > lot of memory irrespective of the sparsity of the matrix. For example, on a > matrix of order 9600x9600 no matter how sparse/dense it is (I tested from > 0.02% to 75% sparsity), it used 1.3gb of peak memory every time, whereas > the part of the program in which the matrix is assembled in AIJ format (in > case of 75% sparsity) takes only 550mb. The MKL program takes 3.1gb. > Although memory requirement is less, the difference is constant irrespective > of sparsity.So, it seems that the LAPACK method will use memory dependent on > the order of the matrix, irrespective of the sparsity. So I guess its not > very suitable for my purpose ? I also tried changing the NCV and MPD > (Maximum projected Dimension) parameters hoping they would somehow reduce > memory requirements even if at the cost of more time, but whatever I pass > NCV and MPD (in EPSSetDimensions) as, they are not considered at all. They > remain the same as the default values (as I verified with EPSGetDimensions). > > The EPSLAPACK solver converts the matrix to dense format. As explained in > the documentation, this solver must be used only on small matrices for > debugging purposes. NCV and MPD are not relevant in this setting. > > > > > So I tried other solvers like the default, krylovschur, lanczos, but I > think they are built to find only a few eigenvalues and not all, because > they almost never converged to find ALL eigenvalues, even if I gave them a > lot of time (big max number of iterations parameter). But their memory > requirements seem to be pretty less. For example, I ran the program with > EPSLANCZOS on the same 9600x9600 matrix (75% sparsity) with maximum number > of iterations as 500,000. It ran it for 6.5 hrs (then I killed it), and at > the end it was using 530mb only (although I dont know how near it was to > completion). > > Krylov-Schur is almost always preferred over Lanczos, since the latter > implements an explicit restart which is much worse than implicit restart in > Krylov-Schur. Anyway, in both of them if you set NEV=9600 you are building a > projected problem (dense matrix) of order equal to the original matrix so > you have the same cost as with EPSLAPACK together with the cost of building > the Krylov subspace. Again: do not use SLEPc for all eigenvalues! > > > > > I am somewhat illiterate about the mathematics behind all this, and how > the lapack,krylovschur, etc. work and I have no idea if what I am > experiencing was expected/obvious, although I half-expected some things by > reading the slepc documentation, where a table was given comparing different > eigensolvers. > > A general recommendation is to try to understand a little bit how the > algorithms work before attempting to use them. > > > > > Also, I have observed that as the sparsity of the matrix increases (more > zeroes), more eigenvalues also become zeroes. Is there a way to exploit this > using probably krylovschur/lanczos so that there is no need to converge to > all eigenvalues (meaning that the time they will take will be reasonable), > but to only the number of eigenvalues that are non-zero,so that I can assume > the non-converged eigenvalues to be simply zeroes (but is there a way to > know beforehand by sparsity approximately how many non-zeroe eigenvalues > will be there, and the corresponding value of maximum iterations that are > required ?) > > There is no direct connection between the number of zero entries and the > number of zero eigenvalues, unless you have rows/columns with all zeros, in > which case the (permuted) matrix can be written as A=[A11 0; 0 0] so it is > enough to compute eigenvalues of A11. > > > > > Is there a way to do what I want to do through SLEPc ??? I want ALL > eigenvalues and want that the memory requirements of the eigensolver should > be less/in accordance with the sparsity of the matrix, otherwise the very > purpose of using a sparse matrix representation is somewhat defeated. I am > doing all this in FORTRAN by the way. > > Ask yourself this question: is it really necessary to compute all > eigenvalues? If the answer is yes then maybe ScaLAPACK is what you need. > > Jose > > > > > > > Thanks a lot in advance !!! > > > > > > Shitij > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110804/c178ca4b/attachment.htm>
