On Mon, 2006-10-16 at 17:38 +1000, Edward d'Auvergne wrote: > It does seem a little bizarre that it took so long on fast machines. > I'm still not sure whether there is a problem or whether it's due to > the size of the system and the data sets. The final stage of the > script, which generates the results file in the 'opt' directory, can > be the most computationally expensive. This is because it is the > global model (the diffusion model together with all the model-free > models of all residues) which is being optimised. The dimensionality > of this model is equal to the total of all parameters in your model > (sum the model-free parameters for all residues and the diffusion > parameters). Because of this dimensionality, optimisation may take a > while.
As I said previously, when I have worked with similar sized datasets I have found the calulation to be considerably quicker than this. Admittedly I rarely used the larger models, and did see some variation in calculation time depending on where the calculation started. > Was the 2nd iteration much quicker to complete? Each iteration of the > 'full_analysis.py' script should complete quicker and quicker. To > start with you have a collapse of the complexity of the global model. > This collapse should significantly speed up the grid searches and > optimisations between iterations. Secondly, as iterations pass the > diffusion tensor gets closer and closer to the optimal values. This > means that the optimisation of the global model will be much quicker. > > If after a few iterations the calculations are still slow, there may > be a problem somewhere. Do the results look reasonable? If you're > using a Linux system you can type 'less -S opt/results.bz2' to view > the results file. Can you see any issues or bizarre values in the > results files? One issue that might cause very slow optimisation at this stage is lots of larger models. Typically the majority of residues should fit to model 1, with relatively few in the larger models. If this is not the case, it probably indicates some sort of problem with the data or the diffusion estimate (or just a protein with particularly unusual dynamics?). Of course larger models implies more parameters and longer calculations. Improving the efficiency of relax has been something we have thought a lot about, and remains a goal for further development. The more information we have about where the bottlenecks are, the better our chances of improving things. Chris > > Edward > > > On 10/14/06, Sebastien Morin <[EMAIL PROTECTED]> wrote: > > Hi > > > > The simplest diffusion model ended the first round after 42h00 on a > > Intel Pentium4 3.20GHz (Pelican)... > > > > Now, after more than 50h00, the oblate (Intel Xeon CPU 2.80GHz, Hibou > > 1st CPU), prolate (Intel Xeon CPU 2.80GHz, Hibou 2nd CPU) and ellipsoid > > (Intel Pentium4 CPU 3.20GHz, Pigeon) diffusion models are still running > > on the first round... > > > > Here are the last 80 lines of the log file for one of these stuck runs : > > > > ======================================================= > > Entering sub-algorithm. > > > > Newton minimisation > > ~~~~~~~~~~~~~~~~~~~ > > Line search: Backtracking line search. > > Hessian modification: The Gill, Murray, and Wright modified > > Cholesky algorithm. > > > > k: 0 xk: array([ 1.27705900e+04, 3.17894496e-01, > > 4.30908875e-01, 4.55262677e-01, > > 8.50838124e-01, 2.47861916e+01, 7.92671118e-01, > > 2.14907851e+03, > > 8.81854967e-01, 1.50334824e+01, 8.54027315e-01, > > 1.64578232e+03, > > 8.76787298e-01, 7.92195168e-01, 1.79787560e+03, > > 9.87557449e-01, > > 8.21228607e-01, 7.74117949e-01, 1.70184872e+03, > > 1.73932210e+00, > > 9.49188204e-01, 8.85871296e-01, 1.04788711e+01, > > 9.17771174e-01, > > 9.04377403e-01, 4.08815580e+02, 7.65590609e-01, > > 9.11933036e-01, > > 8.91496184e-01, 5.56655768e+02, 1.44843220e+00, > > 8.90378757e-01, > > 8.50622278e-01, 1.49912928e+03, 1.06027214e+00, > > 8.65625762e-01, > > 3.88839270e+00, 8.35578101e-01, 5.25658520e+03, > > 1.23377507e+00, > > 8.50832366e-01, 1.14258349e+01, 1.25778023e+00, > > 8.58099834e-01, > > 8.33315784e-01, 1.04555350e+03, 8.80166810e-01, > > 8.42391353e-01, > > 1.07328360e+03, 2.70180125e-01, 8.88393547e-01, > > 1.26380692e+01, > > 7.51137909e-01, 7.16933985e+00, 7.01915972e-01, > > 1.58882726e+03, > > 2.68906901e+00, 8.51688163e-01, 8.27015263e-01, > > 6.25535130e+02, > > 1.51074548e+00, 9.03661361e-01, 8.87558977e-01, > > 4.80493151e+02, > > 8.68556161e-01, 9.97819222e-01, 9.26383233e-01, > > 1.31515397e+01, > > 5.99525186e-01, 8.57525619e-01, 8.30638445e-01, > > 1.85283345e+03, > > 1.75756916e+00, 8.72986688e-01, 1.71531471e+01, > > 4.27254670e-01, > > 8.80694475e-01, 1.81533883e+00, 8.70141488e-01, > > 1.60397307e+01, > > 1.03406591e+00, 9.58191732e-01, 8.90603591e-01, > > 4.15861810e+02, > > 8.30733937e-01, 1.09272538e+01, 7.23427079e-01, > > 8.93017099e+02, > > 7.89229531e-01, 1.33440644e+01, 7.33002966e-01, > > 1.87268372e+03, > > 1.42199321e+00, 7.53967611e-01, 1.66782443e+01, > > 6.61064763e-01, > > 2.89157277e+03, 2.33751641e+00, 8.78699341e-01, > > 2.97736701e+01, > > 8.38759921e-01, 3.18889549e+00, 8.28754282e-01, > > 8.17093600e-01, > > 2.86614347e+02, 1.98926865e+00, 8.07960444e-01, > > 7.79624226e-01, > > 8.20216424e+02, 1.69602623e+00, 8.69023915e-01, > > 1.29721545e+00, > > 8.98814498e-01, 5.63508354e-01, 8.92301958e-01, > > 8.72610000e-01, > > 5.83216661e+02, 1.72077954e+00, 8.31900384e-01, > > 7.95421005e-01, > > 3.27213483e+03, 1.47608324e+00, 8.84116554e-01, > > 3.00188413e+01, > > 8.42580844e-01, 8.13678584e-01, 1.28370304e+03, > > 1.52597551e+00, > > 9.50467888e-01, 8.94644170e-01, 8.41203713e-01, > > 1.10218956e+03, > > 2.26006275e+00, 9.68607682e-01, 3.25379935e+01, > > 8.41768913e-01, > > 9.74570652e-01, 1.08978427e+00, 8.32856285e-01, > > 8.07695592e-01, > > 2.11975345e+03, 1.95149433e+00, 8.75200671e-01, > > 1.18320570e+00, > > 8.82565115e-01, 1.72699728e+00, 8.47796885e-01, > > 1.51362535e+00, > > 8.69079500e-01, 5.14374767e+00, 2.29480254e+00, > > 9.13799620e-01, > > 1.72758307e+01, 2.99204443e-01, 8.57041668e-01, > > 2.35031795e+00, > > 8.96758624e-01, 2.18867392e+00, 8.88004177e-01, > > 1.55494958e+00, > > 8.77947179e-01, 1.11920801e+01, 9.15181658e-01, > > 8.46939161e-01, > > 1.83610590e+00, 8.38653701e-01, 8.18943616e-01, > > 3.56598335e+02, > > 8.52078429e-01, 8.34895555e-01, 3.43932619e+02, > > 7.87542654e-01, > > 8.37946629e-01, 1.24587108e+01, 1.64066609e+00, > > 8.40856744e-01, > > 7.95194162e-01, 5.67222774e+02, 8.18578387e-01, > > 7.71610058e-01, > > 1.65280184e+03, 1.22176221e+00, 8.50605183e-01, > > 7.42576359e+00, > > 1.18569636e+00, 8.20365931e-01, 8.04178929e-01, > > 4.23814384e+02, > > 1.63292525e+00, 7.87312635e-01, 7.21123309e-01, > > 1.55627601e+03, > > 2.10657158e+00, 8.26679923e-01, 7.90596693e-01, > > 2.58068396e+02, > > 2.69944847e-01, 8.24304505e-01, 7.88039728e-01, > > 3.81540776e+02, > > 4.12294846e-01, 8.10539892e-01, 1.76756373e+01, > > 1.59567054e+00, > > 8.69519794e-01, 2.27950293e+01, 1.60108929e+00, > > 8.46240651e-01, > > 8.16750840e-01, 3.36726192e+02, 6.52545204e-01, > > 7.98871329e-01, > > 1.67585865e+01, 1.29442868e+00, 8.72012584e-01, > > 1.58721788e+01, > > 8.26110419e-01, 1.83079181e+03, 8.32081822e-01, > > 9.35711588e-01, > > 4.44948915e+01, 9.47239844e-01, 8.69414580e-01, > > 2.60489705e+00, > > 8.77732345e-01, 2.73754199e+00, 8.83674982e-01, > > 8.54351418e-01, > > 2.31262785e+02, 3.66419461e-01, 7.86155599e-01, > > 1.01394952e+01, > > 1.14673187e+00, 7.92655557e-01, 1.82156160e+01, > > 1.86619078e+00, > > 8.73237261e-01, 8.49023676e-01, 3.69985402e+02, > > 8.63263470e-01, > > 8.05115541e-01, 7.80479811e-01, 4.72043168e+02, > > 2.03730546e+00, > > 7.71825768e-01, 7.31245168e-01, 5.45596104e+02, > > 8.32533217e-01, > > 7.83786194e-01, 7.18560779e-01, 1.00144691e+03, > > 1.92514722e+00, > > 8.98011263e-01, 1.61615950e+01, 8.72635732e-01, > > 1.94464132e+00, > > 8.13134043e-01, 7.55765479e+00, 2.97987565e+00, > > 8.18914895e-01, > > 7.86593136e-01, 2.12749118e+03, 2.53631014e+00, > > 8.64185295e-01, > > 6.13997562e+00, 1.89991509e+00, 8.44947845e-01, 2 > > > > ======================================================= > > > > It seems the runs take a very long time making the final step of each > > round (the step when they create the 'opt' directory, since all other > > directories are already done). > > > > Maybe it just the size of the dataset that slows down things... Let's be > > patient !! > > > > Cheers > > > > > > Séb > > > > > _______________________________________________ relax (http://nmr-relax.com) This is the relax-users mailing list [email protected] To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
