On Mon, Jul 23, 2018 at 7:28 AM, 'Andrey Brukhno' via votca <[email protected]> wrote: > Christoph, > > I think I figured why there is no log msg for critical csg_stat in > calc_rdfgeneric.sh. > The fact is I override the inverse.sh script with the one having this (to > avoid all the frame numbers when reading trajectory files): > > log="${PWD}/${log##*/}" > export CSGLOG="$log" > if [[ -f $CSGLOG ]]; then > exec 3>&1 4>&2 >> "$CSGLOG" 2>&1 > echo -e "\n\n##################################" > # echo "# Appending to existing logfile #" > echo "# Appending to logfile (+errors) #" > echo -e "##################################\n\n" > msg --color blue "Appending to existing logfile (+errors) > ${CSGLOG##*/}" > else > exec 3>&1 4>&2 >> "$CSGLOG" 2>/dev/null > echo -e "\n\n##################################" > echo "# Creating new logfile (-errors) #" > echo -e "##################################\n\n" > msg "For a more verbose log see: ${CSGLOG##*/} (or csg_inverse.log)" > fi > > This worked before and did not affect the workflow. Glad, you found it and it is not VOTCA's original code. I remember now that you did this modification for a reason, which I didn't like exactly because error message will get lost.
Anyhow, well in conclusion if you send the errors to dev/null, you won't see some messages (like critical) in the log file. Christoph > > Andrey > > On Mon, Jul 23, 2018 at 1:50 PM, Christoph Junghans <[email protected]> > wrote: >> >> On Mon, Jul 23, 2018 at 6:46 AM, 'Andrey Brukhno' via votca >> <[email protected]> wrote: >> > Sorry, I forgot - below is the output of `grep critical inverse.log' for >> > one >> > iteration: >> > >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p >> > topol.top >> > -o topol.tpr -c conf.gro' >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p >> > topol.top >> > -o topol.tpr -c conf.gro' >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p >> > topol.top >> > -o topol.tpr -c conf.gro' >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p >> > topol.top >> > -o topol.tpr -c conf.gro' >> > Running critical command 'mpirun -n 4 mdrun_mpi -s topol.tpr -c >> > confout.gro >> > -o traj.trr -x traj.xtc -cpi state.cpt -maxh 47.8289 -multidir sim_0 >> > sim_1 >> > sim_2 sim_3 -tablep table.xvg -tableb table_b1.xvg table_b2.xvg >> > table_b3.xvg >> > table_b4.xvg table_b5.xvg table_a1.xvg table_a2.xvg table_a3.xvg >> > table_a4.xvg table_a5.xvg table_d1.xvg table_d2.xvg table_d3.xvg >> > table_d4.xvg table_d5.xvg table_d6.xvg' >> > Doing: critical csg_stat --nt 4 --options >> > >> > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cNT/dopc_cg-int-map11.xml >> > --top topol-rdf.tpr --trj traj.xtc --begin 0 --first-frame 0 >> > --block-length >> > 1000000 --ext dist.block --cg >> > >> > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cNT/dopc_cg-map11.xml; >> > >> > As you can see, the only mention of `critical csg_stat' is due to my >> > addition of echo. >> Well, the other lines with "Running critical command" show that the >> logging actually works. >> Did you overload calc_rdf_generic.sh with your own version? >> Otherwise, I would say your installation is bourked up. >> >> Christoph >> > >> > Andrey >> > >> > On Monday, July 23, 2018 at 1:17:32 PM UTC+1, Christoph Junghans wrote: >> >> >> >> On Mon, Jul 23, 2018 at 2:39 AM, 'Andrey Brukhno' via votca >> >> <[email protected]> wrote: >> >> > >> >> > On Mon, Jul 23, 2018 at 2:22 AM, Christoph Junghans >> >> > <[email protected]> >> >> > wrote: >> >> >> >> >> >> On Sun, Jul 22, 2018 at 1:27 PM, 'Andrey Brukhno' via votca >> >> >> <[email protected]> wrote: >> >> >> > Thanks for your reply Christoph. >> >> >> > >> >> >> > I did check it, but inverse.log does not contain sufficient info, >> >> >> > see >> >> >> > the >> >> >> > extract below. >> >> >> > I will be investigating it further to make sure I did not do >> >> >> > anything >> >> >> > stupid >> >> >> > myself, although I haven't changed much for the test iteration >> >> >> > except >> >> >> > recompiling the tpr files for the newer version of Gromacs). >> >> >> The " rdf calculation is already done" tells me that this is not a >> >> >> run >> >> >> from scratch. >> >> >> Try a fresh run, because some chance in the xml don't get populated >> >> >> until the next iteration step. >> >> >> And grep for "csg_stat" (not rdf), you should see actually command, >> >> >> which got run, e.g.: >> >> >> "Running critical command 'csg_stat --nt 4 --options settings.xml >> >> >> --top topol.tpr --trj traj.xtc --begin 20 --first-frame 0' " >> >> >> From that you can see if all the options are read as excepted. >> >> >> >> >> > >> >> > I knew you were going to say this... >> >> > As a matter of fact, there is no such message from >> >> > calc_rdf_generic.sh, >> >> > but >> >> > instead: >> >> > >> >> > msg "Calculating rdfs with csg_stat using $tasks tasks" >> >> > critical csg_stat --nt $tasks --options "$CSGXMLFILE" --top >> >> > "$topol" >> >> > --trj >> >> > "$traj" --begin $equi_time --first-frame $first_frame ${error_opts} >> >> > ${maps:+--cg ${maps}} >> >> > mark_done "rdf_calculation${suffix}" >> >> Well the "critical" function includes the print statement: >> >> critical() { >> >> [[ $quiet = "no" ]] && echo "Running critical command '$*'" >&2 >> >> "$@" || die "${FUNCNAME[0]}: '$*' failed" >> >> } >> >> So I am not sure when that got lost in your case. >> >> > >> >> > so I had to add this after the msg and before the csg_stat call: >> >> > echo "Doing: critical csg_stat --nt $tasks --options "$CSGXMLFILE" >> >> > --top >> >> > "$topol" --trj "$traj" --begin $equi_time --first-frame $first_frame >> >> > ${error_opts} ${maps:+--cg ${maps}}" >> >> > >> >> > This shows me that topol-rdf.tpr is actually used (extract from >> >> > inverse.log): >> >> > >> >> > Make update for ibi >> >> > Calculating rdfs with csg_stat using 4 tasks >> >> > Doing: critical csg_stat --nt 4 --options >> >> > >> >> > >> >> > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cT/dopc_cg-int-map11.xml >> >> > --top topol-rdf.tpr --trj traj.xtc --begin 0 --first-frame 0 >> >> > --block-length >> >> > 1000000 --ext dist.block --cg >> >> > >> >> > >> >> > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cT/dopc_cg-map11.xml; >> >> > begin to calculate distribution functions >> >> > >> >> > However, the distributions are still wrong, and I am lost now as to >> >> > which >> >> > program to blame (again all the initial files are the same as in my >> >> > previous >> >> > iteration). >> >> > I compared the outputs of `gmx dump topol-rdf.tpr' and `gmx dump >> >> > topol.tpr' >> >> > generated with the new Gromacs and everything seems alright, the >> >> > relevant >> >> > differences in exclusion lists are there. >> >> > The problem is also that the original calculation was done on a >> >> > cluster >> >> > more >> >> > than a year ago, so I cannot reproduce exactly all the installation >> >> > setup, >> >> > not on my desktop anyway. But I know that those calculations were >> >> > correct >> >> > and well converged. >> >> > >> >> > So something should have changed somewhere within the newly installed >> >> > Gromacs/VOTCA combination. >> >> You can see what VOTCA has with: >> >> csg_dump --top topol-rdf.tpr --cg dopc_cg-map11.xml >> >> >> >> Remember, if you are using a mapping file ("--cg" option) exclusions >> >> from the topology file are ignored and the ones from the mapping file >> >> are used instead. >> >> >> >> Christoph >> >> > >> >> > Andrey >> >> > >> >> >> Christoph >> >> >> > >> >> >> > Andrey >> >> >> > ----- >> >> >> > $ grep rdf inverse.log >> >> >> > ... >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr >> >> >> > table_??.xvg >> >> >> > index.ndx* conf.gro*' >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr >> >> >> > table_??.xvg >> >> >> > index.ndx* conf.gro*' >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr >> >> >> > table_??.xvg >> >> >> > index.ndx* conf.gro*' >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr >> >> >> > table_??.xvg >> >> >> > index.ndx* conf.gro*' >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr >> >> >> > table_??.xvg >> >> >> > index.ndx* conf.gro*' >> >> >> > ‘sim_0/topol-rdf.tpr’ -> ‘./topol-rdf.tpr’ >> >> >> > Calculating rdfs with csg_stat using 4 tasks >> >> >> > Calculating average rdfs and its errors for interaction TCH-TCH >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction TCH-TCO >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction TCO-TCO >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction TCH-PO >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction TCH-NH >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction TCO-PO >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction TCO-NH >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction PO-PO >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction PO-NH >> >> >> > rdf calculation is already done >> >> >> > Calculating average rdfs and its errors for interaction NH-NH >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Sun, Jul 22, 2018 at 1:26 PM, Christoph Junghans >> >> >> > <[email protected]> >> >> >> > wrote: >> >> >> >> >> >> >> >> On Sat, Jul 21, 2018 at 9:01 PM, 'Andrey Brukhno' via votca >> >> >> >> <[email protected]> wrote: >> >> >> >> > Hello, >> >> >> >> > >> >> >> >> > I encountered a very strange (looks like legacy) issue with >> >> >> >> > v1.4.1. >> >> >> >> > In my earlier IBI iterations (with v1.3) I normally used >> >> >> >> > special >> >> >> >> > topol-rdf.tpr which was different from normal CG simulation >> >> >> >> > topol.tpr >> >> >> >> > by >> >> >> >> > not >> >> >> >> > including the intra-molecular pairs in the RDF stats (done by >> >> >> >> > csg_stat). >> >> >> >> > I >> >> >> >> > wanted to make sure my previous results are reproduced by >> >> >> >> > v1.4.1, >> >> >> >> > so >> >> >> >> > copied >> >> >> >> > all the initialisation files into a new directory and tried to >> >> >> >> > run >> >> >> >> > once >> >> >> >> > again. After the very first iteration I checked the obtained >> >> >> >> > RDFs >> >> >> >> > and >> >> >> >> > they >> >> >> >> > were all wrong - corresponding to normal topol.tpr (including >> >> >> >> > intra-pairs) >> >> >> >> > and not topol-rdf.tpr (excluding intra-pairs)! >> >> >> >> > >> >> >> >> > I guess there might have been changes made to locations or >> >> >> >> > names >> >> >> >> > of >> >> >> >> > tags >> >> >> >> > in >> >> >> >> > settings.xml between the VOTCA versions, but I could not >> >> >> >> > identify >> >> >> >> > those >> >> >> >> > changes by referring to the most recent manual (both v1.4.1 and >> >> >> >> > development >> >> >> >> > versions). >> >> >> >> > >> >> >> >> > Below is the relevant extract from my settings.xml file. >> >> >> >> > >> >> >> >> > Any clue, hint, advice are most welcome! >> >> >> >> We haven't changes any tags in IBI since v1.4.1 and the special >> >> >> >> topology for IBI can be set by cg.inverse.<sim_prog>.rdf.topol, >> >> >> >> which >> >> >> >> you did below. >> >> >> >> You could try "grep csg_stat inverse.log" to figure out if the >> >> >> >> right >> >> >> >> topology got appended to the command line of csg_stat. >> >> >> >> >> >> >> >> Christoph >> >> >> >> > >> >> >> >> > Thanks in advance. >> >> >> >> > Andrey >> >> >> >> > >> >> >> >> > ----------- >> >> >> >> > <gromacs> >> >> >> >> > <grompp> >> >> >> >> > <bin>gmx grompp</bin> >> >> >> >> > <!--opts></opts--> >> >> >> >> > </grompp> >> >> >> >> > <mdrun> >> >> >> >> > <!--command>grmx-vtc-rerun.sh</command--> >> >> >> >> > <!--command>mpirun -lsf mdrun_mpi</command--> >> >> >> >> > <!--opts>-multidir sim_0 sim_1 sim_2 sim_3</opts--> >> >> >> >> > <command>mpirun -n 4 mdrun_mpi</command> >> >> >> >> > <opts>-multidir sim_0 sim_1 sim_2 sim_3 -tablep >> >> >> >> > table.xvg >> >> >> >> > -tableb >> >> >> >> > table_b?.xvg table_a?.xvg table_d?.xvg</opts> >> >> >> >> > </mdrun> >> >> >> >> > <!--mdrun><opts>-nt 2</opts></mdrun--> >> >> >> >> > <!-- trash so many frames at the beginning --> >> >> >> >> > <equi_time>0</equi_time> >> >> >> >> > <!-- grid for table*.xvg !--> >> >> >> >> > <table_bins>0.001</table_bins> >> >> >> >> > <!-- cut the potential at this value (gromacs bug) --> >> >> >> >> > <pot_max>10000</pot_max> >> >> >> >> > <!-- extend the tables to this value --> >> >> >> >> > <table_end>2.5</table_end> >> >> >> >> > <initial_configuration>maindir</initial_configuration> >> >> >> >> > <rdf> >> >> >> >> > <block_length><DESC>Length of the block for the error >> >> >> >> > analysis</DESC> >> >> >> >> > 1000000 >> >> >> >> > </block_length> >> >> >> >> > <map><DESC>Special mapping file for rdf calculations >> >> >> >> > needed >> >> >> >> > for >> >> >> >> > bonded interactions</DESC> >> >> >> >> > dopc_cg-map11.xml >> >> >> >> > </map> >> >> >> >> > <topol><DESC> Gromacs binary topol (tpr) file to be >> >> >> >> > used >> >> >> >> > for >> >> >> >> > csg_stat</DESC> >> >> >> >> > topol-rdf.tpr >> >> >> >> > </topol> >> >> >> >> > <with_errors>yes<DESC>calculate error on the rdf: >> >> >> >> > yes/no</DESC> >> >> >> >> > </with_errors> >> >> >> >> > </rdf> >> >> >> >> > </gromacs> >> >> >> >> > ----------- >> >> >> >> > >> >> >> >> > -- >> >> >> >> > You received this message because you are subscribed to the >> >> >> >> > Google >> >> >> >> > Groups >> >> >> >> > "votca" group. >> >> >> >> > To unsubscribe from this group and stop receiving emails from >> >> >> >> > it, >> >> >> >> > send >> >> >> >> > an >> >> >> >> > email to [email protected]. >> >> >> >> > To post to this group, send email to [email protected]. >> >> >> >> > Visit this group at https://groups.google.com/group/votca. >> >> >> >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Christoph Junghans >> >> >> >> Web: http://www.compphys.de >> >> >> >> >> >> >> >> -- >> >> >> >> You received this message because you are subscribed to a topic >> >> >> >> in >> >> >> >> the >> >> >> >> Google Groups "votca" group. >> >> >> >> To unsubscribe from this topic, visit >> >> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe. >> >> >> >> To unsubscribe from this group and all its topics, send an email >> >> >> >> to >> >> >> >> [email protected]. >> >> >> >> To post to this group, send email to [email protected]. >> >> >> >> Visit this group at https://groups.google.com/group/votca. >> >> >> >> For more options, visit https://groups.google.com/d/optout. >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > You received this message because you are subscribed to the Google >> >> >> > Groups >> >> >> > "votca" group. >> >> >> > To unsubscribe from this group and stop receiving emails from it, >> >> >> > send >> >> >> > an >> >> >> > email to [email protected]. >> >> >> > To post to this group, send email to [email protected]. >> >> >> > Visit this group at https://groups.google.com/group/votca. >> >> >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Christoph Junghans >> >> >> Web: http://www.compphys.de >> >> >> >> >> >> -- >> >> >> You received this message because you are subscribed to a topic in >> >> >> the >> >> >> Google Groups "votca" group. >> >> >> To unsubscribe from this topic, visit >> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe. >> >> >> To unsubscribe from this group and all its topics, send an email to >> >> >> [email protected]. >> >> >> To post to this group, send email to [email protected]. >> >> >> Visit this group at https://groups.google.com/group/votca. >> >> >> For more options, visit https://groups.google.com/d/optout. >> >> > >> >> > >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups >> >> > "votca" group. >> >> > To unsubscribe from this group and stop receiving emails from it, >> >> > send >> >> > an >> >> > email to [email protected]. >> >> > To post to this group, send email to [email protected]. >> >> > Visit this group at https://groups.google.com/group/votca. >> >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> >> >> >> -- >> >> Christoph Junghans >> >> Web: http://www.compphys.de >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "votca" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at https://groups.google.com/group/votca. >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> Christoph Junghans >> Web: http://www.compphys.de >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "votca" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/votca. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "votca" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/votca. > For more options, visit https://groups.google.com/d/optout. -- Christoph Junghans Web: http://www.compphys.de -- You received this message because you are subscribed to the Google Groups "votca" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/votca. For more options, visit https://groups.google.com/d/optout.
