In this case, it's not an error message which is lost ;) Why does it go to stderr at all? Moreover, if I try to run the iteration a second time, everything including errors is reported back
Anyway, my primary issue is not about messages, but to make it work correctly for distributions. Andrey On Monday, July 23, 2018 at 2:33:44 PM UTC+1, Christoph Junghans wrote: > > On Mon, Jul 23, 2018 at 7:28 AM, 'Andrey Brukhno' via votca > <[email protected] <javascript:>> wrote: > > Christoph, > > > > I think I figured why there is no log msg for critical csg_stat in > > calc_rdfgeneric.sh. > > The fact is I override the inverse.sh script with the one having this > (to > > avoid all the frame numbers when reading trajectory files): > > > > log="${PWD}/${log##*/}" > > export CSGLOG="$log" > > if [[ -f $CSGLOG ]]; then > > exec 3>&1 4>&2 >> "$CSGLOG" 2>&1 > > echo -e "\n\n##################################" > > # echo "# Appending to existing logfile #" > > echo "# Appending to logfile (+errors) #" > > echo -e "##################################\n\n" > > msg --color blue "Appending to existing logfile (+errors) > > ${CSGLOG##*/}" > > else > > exec 3>&1 4>&2 >> "$CSGLOG" 2>/dev/null > > echo -e "\n\n##################################" > > echo "# Creating new logfile (-errors) #" > > echo -e "##################################\n\n" > > msg "For a more verbose log see: ${CSGLOG##*/} (or > csg_inverse.log)" > > fi > > > > This worked before and did not affect the workflow. > Glad, you found it and it is not VOTCA's original code. > I remember now that you did this modification for a reason, which I > didn't like exactly because error message will get lost. > > Anyhow, well in conclusion if you send the errors to dev/null, you > won't see some messages (like critical) in the log file. > > Christoph > > > > Andrey > > > > On Mon, Jul 23, 2018 at 1:50 PM, Christoph Junghans <[email protected] > <javascript:>> > > wrote: > >> > >> On Mon, Jul 23, 2018 at 6:46 AM, 'Andrey Brukhno' via votca > >> <[email protected] <javascript:>> wrote: > >> > Sorry, I forgot - below is the output of `grep critical inverse.log' > for > >> > one > >> > iteration: > >> > > >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p > >> > topol.top > >> > -o topol.tpr -c conf.gro' > >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p > >> > topol.top > >> > -o topol.tpr -c conf.gro' > >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p > >> > topol.top > >> > -o topol.tpr -c conf.gro' > >> > Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p > >> > topol.top > >> > -o topol.tpr -c conf.gro' > >> > Running critical command 'mpirun -n 4 mdrun_mpi -s topol.tpr -c > >> > confout.gro > >> > -o traj.trr -x traj.xtc -cpi state.cpt -maxh 47.8289 -multidir sim_0 > >> > sim_1 > >> > sim_2 sim_3 -tablep table.xvg -tableb table_b1.xvg table_b2.xvg > >> > table_b3.xvg > >> > table_b4.xvg table_b5.xvg table_a1.xvg table_a2.xvg table_a3.xvg > >> > table_a4.xvg table_a5.xvg table_d1.xvg table_d2.xvg table_d3.xvg > >> > table_d4.xvg table_d5.xvg table_d6.xvg' > >> > Doing: critical csg_stat --nt 4 --options > >> > > >> > > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cNT/dopc_cg-int-map11.xml > > > >> > --top topol-rdf.tpr --trj traj.xtc --begin 0 --first-frame 0 > >> > --block-length > >> > 1000000 --ext dist.block --cg > >> > > >> > > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cNT/dopc_cg-map11.xml; > > > >> > > >> > As you can see, the only mention of `critical csg_stat' is due to my > >> > addition of echo. > >> Well, the other lines with "Running critical command" show that the > >> logging actually works. > >> Did you overload calc_rdf_generic.sh with your own version? > >> Otherwise, I would say your installation is bourked up. > >> > >> Christoph > >> > > >> > Andrey > >> > > >> > On Monday, July 23, 2018 at 1:17:32 PM UTC+1, Christoph Junghans > wrote: > >> >> > >> >> On Mon, Jul 23, 2018 at 2:39 AM, 'Andrey Brukhno' via votca > >> >> <[email protected]> wrote: > >> >> > > >> >> > On Mon, Jul 23, 2018 at 2:22 AM, Christoph Junghans > >> >> > <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> On Sun, Jul 22, 2018 at 1:27 PM, 'Andrey Brukhno' via votca > >> >> >> <[email protected]> wrote: > >> >> >> > Thanks for your reply Christoph. > >> >> >> > > >> >> >> > I did check it, but inverse.log does not contain sufficient > info, > >> >> >> > see > >> >> >> > the > >> >> >> > extract below. > >> >> >> > I will be investigating it further to make sure I did not do > >> >> >> > anything > >> >> >> > stupid > >> >> >> > myself, although I haven't changed much for the test iteration > >> >> >> > except > >> >> >> > recompiling the tpr files for the newer version of Gromacs). > >> >> >> The " rdf calculation is already done" tells me that this is not > a > >> >> >> run > >> >> >> from scratch. > >> >> >> Try a fresh run, because some chance in the xml don't get > populated > >> >> >> until the next iteration step. > >> >> >> And grep for "csg_stat" (not rdf), you should see actually > command, > >> >> >> which got run, e.g.: > >> >> >> "Running critical command 'csg_stat --nt 4 --options settings.xml > >> >> >> --top topol.tpr --trj traj.xtc --begin 20 --first-frame 0' " > >> >> >> From that you can see if all the options are read as excepted. > >> >> >> > >> >> > > >> >> > I knew you were going to say this... > >> >> > As a matter of fact, there is no such message from > >> >> > calc_rdf_generic.sh, > >> >> > but > >> >> > instead: > >> >> > > >> >> > msg "Calculating rdfs with csg_stat using $tasks tasks" > >> >> > critical csg_stat --nt $tasks --options "$CSGXMLFILE" --top > >> >> > "$topol" > >> >> > --trj > >> >> > "$traj" --begin $equi_time --first-frame $first_frame > ${error_opts} > >> >> > ${maps:+--cg ${maps}} > >> >> > mark_done "rdf_calculation${suffix}" > >> >> Well the "critical" function includes the print statement: > >> >> critical() { > >> >> [[ $quiet = "no" ]] && echo "Running critical command '$*'" >&2 > >> >> "$@" || die "${FUNCNAME[0]}: '$*' failed" > >> >> } > >> >> So I am not sure when that got lost in your case. > >> >> > > >> >> > so I had to add this after the msg and before the csg_stat call: > >> >> > echo "Doing: critical csg_stat --nt $tasks --options > "$CSGXMLFILE" > >> >> > --top > >> >> > "$topol" --trj "$traj" --begin $equi_time --first-frame > $first_frame > >> >> > ${error_opts} ${maps:+--cg ${maps}}" > >> >> > > >> >> > This shows me that topol-rdf.tpr is actually used (extract from > >> >> > inverse.log): > >> >> > > >> >> > Make update for ibi > >> >> > Calculating rdfs with csg_stat using 4 tasks > >> >> > Doing: critical csg_stat --nt 4 --options > >> >> > > >> >> > > >> >> > > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cT/dopc_cg-int-map11.xml > > > >> >> > --top topol-rdf.tpr --trj traj.xtc --begin 0 --first-frame 0 > >> >> > --block-length > >> >> > 1000000 --ext dist.block --cg > >> >> > > >> >> > > >> >> > > /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cT/dopc_cg-map11.xml; > > > >> >> > begin to calculate distribution functions > >> >> > > >> >> > However, the distributions are still wrong, and I am lost now as > to > >> >> > which > >> >> > program to blame (again all the initial files are the same as in > my > >> >> > previous > >> >> > iteration). > >> >> > I compared the outputs of `gmx dump topol-rdf.tpr' and `gmx dump > >> >> > topol.tpr' > >> >> > generated with the new Gromacs and everything seems alright, the > >> >> > relevant > >> >> > differences in exclusion lists are there. > >> >> > The problem is also that the original calculation was done on a > >> >> > cluster > >> >> > more > >> >> > than a year ago, so I cannot reproduce exactly all the > installation > >> >> > setup, > >> >> > not on my desktop anyway. But I know that those calculations were > >> >> > correct > >> >> > and well converged. > >> >> > > >> >> > So something should have changed somewhere within the newly > installed > >> >> > Gromacs/VOTCA combination. > >> >> You can see what VOTCA has with: > >> >> csg_dump --top topol-rdf.tpr --cg dopc_cg-map11.xml > >> >> > >> >> Remember, if you are using a mapping file ("--cg" option) exclusions > >> >> from the topology file are ignored and the ones from the mapping > file > >> >> are used instead. > >> >> > >> >> Christoph > >> >> > > >> >> > Andrey > >> >> > > >> >> >> Christoph > >> >> >> > > >> >> >> > Andrey > >> >> >> > ----- > >> >> >> > $ grep rdf inverse.log > >> >> >> > ... > >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr > >> >> >> > table_??.xvg > >> >> >> > index.ndx* conf.gro*' > >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr > >> >> >> > table_??.xvg > >> >> >> > index.ndx* conf.gro*' > >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr > >> >> >> > table_??.xvg > >> >> >> > index.ndx* conf.gro*' > >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr > >> >> >> > table_??.xvg > >> >> >> > index.ndx* conf.gro*' > >> >> >> > cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr > >> >> >> > table_??.xvg > >> >> >> > index.ndx* conf.gro*' > >> >> >> > ‘sim_0/topol-rdf.tpr’ -> ‘./topol-rdf.tpr’ > >> >> >> > Calculating rdfs with csg_stat using 4 tasks > >> >> >> > Calculating average rdfs and its errors for interaction TCH-TCH > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction TCH-TCO > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction TCO-TCO > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction TCH-PO > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction TCH-NH > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction TCO-PO > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction TCO-NH > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction PO-PO > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction PO-NH > >> >> >> > rdf calculation is already done > >> >> >> > Calculating average rdfs and its errors for interaction NH-NH > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > On Sun, Jul 22, 2018 at 1:26 PM, Christoph Junghans > >> >> >> > <[email protected]> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> On Sat, Jul 21, 2018 at 9:01 PM, 'Andrey Brukhno' via votca > >> >> >> >> <[email protected]> wrote: > >> >> >> >> > Hello, > >> >> >> >> > > >> >> >> >> > I encountered a very strange (looks like legacy) issue with > >> >> >> >> > v1.4.1. > >> >> >> >> > In my earlier IBI iterations (with v1.3) I normally used > >> >> >> >> > special > >> >> >> >> > topol-rdf.tpr which was different from normal CG simulation > >> >> >> >> > topol.tpr > >> >> >> >> > by > >> >> >> >> > not > >> >> >> >> > including the intra-molecular pairs in the RDF stats (done > by > >> >> >> >> > csg_stat). > >> >> >> >> > I > >> >> >> >> > wanted to make sure my previous results are reproduced by > >> >> >> >> > v1.4.1, > >> >> >> >> > so > >> >> >> >> > copied > >> >> >> >> > all the initialisation files into a new directory and tried > to > >> >> >> >> > run > >> >> >> >> > once > >> >> >> >> > again. After the very first iteration I checked the obtained > >> >> >> >> > RDFs > >> >> >> >> > and > >> >> >> >> > they > >> >> >> >> > were all wrong - corresponding to normal topol.tpr > (including > >> >> >> >> > intra-pairs) > >> >> >> >> > and not topol-rdf.tpr (excluding intra-pairs)! > >> >> >> >> > > >> >> >> >> > I guess there might have been changes made to locations or > >> >> >> >> > names > >> >> >> >> > of > >> >> >> >> > tags > >> >> >> >> > in > >> >> >> >> > settings.xml between the VOTCA versions, but I could not > >> >> >> >> > identify > >> >> >> >> > those > >> >> >> >> > changes by referring to the most recent manual (both v1.4.1 > and > >> >> >> >> > development > >> >> >> >> > versions). > >> >> >> >> > > >> >> >> >> > Below is the relevant extract from my settings.xml file. > >> >> >> >> > > >> >> >> >> > Any clue, hint, advice are most welcome! > >> >> >> >> We haven't changes any tags in IBI since v1.4.1 and the > special > >> >> >> >> topology for IBI can be set by > cg.inverse.<sim_prog>.rdf.topol, > >> >> >> >> which > >> >> >> >> you did below. > >> >> >> >> You could try "grep csg_stat inverse.log" to figure out if the > >> >> >> >> right > >> >> >> >> topology got appended to the command line of csg_stat. > >> >> >> >> > >> >> >> >> Christoph > >> >> >> >> > > >> >> >> >> > Thanks in advance. > >> >> >> >> > Andrey > >> >> >> >> > > >> >> >> >> > ----------- > >> >> >> >> > <gromacs> > >> >> >> >> > <grompp> > >> >> >> >> > <bin>gmx grompp</bin> > >> >> >> >> > <!--opts></opts--> > >> >> >> >> > </grompp> > >> >> >> >> > <mdrun> > >> >> >> >> > <!--command>grmx-vtc-rerun.sh</command--> > >> >> >> >> > <!--command>mpirun -lsf mdrun_mpi</command--> > >> >> >> >> > <!--opts>-multidir sim_0 sim_1 sim_2 sim_3</opts--> > >> >> >> >> > <command>mpirun -n 4 mdrun_mpi</command> > >> >> >> >> > <opts>-multidir sim_0 sim_1 sim_2 sim_3 -tablep > >> >> >> >> > table.xvg > >> >> >> >> > -tableb > >> >> >> >> > table_b?.xvg table_a?.xvg table_d?.xvg</opts> > >> >> >> >> > </mdrun> > >> >> >> >> > <!--mdrun><opts>-nt 2</opts></mdrun--> > >> >> >> >> > <!-- trash so many frames at the beginning --> > >> >> >> >> > <equi_time>0</equi_time> > >> >> >> >> > <!-- grid for table*.xvg !--> > >> >> >> >> > <table_bins>0.001</table_bins> > >> >> >> >> > <!-- cut the potential at this value (gromacs bug) --> > >> >> >> >> > <pot_max>10000</pot_max> > >> >> >> >> > <!-- extend the tables to this value --> > >> >> >> >> > <table_end>2.5</table_end> > >> >> >> >> > <initial_configuration>maindir</initial_configuration> > >> >> >> >> > <rdf> > >> >> >> >> > <block_length><DESC>Length of the block for the > error > >> >> >> >> > analysis</DESC> > >> >> >> >> > 1000000 > >> >> >> >> > </block_length> > >> >> >> >> > <map><DESC>Special mapping file for rdf calculations > >> >> >> >> > needed > >> >> >> >> > for > >> >> >> >> > bonded interactions</DESC> > >> >> >> >> > dopc_cg-map11.xml > >> >> >> >> > </map> > >> >> >> >> > <topol><DESC> Gromacs binary topol (tpr) file to be > >> >> >> >> > used > >> >> >> >> > for > >> >> >> >> > csg_stat</DESC> > >> >> >> >> > topol-rdf.tpr > >> >> >> >> > </topol> > >> >> >> >> > <with_errors>yes<DESC>calculate error on the rdf: > >> >> >> >> > yes/no</DESC> > >> >> >> >> > </with_errors> > >> >> >> >> > </rdf> > >> >> >> >> > </gromacs> > >> >> >> >> > ----------- > >> >> >> >> > > >> >> >> >> > -- > >> >> >> >> > You received this message because you are subscribed to the > >> >> >> >> > Google > >> >> >> >> > Groups > >> >> >> >> > "votca" group. > >> >> >> >> > To unsubscribe from this group and stop receiving emails > from > >> >> >> >> > it, > >> >> >> >> > send > >> >> >> >> > an > >> >> >> >> > email to [email protected]. > >> >> >> >> > To post to this group, send email to [email protected]. > > >> >> >> >> > Visit this group at https://groups.google.com/group/votca. > >> >> >> >> > For more options, visit https://groups.google.com/d/optout. > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> -- > >> >> >> >> Christoph Junghans > >> >> >> >> Web: http://www.compphys.de > >> >> >> >> > >> >> >> >> -- > >> >> >> >> You received this message because you are subscribed to a > topic > >> >> >> >> in > >> >> >> >> the > >> >> >> >> Google Groups "votca" group. > >> >> >> >> To unsubscribe from this topic, visit > >> >> >> >> > https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe. > >> >> >> >> To unsubscribe from this group and all its topics, send an > email > >> >> >> >> to > >> >> >> >> [email protected]. > >> >> >> >> To post to this group, send email to [email protected]. > >> >> >> >> Visit this group at https://groups.google.com/group/votca. > >> >> >> >> For more options, visit https://groups.google.com/d/optout. > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > You received this message because you are subscribed to the > Google > >> >> >> > Groups > >> >> >> > "votca" group. > >> >> >> > To unsubscribe from this group and stop receiving emails from > it, > >> >> >> > send > >> >> >> > an > >> >> >> > email to [email protected]. > >> >> >> > To post to this group, send email to [email protected]. > >> >> >> > Visit this group at https://groups.google.com/group/votca. > >> >> >> > For more options, visit https://groups.google.com/d/optout. > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> Christoph Junghans > >> >> >> Web: http://www.compphys.de > >> >> >> > >> >> >> -- > >> >> >> You received this message because you are subscribed to a topic > in > >> >> >> the > >> >> >> Google Groups "votca" group. > >> >> >> To unsubscribe from this topic, visit > >> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe. > >> >> >> To unsubscribe from this group and all its topics, send an email > to > >> >> >> [email protected]. > >> >> >> To post to this group, send email to [email protected]. > >> >> >> Visit this group at https://groups.google.com/group/votca. > >> >> >> For more options, visit https://groups.google.com/d/optout. > >> >> > > >> >> > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > >> >> > Groups > >> >> > "votca" group. > >> >> > To unsubscribe from this group and stop receiving emails from it, > >> >> > send > >> >> > an > >> >> > email to [email protected]. > >> >> > To post to this group, send email to [email protected]. > >> >> > Visit this group at https://groups.google.com/group/votca. > >> >> > For more options, visit https://groups.google.com/d/optout. > >> >> > >> >> > >> >> > >> >> -- > >> >> Christoph Junghans > >> >> Web: http://www.compphys.de > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "votca" group. > >> > To unsubscribe from this group and stop receiving emails from it, > send > >> > an > >> > email to [email protected] <javascript:>. > >> > To post to this group, send email to [email protected] > <javascript:>. > >> > Visit this group at https://groups.google.com/group/votca. > >> > For more options, visit https://groups.google.com/d/optout. > >> > >> > >> > >> -- > >> Christoph Junghans > >> Web: http://www.compphys.de > >> > >> -- > >> You received this message because you are subscribed to a topic in the > >> Google Groups "votca" group. > >> To unsubscribe from this topic, visit > >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe. > >> To unsubscribe from this group and all its topics, send an email to > >> [email protected] <javascript:>. > >> To post to this group, send email to [email protected] > <javascript:>. > >> Visit this group at https://groups.google.com/group/votca. > >> For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "votca" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > To post to this group, send email to [email protected] > <javascript:>. > > Visit this group at https://groups.google.com/group/votca. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > Christoph Junghans > Web: http://www.compphys.de > -- You received this message because you are subscribed to the Google Groups "votca" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/votca. For more options, visit https://groups.google.com/d/optout.
