Basically you need to get the core dump of the segfault. Make sure the core size of your system is unlimited:
ulimit -c will show you the size limit of the core file, and ulimit -c unlimited sets it to be unlimited. You probably need to set the following environment variable: export DMTCP_ABORT_ON_FAILED_ASSERT=1 which tells dmtcp to generate a core when an assertion fails. Once you have the core file, you can use gdb to get the backtrace: gdb -c <core file>, or gdb <your original binary> <core file> On Tue, Jul 25, 2017 at 03:56:09PM -0400, Stas Vernon wrote: > Hi Jiajun, > > Its my own c++ code, thats using MPI library and some other libraries (e.g. > trillinos). It requires installation of several libraries for run. > Could you please advise how can I get backtrace of the segfault? > Thank you. > > On Mon, Jul 24, 2017 at 1:12 PM, Jiajun Cao <jia...@ccs.neu.edu> wrote: > > > What application were you running? If it is difficult to share the > > information of the binary, can you send us the backtrace of the > > segfault? > > > > Best, > > Jiajun > > > > On Mon, Jul 24, 2017 at 12:31:57AM -0400, Stas Vernon wrote: > > > Hi guys, > > > I'm getting 'Segmentation fault (core dumped)' error when running (in the > > > very beginning): > > > ./dmtcp_launch mycode > > > Tried both in Virtualbox linux (with master version from zip file) and in > > > interactive session on our cluster (i.e. no scheduler, just from > > terminal, > > > with 2.5.1 version) - same error. In both cases I didn't use mpirun that > > is > > > without MPI. Tried with mpirun also - same error. > > > Tried for test: > > > ./dmtcp_launch ls > > > - works. > > > Is there a way to troubleshoot? Thinking of Valgrind, though not sure > > what > > > should be the order of valgrind and dmtp_launch in command line. > > > Thank you. > > > > > > On Fri, Jul 21, 2017 at 10:28 PM, Rohan Garg <rohg...@ccs.neu.edu> > > wrote: > > > > > > > The PR 454 did make it to the 2.5.0 release but the PR 510 didn't. It > > > > should be a part of the next release -- 2.5.1. > > > > > > > > You could try the 2.5.0 release to see if it works. PR 510 was for > > > > a bug that someone had reported with their specific PBS environment; > > > > it shouldn't affect most common PBS setups. In order to get the > > > > patch corresponding to PR 510, you'd have to get the source directly > > > > from the 2.5 git branch. In either case, if see any issues, please > > > > feel free to write back to us. > > > > > > > > On Fri, Jul 21, 2017 at 10:18:56PM -0400, Stas Vernon wrote: > > > > > Thank you guys for reply! > > > > > From Gene's email it seems that current stable release 2.5.0 doesn't > > > > > contain the code fixing bug that Rohan describe, is this correct? > > > > > I installed 2.5.0 stable release only. Should I install the developer > > > > > version instead or wait for next stable release? > > > > > > > > > > On Fri, Jul 21, 2017 at 9:23 PM, Gene Cooperman <g...@ccs.neu.edu> > > > > wrote: > > > > > > > > > > > Hi Rohan, > > > > > > Thanks for your reply. > > > > > > Should we add your code to the upcoming release of DMTCP 2.5.1 > > > > > > with a label "experimental" in the README, and then say in the FAQ > > > > > > that we now support PBS on an experimental basis, with comments > > > > > > welcome? > > > > > > We've been delayed in getting the 2.5.1 release out, just due > > > > > > to lack of our time, but we definitely owe people the enhancements > > > > > > and bug fixes. In one week, I will have more time available. > > > > > > > > > > > > Best, > > > > > > - Gene > > > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Rohan Garg" <rohg...@ccs.neu.edu> > > > > > > To: "Stas Vernon" <stasvern...@gmail.com> > > > > > > Cc: dmtcp-forum@lists.sourceforge.net > > > > > > Sent: Friday, July 21, 2017 6:29:17 PM > > > > > > Subject: Re: [Dmtcp-forum] PBS scheduler support > > > > > > > > > > > > Hi Stas, > > > > > > > > > > > > You should be able to use DMTCP with PBS. We have worked with the > > > > > > PBS developers in the past to squash a few bugs related to > > DMTCP/PBS > > > > > > interaction (for example, see: https://github.com/dmtcp/ > > dmtcp/pull/454 > > > > , > > > > > > and https://github.com/dmtcp/dmtcp/pull/510). The reason we don't > > list > > > > > > it on the website is that the DMTCP/PBS integration has been tested > > > > > > well enough. > > > > > > > > > > > > You can view my PBS scripts here: > > > > > > > > > > > > - https://github.com/rohgarg/dmtcp-job-scripts/blob/master/ > > > > > > dmtcp_pbs_launch.job > > > > > > - https://github.com/rohgarg/dmtcp-job-scripts/blob/master/ > > > > > > dmtcp_pbs_restart.job > > > > > > > > > > > > Please let us know, if you face any issues. > > > > > > > > > > > > Thanks, > > > > > > Rohan > > > > > > > > > > > > On Thu, Jul 20, 2017 at 10:02:11PM -0400, Stas Vernon wrote: > > > > > > > Hello, > > > > > > > I'm new to DMTCP. Our cluster uses PBS scheduler, my code > > (written in > > > > > > C++) > > > > > > > uses MPI. A job is usually ran on several nodes. > > > > > > > FAQ says 'DMTCP supports SLURM and Torque', but PBS not > > mentioned. > > > > > > > Does that mean I cannot use it with PBS? > > > > > > > Or if still possible, could smbd post a simple example script for > > > > PBS? > > > > > > > Thank you very much! > > > > > > > > > > > > > ------------------------------------------------------------ > > > > > > ------------------ > > > > > > > Check out the vibrant tech community on one of the world's most > > > > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > > > > > > > > > > _______________________________________________ > > > > > > > Dmtcp-forum mailing list > > > > > > > Dmtcp-forum@lists.sourceforge.net > > > > > > > https://lists.sourceforge.net/lists/listinfo/dmtcp-forum > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > > > > ------------------ > > > > > > Check out the vibrant tech community on one of the world's most > > > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > > > _______________________________________________ > > > > > > Dmtcp-forum mailing list > > > > > > Dmtcp-forum@lists.sourceforge.net > > > > > > https://lists.sourceforge.net/lists/listinfo/dmtcp-forum > > > > > > > > > > > > > > > ------------------------------------------------------------ > > ------------------ > > > Check out the vibrant tech community on one of the world's most > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > > _______________________________________________ > > > Dmtcp-forum mailing list > > > Dmtcp-forum@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/dmtcp-forum > > > > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Dmtcp-forum mailing list Dmtcp-forum@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dmtcp-forum
