Hi Gad,
Currently, plugins do now write any headers into the checkpoint image.
Instead, each plugin is a library, and therefore part of the memory image.
A plugin can save anything at all in its own memory in any format at all.
At the time of restart, the plugin will understand and use its own
internal data structure.
This is how it's done currently:
data in flight in sockets - <dmtcp>/lib/dmtcp/libdmtcp_ipc.so
(corresponding specifically to the source:
<dmtcp>/srcp/plugin/ipc/socket
and so on.
For the kernel state, different pieces are stored in different plugins.
Look at the directories under <dmtcp>/srcp/plugin for a guide
to what DMTCP internal plugins exist.
As for the outer headers from DMTCP and MTCP, those are indeed in
multiples of 4K. The purpose of these outer headers is purely
to help the "restart loader" (bin/mtcp_restart) to know where
are the start functions for restart, any high-level state carried over,
etc. So, the headers at the beginning of ckp_*.dmtcp are analogous
to the headers at the beginning of an ELF file, and the DMTCP area
headers are analogous to the section headers in an ELF file.
DMTCP has internal plugins with a specialized expertise, which
are analogous to ELF sections with a specialized purpose (e.g., ELF symbols
section). In the case of DMTCP, there is no ABI for the information
in a plugin. It's stored directly in C++ data structures, and the
compiler is free to lay it out in memory as the compiler desires.
This works because only that plugin needs to read the information.
I hope this helps.
Best,
- Gene
On Mon, Mar 30, 2015 at 03:56:02PM +0000, Gad, Ramy wrote:
> Hi Gene,
>
> Thank you for this information it is very helpful.
>
> I have noticed that "the header section" , "the header for each contiguous
> memory area" and 'the data for each contiguous memory area' are all
> multiples of 4K (which is the page size).
>
> Does this means that other headers (written by plugins) to "represent
> additional state in their memory, for the kernel state, state of the network,
> data in flight in sockets, etc." are also muliple of 4k ?
>
>
> This other headers are also included in the ckpt_*.dmtcp ?
>
> How big is this headers (how many pages) ?
>
>
> Looking forward for your reply.
>
> Best regards,
>
> Ramy Gad
> Johannes Gutenberg - Universität Mainz
> Zentrums für Datenverarbeitung (ZDV)
>
> Anselm-Franz-von-Bentzel-Weg 12
> 55128 Mainz
> Germany
> E-Mail: g...@uni-mainz.de
> Office Phone: +49-6131-39-26437
>
>
> ________________________________________
> From: Gene Cooperman <g...@ccs.neu.edu>
> Sent: Friday, March 27, 2015 8:20 PM
> To: Gad, Ramy
> Cc: Gene Cooperman; dmtcp-forum@lists.sourceforge.net
> Subject: Re: [Dmtcp-forum] How similar is DMTCP checkpoint file to a process
> core memory dump ?
>
> Hi Ramy,
> Unfortunately, the best document for that is the code for
> util/readdmtcp.sh. This will lead you to:
> src/mtcp/mtcp_restart.cpp with the flag --simulate, which will
> lead you to the function mtcp_simulateread().
> I would run this under GDB. It's less painful than searching for
> the headers in the .h file. So:
> cp ckpt_*.dmtcp > tmp.dmtcp.gz
> gunzip tmp.dmtcp.gz
> gdb --args bin/mtcp_restart --simulate tmp.dmtcp
> (gdb) b mtcp_simulateread
> (gdb) run
> You'll then want to do things like:
> (gdb) b main
> (gdb) run
> (gdb) b 258
> (gdb) c
> (gdb) p mtcpHdr # second header (how to read the memory areas)
> (gdb) b 284
> (gdb) p rinfo # first header (organization of ckpt image)
> # Then continue to loop, and read the detailed header of each memory area
> (gdb) ptype area
> (gdb) p area
>
> Best,
> - Gene
>
> On Fri, Mar 27, 2015 at 04:16:20PM +0000, Gad, Ramy wrote:
> > Hi,
> >
> > Is there any detail documentation that can explain how to decode the
> > ckpt_*.dmtcp file ?
> >
> > i.e. decode
> >
> > 1) the header section
> > 2) the header for each contiguous memory area (addr range, permissions,
> > etc.)
> > 3) the data for each contiguous memory area
> >
> > I want to know exactly where are the heap, the stack and dynamic libraries,
> > etc. in the ckpt_*.dmtcp file
> >
> > Having a documentation is a easier than reverse engineering the code :-)
> >
> > Thank you in advance for your reply.
> >
> > Best regards,
> >
> > Ramy Gad
> > Johannes Gutenberg - Universität Mainz
> > Zentrums für Datenverarbeitung (ZDV)
> >
> > Anselm-Franz-von-Bentzel-Weg 12
> > 55128 Mainz
> > Germany
> > E-Mail: g...@uni-mainz.de
> > Office Phone: +49-6131-39-26437
> >
> >
> > ________________________________________
> > From: Gene Cooperman <g...@ccs.neu.edu>
> > Sent: Wednesday, March 18, 2015 6:54 PM
> > To: Gad, Ramy
> > Cc: dmtcp-forum@lists.sourceforge.net
> > Subject: Re: [Dmtcp-forum] How similar is DMTCP checkpoint file to a
> > process core memory dump ?
> >
> > Hi Ramy,
> > DMTCP is very similar to a process core memory dumps, but it
> > uses its own internal format, with certain enhancements. It includes:
> > 1) a header section
> > 2) a header for each contiguous memory area (addr range, permissions, etc.)
> > 3) the data for each contiguous memory area
> >
> > A memory area with zero pages is saved specially to save space.
> >
> > If you want to look at a DMTCP ckpt image, do:
> > gzip -dc ckpt_*.dmtcp > tmp.dmtcp
> > <DMTCP_DIR>/util/readdmtcp.sh tmp.dmtcp
> >
> > You'll see that DMTCP also saves the memory of all resident libraries.
> > In particular, the DMTCP libraries (plugins) will save additional state
> > in their memory, for the kernel state, state of the network, data in
> > flight in sockets, etc.
> >
> > Best,
> > - Gene
> >
> >
> > On Wed, Mar 18, 2015 at 03:45:16PM +0000, Gad, Ramy wrote:
> > > Hi all,
> > >
> > >
> > > How similar is DMTCP checkpoint file to a process core memory dump ?
> > >
> > >
> > > What are the common info and what are the differences ?
> > >
> > >
> > > Best regards,
> > >
> > > Ramy Gad
> > > Johannes Gutenberg - Universität Mainz
> > > Zentrums für Datenverarbeitung (ZDV)
> > >
> > > Anselm-Franz-von-Bentzel-Weg 12
> > > 55128 Mainz
> > > Germany
> > > E-Mail: g...@uni-mainz.de<mailto:g...@uni-mainz.de>
> > > Office Phone: +49-6131-39-26437
> > >
> >
> > > ------------------------------------------------------------------------------
> > > Dive into the World of Parallel Programming The Go Parallel Website,
> > > sponsored
> > > by Intel and developed in partnership with Slashdot Media, is your hub
> > > for all
> > > things parallel software development, from weekly thought leadership
> > > blogs to
> > > news, videos, case studies, tutorials and more. Take a look and join the
> > > conversation now. http://goparallel.sourceforge.net/
> >
> > > _______________________________________________
> > > Dmtcp-forum mailing list
> > > Dmtcp-forum@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/dmtcp-forum
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Dmtcp-forum mailing list
Dmtcp-forum@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dmtcp-forum
