Re: [systemd-devel] Skipping temporary coredump file during coredump generation
On Sun, Dec 04, 2016 at 06:24:45AM +, P.R.Dinesh wrote:
> Thank you for the insight.
> Shall I propose the patch with the following behaviour.
>
> Check if "backtrace is enabled" or "compression is not enabled or not
> supported"
> Then store the temporary uncompressed file.
> If not skip the uncompressed file generation.
>
> if( backtrace_enabled || ( compression_not_enabled ||
> compress_not_supported))
> {
> generate and store uncompressed coredump
> }
> else
> {
> skip uncompressed coredump file, compress the STDIN COREFILE.
> }
Sure. To make things even nicer, if backtrace_enabled=1, but we don't
have enough disk space to write the uncompressed image, we could force
backtrace_enabled=0 and fall back to the else branch.
Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Skipping temporary coredump file during coredump generation
Thank you for the insight.
Shall I propose the patch with the following behaviour.
Check if "backtrace is enabled" or "compression is not enabled or not
supported"
Then store the temporary uncompressed file.
If not skip the uncompressed file generation.
if( backtrace_enabled || ( compression_not_enabled ||
compress_not_supported))
{
generate and store uncompressed coredump
}
else
{
skip uncompressed coredump file, compress the STDIN COREFILE.
}
On Sat, 3 Dec 2016 at 23:46 Zbigniew Jędrzejewski-Szmek
wrote:
> On Fri, Dec 02, 2016 at 05:53:59PM +, P.R.Dinesh wrote:
> > During coredump generation I could find a temporary uncompressed file
> > getting generated from the corefile and written to the harddisk, later
> this
> > file is getting compressed (if compression was enabled) and then the file
> > coredump file is stored and this temporary file is removed.
>
> We write the uncompressed file to disk to generate the backtrace.
> If we aren't doing that, we can just compress on-the-fly.
>
> > I have a process whose memory consumption is typically around 5GB, it
> > generates around 13GB of uncompressed coredump ( coredump_filter = 0x33).
> > Later this file is compressed to 20MB and the uncompressed file is
> removed.
> > I have set the
> > ProcessSizeMax=16GB
> > ExternalSizeMax= 16GB
> > But sometimes my disk doesn't have sufficient space to store this
> temporary
> > file, hence systemd-coredump aborts the coredump processing.
> >
> > Is it possible to skip this temporary file generation and generate the
> > compressed file directly from the Corefile passed through STDIN?
>
> We could be smarter, and if we see that there isn't enough disk space
> to store the uncompressed core file, skip this step, and immediately
> try to write compressed data. I don't know how complicated that would be:
> we write the core either to the file or to the journal, so the code is
> pretty complicated. Patches are welcome ;)
>
> Zbyszek
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Skipping temporary coredump file during coredump generation
On Fri, Dec 02, 2016 at 05:53:59PM +, P.R.Dinesh wrote: > During coredump generation I could find a temporary uncompressed file > getting generated from the corefile and written to the harddisk, later this > file is getting compressed (if compression was enabled) and then the file > coredump file is stored and this temporary file is removed. We write the uncompressed file to disk to generate the backtrace. If we aren't doing that, we can just compress on-the-fly. > I have a process whose memory consumption is typically around 5GB, it > generates around 13GB of uncompressed coredump ( coredump_filter = 0x33). > Later this file is compressed to 20MB and the uncompressed file is removed. > I have set the > ProcessSizeMax=16GB > ExternalSizeMax= 16GB > But sometimes my disk doesn't have sufficient space to store this temporary > file, hence systemd-coredump aborts the coredump processing. > > Is it possible to skip this temporary file generation and generate the > compressed file directly from the Corefile passed through STDIN? We could be smarter, and if we see that there isn't enough disk space to store the uncompressed core file, skip this step, and immediately try to write compressed data. I don't know how complicated that would be: we write the core either to the file or to the journal, so the code is pretty complicated. Patches are welcome ;) Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Skipping temporary coredump file during coredump generation
During coredump generation I could find a temporary uncompressed file getting generated from the corefile and written to the harddisk, later this file is getting compressed (if compression was enabled) and then the file coredump file is stored and this temporary file is removed. I have a process whose memory consumption is typically around 5GB, it generates around 13GB of uncompressed coredump ( coredump_filter = 0x33). Later this file is compressed to 20MB and the uncompressed file is removed. I have set the ProcessSizeMax=16GB ExternalSizeMax= 16GB But sometimes my disk doesn't have sufficient space to store this temporary file, hence systemd-coredump aborts the coredump processing. Is it possible to skip this temporary file generation and generate the compressed file directly from the Corefile passed through STDIN? Is there any impact on doing so? I tried modifying the coredump.c and was able to get this compressed file directly from the core file. But I am not sure on whether it will fail on some scenarios or I am missing some important data. Also please let me know if my understanding of systemd coredump behaviour is wrong. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
