Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Lennart Poettering]

On Tue, 24.06.14 15:21, Lennart Poettering (lenn...@poettering.net) wrote:

 On Mon, 23.06.14 14:29, Dave Reisner (d...@falconindy.com) wrote:
 
   Anyway, I hope this makes sense.
   
   With these changes coredumpctl actually is now really useful and just
   works. I have thus dropped the systemd- prefix. We should probably
   start advertising it more.
  
  Are there plans to limit the size of the directory in any way? As is,
  the default setup is prone to a simple DoS attack as a non-root user:
  
while true; do bash -c 'kill -SEGV $$'; done
 
 Currently clean-up is done via tmpfiles-based aging. But yeah, you are
 right, we need something there that makes sure people cannot flood the
 directory with stuff. I have added this to the TODO list. It should be
 simple enough, we can take some inspiration from the journal vacuum
 logic, however modify it slightly since we probably don't want to to
 ratelimiting per-user/service, but I am not sure.

This is implemented now.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Lennart Poettering]

On Mon, 23.06.14 14:29, Dave Reisner (d...@falconindy.com) wrote:

  Anyway, I hope this makes sense.
  
  With these changes coredumpctl actually is now really useful and just
  works. I have thus dropped the systemd- prefix. We should probably
  start advertising it more.
 
 Are there plans to limit the size of the directory in any way? As is,
 the default setup is prone to a simple DoS attack as a non-root user:
 
   while true; do bash -c 'kill -SEGV $$'; done

Currently clean-up is done via tmpfiles-based aging. But yeah, you are
right, we need something there that makes sure people cannot flood the
directory with stuff. I have added this to the TODO list. It should be
simple enough, we can take some inspiration from the journal vacuum
logic, however modify it slightly since we probably don't want to to
ratelimiting per-user/service, but I am not sure.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Frederic Crozat]

Le vendredi 20 juin 2014 à 20:02 +0200, Lennart Poettering a écrit :

 2) I change the paths to store this in. I drop the coredumps in
/var/lib/systemd/coredump/ now. While the journal logs appear to be
something worth sharing across the network as logs; I am not
convinced that the coredumps would fit that category. 

How about storing those in /var/cache/systemd/coredump ? This would make
obvious they can be safely removed at any time..



-- 
Frederic Crozat
Project Manager Enterprise Desktop
SUSE

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Michael Biebl]

2014-06-23 10:28 GMT+02:00 Frederic Crozat fcro...@suse.com:
 Le vendredi 20 juin 2014 à 20:02 +0200, Lennart Poettering a écrit :

 2) I change the paths to store this in. I drop the coredumps in
/var/lib/systemd/coredump/ now. While the journal logs appear to be
something worth sharing across the network as logs; I am not
convinced that the coredumps would fit that category.

 How about storing those in /var/cache/systemd/coredump ? This would make
 obvious they can be safely removed at any time..

Data in /var/cache is supposed to be data which can be (easily)
regenerated. coredumps do not really fit that.

I think  /var/lib/systemd/coredump/ is fine for that purpose.

end_bike_shedding


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Dave Reisner]

On Fri, Jun 20, 2014 at 08:02:54PM +0200, Lennart Poettering wrote:
 On Tue, 14.05.13 19:09, Oleksii Shevchuk (alx...@gmail.com) wrote:
 
 Heya!
 
 Sorry for resurrecting this thread from last year. I never found the
 time to merge this, but I finally had a closer look and then sat down
 and tried to isolate out of it what I liked and what I didn't. I
 commited different patches for this though. Sorry for the long
 delay!
 
 So here's what is implemented in git now:
 
 a) There's a configuration file /etc/systemd/coredump.conf with some of
the options you proposed.
 
 b) We will now store coredumps outside of the journal by default, but
you can also place them in the journal only, or at both places.
 
 c) I hooked this thing up with elfutils' libdw, which gives us pretty,
native backtraces in the journal now, without involving gdb or
anything like that. Only a minimal (optional) dependency on libdw to
get them. And the best thing is that elfutils is actually maintained
and can read debuginfo files,  unlike some other options for stuff
like this (like libunwind).
 
 d) I hooked this up with ACLs so that a user can read but not change his
own coredumps stored in /var.

This all sounds great.

 What I didn't take: 
 
 1) the API to specify external programs for compressing or processing
the coredumps. I am really not too fond of doing things with invokign
external programs. THat's always chickening out in my eyes, avoiding
to solve the problems properly. By using elfutils' libdw we get the
backtrace feature nicely integrated now without invoking external
programs, I guess the need for PreprocessJournal= is hence redundant
with that. There's no support for compression though, but I'd be fine
with taking a simple patch for that that directly speaks to the xz
APIs. After all we link against the xz already. Of course this would
need support in both the coredump hook to transparently compress the
coredumps and in coredumpctl on the client side so that coredumpctl
gdb just works without manual decompression.
 
 2) I change the paths to store this in. I drop the coredumps in
/var/lib/systemd/coredump/ now. While the journal logs appear to be
something worth sharing across the network as logs; I am not
convinced that the coredumps would fit that category. 

The fact that this path is hardcoded is kind of lousy. See below...

 Anyway, I hope this makes sense.
 
 With these changes coredumpctl actually is now really useful and just
 works. I have thus dropped the systemd- prefix. We should probably
 start advertising it more.

Are there plans to limit the size of the directory in any way? As is,
the default setup is prone to a simple DoS attack as a non-root user:

  while true; do bash -c 'kill -SEGV $$'; done

Cheers,
Dave

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Lennart Poettering]

On Tue, 14.05.13 19:09, Oleksii Shevchuk (alx...@gmail.com) wrote:

Heya!

Sorry for resurrecting this thread from last year. I never found the
time to merge this, but I finally had a closer look and then sat down
and tried to isolate out of it what I liked and what I didn't. I
commited different patches for this though. Sorry for the long
delay!

So here's what is implemented in git now:

a) There's a configuration file /etc/systemd/coredump.conf with some of
   the options you proposed.

b) We will now store coredumps outside of the journal by default, but
   you can also place them in the journal only, or at both places.

c) I hooked this thing up with elfutils' libdw, which gives us pretty,
   native backtraces in the journal now, without involving gdb or
   anything like that. Only a minimal (optional) dependency on libdw to
   get them. And the best thing is that elfutils is actually maintained
   and can read debuginfo files,  unlike some other options for stuff
   like this (like libunwind).

d) I hooked this up with ACLs so that a user can read but not change his
   own coredumps stored in /var.

What I didn't take: 

1) the API to specify external programs for compressing or processing
   the coredumps. I am really not too fond of doing things with invokign
   external programs. THat's always chickening out in my eyes, avoiding
   to solve the problems properly. By using elfutils' libdw we get the
   backtrace feature nicely integrated now without invoking external
   programs, I guess the need for PreprocessJournal= is hence redundant
   with that. There's no support for compression though, but I'd be fine
   with taking a simple patch for that that directly speaks to the xz
   APIs. After all we link against the xz already. Of course this would
   need support in both the coredump hook to transparently compress the
   coredumps and in coredumpctl on the client side so that coredumpctl
   gdb just works without manual decompression.

2) I change the paths to store this in. I drop the coredumps in
   /var/lib/systemd/coredump/ now. While the journal logs appear to be
   something worth sharing across the network as logs; I am not
   convinced that the coredumps would fit that category. 

Anyway, I hope this makes sense.

With these changes coredumpctl actually is now really useful and just
works. I have thus dropped the systemd- prefix. We should probably
start advertising it more.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Michael Biebl]

2014-06-20 20:02 GMT+02:00 Lennart Poettering lenn...@poettering.net:
 So here's what is implemented in git now:

 a) There's a configuration file /etc/systemd/coredump.conf with some of
the options you proposed.

 b) We will now store coredumps outside of the journal by default, but
you can also place them in the journal only, or at both places.

 c) I hooked this thing up with elfutils' libdw, which gives us pretty,
native backtraces in the journal now, without involving gdb or
anything like that. Only a minimal (optional) dependency on libdw to
get them. And the best thing is that elfutils is actually maintained
and can read debuginfo files,  unlike some other options for stuff
like this (like libunwind).

 d) I hooked this up with ACLs so that a user can read but not change his
own coredumps stored in /var.

[..]

 With these changes coredumpctl actually is now really useful and just
 works. I have thus dropped the systemd- prefix. We should probably
 start advertising it more.

Nice stuff, thanks.
We currently disable coredump support in Debian but with those changes
it's time to revisit that.

Cheers,
Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Tollef Fog Heen]

]] Oleksii Shevchuk 

   For security reasons. It will be better if user will not have access to
   own cores by default (situation is the same with journal backend in
   upstream now).
 
  Why?
 
 When apps like gpg/lastpass/whatelse-with-passwords crashes, then user 
 probably
 will not be very happy to have all that data accessible as is.

They should either be sgid some group and then setgid to the user's
group or just use prctl(PR_SET_DUMPABLE, 0).

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Mantas Mikulėnas]

On Wed, May 22, 2013 at 9:22 AM, Tollef Fog Heen tfh...@err.no wrote:
 ]] Oleksii Shevchuk

   For security reasons. It will be better if user will not have access to
   own cores by default (situation is the same with journal backend in
   upstream now).

  Why?

 When apps like gpg/lastpass/whatelse-with-passwords crashes, then user 
 probably
 will not be very happy to have all that data accessible as is.

 They should either be sgid some group and then setgid to the user's
 group or just use prctl(PR_SET_DUMPABLE, 0).

`gpg` already disables core dumps by setting RLIMIT_CORE to zero.

--
Mantas Mikulėnas graw...@gmail.com
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Colin Walters]

On Tue, 2013-05-14 at 19:09 +0300, Oleksii Shevchuk wrote:
 Introduce configuration file: /etc/systemd/coredump.conf with
 configurable core dumps backend storages (journal/file/both/none),
 per storage size limits and preprocessing.

Who would want both and why?  Oh, though maybe you answer this here:

PreprocessFile=gzip -9 %i%o

Ideally for this case it'd be a pipe; that way we avoid writing a
potentially large file to disk only to write a smaller version again.

PreprocessJournal=gdb -nw --batch --quiet --silent --ex 'thread \
apply all bt' --core %i %e  %o

I can see this being quite cool.  Would require having both gdb and
debug symbols on the target system, but a lot of systems have that.

Though maybe instead of this, we allow people to write systemd units
that match core files.

 4. Copying optimizations removed for now. Now core always stored
to temporary folder, then optionaly preprocessed to temporary
files, then temporary files copied to storage.

Ok; we can add this later.

+  listitemparaIf unspecified 
literalsystemd-coredump/literal will drop privileges to nobody user
+  to store core to temporary folder and process it. 
When value is unset, literalsystemd-coredump/literal
+  will drop privileges to process owner uid/gid. 

Does the systemd config system really have a distinction between
unspecified and unset?  It looks to me we're always using
nobody.  

But git grep nobody says that nothing else in systemd is using the
nobody user.  Maybe it would make sense to just use the process owner
by default?   Why does preprocessing happen before dropping creds?

+memcpy(m, memory, memory_size);
+munmap(m, memory_size);
+close(fd);

Should check for errors here, and log if they fail.

+if (system(preprocess_journal))
+unlink(journal_tmp_filename);

In general you have a lack of error logging...it'd be nice to at least
log if the preprocess command fails (or coredumps!), because the kernel
won't coredump anything with an rlimit of 1 which systemd-coredump and
any process it spawns will have.


Anyways, in the big picture there's one feature we should probably have
before landing this, which is some sort of free-space detection like
journald has.  At the moment if this patch lands a repeatedly crashing
process could just fill up the disk quickly.  

At least if the system administrator has set up quotas, we should
probably ensure the files are charged against the dumping user's quota.
At the moment we're charging them to nobody, right?



___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Oleksii Shevchuk]

 Ideally for this case it'd be a pipe; that way we avoid writing a
 potentially large file to disk only to write a smaller version again.

I thought about that. With cores there is no way to do transparent
piping with postproc. All tools mmap cores, so in real world they will 
be dumped anyway..

We dump core before preprocessing to tmpfs, btw.

 Maybe it would make sense to just use the process owner
 by default? Why does preprocessing happen before dropping creds?

For security reasons. It will be better if user will not have access to
own cores by default (situation is the same with journal backend in
upstream now).

 Why does preprocessing happen before dropping creds

Processing done in separate process with dropped creds

 In general you have a lack of error logging...it'd be nice to at least
 log if the preprocess command fails (or coredumps!), because the kernel
 won't coredump anything with an rlimit of 1 which systemd-coredump and
 any process it spawns will have.

Yeah, I'll add some, if idea with temporary files will be accepted in
general.

 Anyways, in the big picture there's one feature we should probably have
 before landing this, which is some sort of free-space detection like
 journald has.  

Cores rotation mechainsm or just avoiding to write new cores?

 At least if the system administrator has set up quotas, we should
 probably ensure the files are charged against the dumping user's quota.
 At the moment we're charging them to nobody, right?

At the moment cores dumped and processed at tmpfs with nobody, then
copied from user creds to journal and with systemd-journal creds to fs.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Colin Walters]

On Tue, 2013-05-21 at 17:05 +0300, Oleksii Shevchuk wrote:

 For security reasons. It will be better if user will not have access to
 own cores by default (situation is the same with journal backend in
 upstream now).

Why?

 Yeah, I'll add some, if idea with temporary files will be accepted in
 general.

I hope so =)

 Cores rotation mechainsm or just avoiding to write new cores?

Probably configurable to either.  A nice default would be to, on space
pressure, keep only the latest crash per executable.

But at the moment, I guess what you have is not too bad because we at
least will honor quota for non-root users; my main concern was filling
up even the ext4 reserved space for example.


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Oleksii Shevchuk]

  For security reasons. It will be better if user will not have access to
  own cores by default (situation is the same with journal backend in
  upstream now).

 Why?

When apps like gpg/lastpass/whatelse-with-passwords crashes, then user probably
will not be very happy to have all that data accessible as is. Distros
like ubuntu even disable ptrace by default for non-root users

 But at the moment, I guess what you have is not too bad because we at
 least will honor quota for non-root users; my main concern was filling
 up even the ext4 reserved space for example.

Probably it isn't, because core fds created with root:systemd-journald, like
journal files
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] [PATCH] [RFCv7] Optionally save core dumps as plain files

[Oleksii Shevchuk]

Introduce configuration file: /etc/systemd/coredump.conf with
configurable core dumps backend storages (journal/file/both/none),
per storage size limits and preprocessing.

Default filestorage choosed as /run/log/coredump or /var/log/coredump
with next reason:
1. These files produced with systemd component
2. These files registered with journal

Main differences between v7 and v6:
1. As Colin suggested, create-then-drop policy used when storing to
   files.

2. {/var/,/run}/log/coredump/MACHINE-ID folder will be created with
   root:systemd-journal owner and drwxr-s-- permissions.

3. Trivial preprocessing support added. For example:

   PreprocessFile=gzip -9 %i%o
   PreprocessJournal=gdb -nw --batch --quiet --silent --ex 'thread \
   apply all bt' --core %i %e  %o

   This will save to file storage compressed cores and backtraces to
   journal COREDUMP= section.

4. Copying optimizations removed for now. Now core always stored
   to temporary folder, then optionaly preprocessed to temporary
   files, then temporary files copied to storage.

Weak places:
   coredump.c:495

   To prevent copying of large memory segments, I contruct virtual
   space manually. Probably it will not work as expected in some cases,
   so recomendations and comments needed..
---
 Makefile-man.am  |   1 +
 Makefile.am  |  14 +-
 man/coredump.conf.xml| 217 +++
 src/core/manager.c   |   1 +
 src/journal/coredump-gperf.gperf |  24 ++
 src/journal/coredump.c   | 764 ---
 src/journal/coredump.conf|  18 +
 src/journal/coredump.h   |  89 +
 src/journal/journald-server.h|   2 +-
 9 files changed, 992 insertions(+), 138 deletions(-)
 create mode 100644 man/coredump.conf.xml
 create mode 100644 src/journal/coredump-gperf.gperf
 create mode 100644 src/journal/coredump.conf
 create mode 100644 src/journal/coredump.h

diff --git a/Makefile-man.am b/Makefile-man.am
index 481423a..e3ff8dd 100644
--- a/Makefile-man.am
+++ b/Makefile-man.am
@@ -8,6 +8,7 @@ MANPAGES += \
man/hostname.5 \
man/journalctl.1 \
man/journald.conf.5 \
+   man/coredump.conf.5 \
man/kernel-command-line.7 \
man/kernel-install.8 \
man/locale.conf.5 \
diff --git a/Makefile.am b/Makefile.am
index eb85c8d..96e5bc3 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -2908,7 +2908,8 @@ nodist_systemunit_DATA += \
units/systemd-journal-flush.service
 
 dist_pkgsysconf_DATA += \
-   src/journal/journald.conf
+   src/journal/journald.conf \
+   src/journal/coredump.conf
 
 pkgconfiglib_DATA += \
src/journal/libsystemd-journal.pc
@@ -2927,10 +2928,12 @@ EXTRA_DIST += \
src/journal/libsystemd-journal.sym \
units/systemd-journald.service.in \
units/systemd-journal-flush.service.in \
-   src/journal/journald-gperf.gperf
+   src/journal/journald-gperf.gperf \
+   src/journal/coredump-gperf.gperf
 
 CLEANFILES += \
-   src/journal/journald-gperf.c
+   src/journal/journald-gperf.c \
+   src/journal/coredump-gperf.c
 
 # 
--
 if HAVE_MICROHTTPD
@@ -2975,10 +2978,13 @@ EXTRA_DIST += \
 # 
--
 if ENABLE_COREDUMP
 systemd_coredump_SOURCES = \
-   src/journal/coredump.c
+   src/shared/specifier.c \
+   src/journal/coredump.c \
+   src/journal/coredump-gperf.c
 
 systemd_coredump_LDADD = \
libsystemd-journal-internal.la \
+   libsystemd-id128-internal.la \
libsystemd-label.la \
libsystemd-shared.la
 
diff --git a/man/coredump.conf.xml b/man/coredump.conf.xml
new file mode 100644
index 000..491b9d0
--- /dev/null
+++ b/man/coredump.conf.xml
@@ -0,0 +1,217 @@
+?xml version='1.0'? !--*-nxml-*--
+?xml-stylesheet type=text/xsl 
href=http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl;?
+!DOCTYPE refentry PUBLIC -//OASIS//DTD DocBook XML V4.2//EN
+http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd;
+
+!--
+  This file is part of systemd.
+
+  Copyright 2013 Lennart Poettering
+ Oleksii Shevchuk
+
+  systemd is free software; you can redistribute it and/or modify it
+  under the terms of the GNU Lesser General Public License as published by
+  the Free Software Foundation; either version 2.1 of the License, or
+  (at your option) any later version.
+
+  systemd is distributed in the hope that it will be useful, but
+  WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+  Lesser General Public License for more details.
+
+  You should have received a copy of the GNU Lesser General Public License
+  along with systemd; If not, see http://www.gnu.org/licenses/.
+--
+
+refentry id=coredump.conf
+refentryinfo
+