Re: file(1) now with seccomp support enabled

[Colin Watson]

On Sat, Jul 27, 2019 at 03:55:36AM +0200, Christoph Biedl wrote:
> LD_PRELOAD ruins your day. From the kernel's point of view there is no
> difference between a syscall coming from the actual application and one
> coming from the code hooked into it. And while the syscalls done by the
> first (i.e. file) are more or less known, the latter requires
> examination of each and every implementation and whitelisting
> everything. Eventually fakeroot-tcp, wishes to open sockets, something
> I certainly would not want to whitelist.

I ran into a ton of annoying problems like that when I added seccomp
filtering to man-db (the idea there being to limit what damage might be
done by potential bugs in groff and friends).  The worst difficulties
are from third-party programs that some people have installed: there are
a couple of apparently fairly popular Linux antivirus tools that work by
installing an LD_PRELOAD wrapper that talks to a private daemon using a
Unix-domain socket and/or a System V message queue; there's a VPN that
does something similar even though it really has no business existing at
this level or interfering with processes that have nothing to do with
networking; and there's the "snoopy" package in Debian that logs
messages to /dev/log on execve.

At the moment my compromise solution is to reluctantly open up the
minimum possible set of syscalls I could find that stopped people
sending me bug reports that were in fact caused by something injected
from outside my software, and to limit most of that to only those cases
where I've detected the relevant LD_PRELOAD wrappers as being present.

This is pretty unsatisfactory, but I haven't found a better alternative
("ignore the problem and tell users to uninstall the third-party
software in question after laboriously tracking down the exact cause of
the seccomp failure in each and every case" isn't a viable solution in
my mind).  I suppose I could interpose a wrapper executable where I've
forced ld.so into secure-execution mode somehow, but all the ways I know
of to do that involve conferring some additional privilege on it which
doesn't seem like the right way forward either.  Or maybe I could have a
wrapper executable and just unset LD_PRELOAD first.  All very annoying.

-- 
Colin Watson   [cjwat...@debian.org]

Re: file(1) now with seccomp support enabled

[Simon McVittie]

On Sat, 27 Jul 2019 at 10:01:41 +0200, Vincent Bernat wrote:
> Just a quick note: seccomp filters may need adaptations from one libc to
> another (and from one kernel to another as the libc may adapt to the
> current kernel). For example, with the introduction of "openat" syscall,
> the libc has started to use it for "open()" and the new syscall has to
> be whitelisted. On the other hand, if you start implementing seccomp
> filters late, you may have whitelisted only the "openat" syscall while
> older libc (or current libc running on older kernels) will invoke the
> "open" syscall.

Writing your whitelist in terms of groups of related syscalls collected
by some actively-maintained project, like the @basic-io, @ipc, etc. groups
understood by systemd's SystemCallFilter (see systemd.exec(5)), seems like
one way to mitigate this problem.

Another way to mitigate this is to split out the distrusted part of the
program into a helper subprocess that does as little I/O as possible,
and in particular sends input and output via pre-opened pipes or sockets
instead of opening things itself - after all, seccomp is named for
"secure computation", not "secure I/O" or "secure networking". However,
this is probably a lot easier to do for new code than as something that
can be retrofitted onto the assumptions of an existing codebase.

smcv

Re: file(1) now with seccomp support enabled

[Christoph Biedl]

Philipp Kern wrote...

> On 2019-07-27 10:01, Vincent Bernat wrote:
> > I am upstream for a project using seccomp since a long time and I have
> > never been comfortable to enable it in Debian for this reason. However,
> > they enable it in Gentoo and I get the occasional patches to update the
> > whitelist (I am not doing anything fancy).
>
> But technically it should be possible to test this in an autopkgtest, no? I
> don't think perfect has to be the enemy of good here, as long as we can
> detect breakage and remediate it afterwards?

Ayup, already working on this, for precisely that reason. There a
question releated I haven't worded yet, stay tuned.

Christoph


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Vincent Bernat]

 ❦ 28 juillet 2019 12:11 +02, Philipp Kern :

>> Just a quick note: seccomp filters may need adaptations from one libc
>> to another (and from one kernel to another as the libc may adapt to
>> the current kernel). For example, with the introduction of "openat"
>> syscall, the libc has started to use it for "open()" and the new
>> syscall has to be whitelisted. On the other hand, if you start
>> implementing seccomp filters late, you may have whitelisted only the
>> "openat" syscall while older libc (or current libc running on older
>> kernels) will invoke the "open" syscall.
>>
>> I am upstream for a project using seccomp since a long time and I have
>> never been comfortable to enable it in Debian for this reason. However,
>> they enable it in Gentoo and I get the occasional patches to update the
>> whitelist (I am not doing anything fancy).
>
> But technically it should be possible to test this in an autopkgtest,
> no? I don't think perfect has to be the enemy of good here, as long as
> we can detect breakage and remediate it afterwards?

Yes, but I was thinking to cases where you run kernel from oldstable
with stable for example. This is something currently allowed that may
not work anymore. I am just providing the information, I don't have a
strong opinion if seccomp should or shouldn't be enabled.
-- 
Terminate input by end-of-file or marker, not by count.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Philipp Kern]

On 2019-07-27 10:01, Vincent Bernat wrote:
Just a quick note: seccomp filters may need adaptations from one libc 
to

another (and from one kernel to another as the libc may adapt to the
current kernel). For example, with the introduction of "openat" 
syscall,

the libc has started to use it for "open()" and the new syscall has to
be whitelisted. On the other hand, if you start implementing seccomp
filters late, you may have whitelisted only the "openat" syscall while
older libc (or current libc running on older kernels) will invoke the
"open" syscall.

I am upstream for a project using seccomp since a long time and I have
never been comfortable to enable it in Debian for this reason. However,
they enable it in Gentoo and I get the occasional patches to update the
whitelist (I am not doing anything fancy).


But technically it should be possible to test this in an autopkgtest, 
no? I don't think perfect has to be the enemy of good here, as long as 
we can detect breakage and remediate it afterwards?


Technically you cannot use any non-vendored libraries when enabling 
seccomp if you reason about it this way. Practically it mostly works 
except sometimes when the filters need to be adjusted. And as you can 
see Gentoo deals with that just fine and we could accept some breakage 
in unstable too, as long as the migration of the breaking library is 
stopped until the fix for the dependencies is in.


Kind regards
Philipp Kern

Re: file(1) now with seccomp support enabled

[Christoph Biedl]

Philipp Kern wrote...

> That being said: It feels like if you face this situation, you could also
> fork off a binary with a clean environment (i.e. without LD_PRELOAD) and
> minimal dependencies and only protect that with seccomp. Of course you lose
> the integration point of LD_PRELOAD that others might want to use if you do
> that, in which case I guess one could offer a flag to skip that fork.

... and I'm back at a point where I've already been: The default has to 
be the secure way else it's not worth the time. So if application really
would break otherwise and can with some reason trade the security,
they'll either have to provide that flag (patching, patching), or 
file(1) would have to detect that situation (hacky, fragile).

> In terms of prior art SSH also forks off an unprivileged worker to handle
> network authentication in preauth and only seccomps that one rather than its
> main process. But it's also not doing the environment cleanup AFAICS.

Yeah, but as I already wrote here, this requires some sorts of IPC and
of lot of joys come with it.

> Kind regards and thanks for making all of us more secure! :)

Trying my best but my hopes are getting low.

Christoph


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Vincent Bernat]

 ❦ 19 juillet 2019 17:18 +02, Christoph Biedl :

> Upstream of the file package added seccomp support a while ago, and
> probably everyone with even a small concern about security will agree
> the file program, often being used on dubious or even doubtless
> malicious input, should use seccomp to make the attack surface smaller.
> However I refrained from enabling this feature back then just weeks
> before the buster freeze, in restrospect: indeed the right decision.
> Now this early moment in the bullseye development cycle is a good time,
> so there's version 1:5.37-2, accepted in unstable a few moments ago.

Hello,

Just a quick note: seccomp filters may need adaptations from one libc to
another (and from one kernel to another as the libc may adapt to the
current kernel). For example, with the introduction of "openat" syscall,
the libc has started to use it for "open()" and the new syscall has to
be whitelisted. On the other hand, if you start implementing seccomp
filters late, you may have whitelisted only the "openat" syscall while
older libc (or current libc running on older kernels) will invoke the
"open" syscall.

I am upstream for a project using seccomp since a long time and I have
never been comfortable to enable it in Debian for this reason. However,
they enable it in Gentoo and I get the occasional patches to update the
whitelist (I am not doing anything fancy).
-- 
Use the good features of a language; avoid the bad ones.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Jakub Wilk]

* Christoph Biedl , 2019-07-27, 03:55:

The file program should[citation needed] not write to any file


...except when using the -C option.

--
Jakub Wilk

Re: file(1) now with seccomp support enabled

[Vincas Dargis]

On 2019-07-27 04:55, Christoph Biedl wrote:

Eventually fakeroot-tcp, wishes to open sockets, something
I certainly would not want to whitelist.


In AppArmor case, "non-standard" use cases can be dealt with by editing 
`/etc/apparmor.d/local/usr.bin.foo`, adding any necessary rules (like allowing to mmap() some 
specific LD_PRELOAD library if needed for that use case), or by disabling profile completely as 
worst case workaround.



TTBOMK apparmor would not provide a sane solution for that problem.
There still might be another use case: The file program should[citation
needed] not write to any file. Reading however must be possible for
every item in the entire file system.


That's how I imagine AA profile for `file` - reads absolutely everything, and not expecting to 
write, unless to some temp file or similar. But as you said, citation needed.

Re: file(1) now with seccomp support enabled

[Philipp Kern]

On 2019-07-27 03:55, Christoph Biedl wrote:

Vincas Dargis wrote...


On 2019-07-26 18:59, Christoph Biedl wrote:
> > tl;dr: The file program in unstable is now built with seccomp support
> > enabled, expect breakage in some rather uncommon use cases.

Interesting, what are these uncommon use cases? Maybe we could confine 
it

with AppArmor instead, since we have it enabled by default?


LD_PRELOAD ruins your day. From the kernel's point of view there is no
difference between a syscall coming from the actual application and one
coming from the code hooked into it. And while the syscalls done by the
first (i.e. file) are more or less known, the latter requires
examination of each and every implementation and whitelisting
everything. Eventually fakeroot-tcp, wishes to open sockets, something
I certainly would not want to whitelist.


FWIW the same is true when you just link against libraries and they 
change their behavior. That makes things pretty brittle.


That being said: It feels like if you face this situation, you could 
also fork off a binary with a clean environment (i.e. without 
LD_PRELOAD) and minimal dependencies and only protect that with seccomp. 
Of course you lose the integration point of LD_PRELOAD that others might 
want to use if you do that, in which case I guess one could offer a flag 
to skip that fork.


In terms of prior art SSH also forks off an unprivileged worker to 
handle network authentication in preauth and only seccomps that one 
rather than its main process. But it's also not doing the environment 
cleanup AFAICS.


Kind regards and thanks for making all of us more secure! :)
Philipp Kern

Re: file(1) now with seccomp support enabled

[Christoph Biedl]

Vincas Dargis wrote...

> On 2019-07-26 18:59, Christoph Biedl wrote:
> > > tl;dr: The file program in unstable is now built with seccomp support
> > > enabled, expect breakage in some rather uncommon use cases.
>
> Interesting, what are these uncommon use cases? Maybe we could confine it
> with AppArmor instead, since we have it enabled by default?

LD_PRELOAD ruins your day. From the kernel's point of view there is no
difference between a syscall coming from the actual application and one
coming from the code hooked into it. And while the syscalls done by the
first (i.e. file) are more or less known, the latter requires
examination of each and every implementation and whitelisting
everything. Eventually fakeroot-tcp, wishes to open sockets, something
I certainly would not want to whitelist.

TTBOMK apparmor would not provide a sane solution for that problem.
There still might be another use case: The file program should[citation
needed] not write to any file. Reading however must be possible for
every item in the entire file system.

Christoph


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Vincas Dargis]

On 2019-07-26 18:59, Christoph Biedl wrote:

tl;dr: The file program in unstable is now built with seccomp support
enabled, expect breakage in some rather uncommon use cases.
Interesting, what are these uncommon use cases? Maybe we could confine it with AppArmor instead, 
since we have it enabled by default?

Re: file(1) now with seccomp support enabled

[Christoph Biedl]

Christoph Biedl wrote...

> tl;dr: The file program in unstable is now built with seccomp support
> enabled, expect breakage in some rather uncommon use cases.

Several issues popped up in the last days as a result of that change,
and in spite of some band-aiding to current implementation of seccomp in
the file program creates way more trouble than I am willing to ignore.
So, sadly, I've reverted seccomp support for the time being to avoid
further disruption of the bullseye development.

However, Helmut Grohne has suggested to confine only that part of the
code that is most likely susceptible to vulnerabilities, details in
#932762, and I agree this is possibly the better way to go. This
requires co-ordination with upstream and will take a bit of time.

Christoph


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Chris Lamb]

[Adding rb-gene...@lists.reproducible-builds.org to CC]

Hi Christoph,

> Overall, I'm just asking to keep an eye on possible breakage, also
> check the kernel log.

I noticed that there were a number of recent regressions in previously
reproducible Java packages being tested by the Reproducible Builds
project's CI platform which I could identify as being caused by our
strip-nondeterminism tool.

However, as there was a very recent change to some strip-nondeterminism
code that uses "monkey patching" I was predisposed to believe that was
the cause, but it eventually turned out to be the call to file(1)
missing a --no-sandbox parameter (where supported / appropriate).

It did not even occur to check my kernel log as you suggest — it was
only when quickly hacking in a:

override_dh_strip_non_determinism:
strace -eexecve -f dh_strip_nondeterminism

… to my test package that I figured the file(1) process was being
killed (without returning any output) with SIGCHLD that things were
perhaps lower-level in nature. This has been resolved in strip-
nondeterminism 1.3.0, uploaded this afternoon.

This mail is not a request for anything, but rather a general heads-up
for you and a way of "keyword stuffing" various terms the above
paragraphs into search indexes for the benefit of others looking for
perhaps-obscure issue like this in the future. It is also an implicit
thanks for pushing security hardening features. :)


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org  chris-lamb.co.uk
   `-

Re: file(1) now with seccomp support enabled [and 1 more messages]

[Ian Jackson]

Christoph Biedl writes ("file(1) now with seccomp support enabled"):
> tl;dr: The file program in unstable is now built with seccomp support
> enabled, expect breakage in some rather uncommon use cases.

Thanks for this work and for the heads-up.

PS: Did you really mean to send your first mail like thiks
  From: Christoph Biedl 
?  It seems rather odd :-).

Christoph Biedl writes ("seccomp woes (was: file(1) now with seccomp support 
enabled)"):
> Solutions I've seen (use codesearch to find examples):
> 
> * People don't care
> * People add a hard-coded list of archs into the dependency clause
>   like "libseccomp-dev [amd64 ...]"
> 
> The first I consider plain ignorant.

Quite.

> * Centralize the list of supported archs in the seccomp packages. By
>   either creating an empty libseccomp-dev for the archs where seccomp
>   is not supported, or by creating a "libseccomp-dev-dummy" for these.
>   In the latter case package maintainers would have to do a one-time
>   change of the build dependency into "libseccomp-dev |
>   libseccomp-dev-dummy" and can focus on other issues then.

Others have pointed out that for good reasons the buildds insist on
the first alternative.

But maybe we could create a meta package like this

  Package: libseccomp-dev-maybe
  Depends: libseccomp-dev [!unsupported arch list]
  Architecture: all

That would be available and installable on all architectures and you
could depend on that.  It would centralise the unsupported arch list
and could be generated easily by src:libseccomp-dev.

Can anyone see a problem with this idea ?

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: seccomp woes (was: file(1) now with seccomp support enabled)

[Simon McVittie]

On Sat, 20 Jul 2019 at 00:21:10 +0200, Christoph Biedl wrote:
> The build system of the file package uses autoconf to check for
> presence of the seccomp library and will just disable that feature if
> support is missing. But just adding "libseccomp-dev" will break the
> build on e.g. alpha for an unsatisfyable build dependency - I don't
> wish to simply ignore that. So I have to make sure lack of seccomp in
> these architectures does not break the build.

... unless seccomp is sufficiently important to your dependent package
that you would prefer for your dependent package not to be available
on non-seccomp architectures until seccomp becomes available there.

For file(1) it certainly seems better to consider seccomp to be a
hardening/quality-of-implementation thing and fall back to not having it;
for flatpak I decided it was better to have flatpak remain unavailable
on non-seccomp architectures, because there are some sandbox escapes
that are only prevented by it using seccomp. Both approaches can be valid,
depending on the dependent package.

smcv

Re: seccomp woes (was: file(1) now with seccomp support enabled)

[Marco d'Itri]

On Jul 20, Christoph Biedl  wrote:

> * Centralize the list of supported archs in the seccomp packages. By
>   either creating an empty libseccomp-dev for the archs where seccomp
>   is not supported, or by creating a "libseccomp-dev-dummy" for these.
>   In the latter case package maintainers would have to do a one-time
>   change of the build dependency into "libseccomp-dev |
>   libseccomp-dev-dummy" and can focus on other issues then.
I tried this with libsystemd-dummy, which creates a libsystemd-dev 
package on hurd and kfreebsd which allows to build unmodifed packages.
The package bit rot long ago and then nobody cared enough to fix it, so 
packages which unconditionally depend on libsystemd-dev nowadays just 
fail on these architectures.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

seccomp woes (was: file(1) now with seccomp support enabled)

[Christoph Biedl]

Russ Allbery wrote...

> Christoph Biedl  writes:
>
> > tl;dr: The file program in unstable is now built with seccomp support
> > enabled, expect breakage in some rather uncommon use cases.
>
> Thank you very much for doing this!  Here's hoping this sets a trend.  It
> will provide so much defense in depth against malicious files.

Thanks for the positive feedback. While I agree seccomp is something
nice to have, I'd like to share two two very different thoughts that
arose while doing this.


The first one is Debian-specific: Declaring build-dependencies on
libaries that are not available in all archs, like seccomp.

This is not at all specific for seccomp, but perhaps it's one of the
places where this problem is seen relatively often. So read "seccomp"
as "$seccomp", describing a library that does not exist in all
architectures.

The build system of the file package uses autoconf to check for
presence of the seccomp library and will just disable that feature if
support is missing. But just adding "libseccomp-dev" will break the
build on e.g. alpha for an unsatisfyable build dependency - I don't
wish to simply ignore that. So I have to make sure lack of seccomp in
these architectures does not break the build.

Solutions I've seen (use codesearch to find examples):

* People don't care
* People add a hard-coded list of archs into the dependency clause
  like "libseccomp-dev [amd64 ...]"

The first I consider plain ignorant.

The second puts work on each package maintainer who uses libseccomp in
the build dependencies: The list of supported archs may change, and
having to maintain this in many places is unrealiable and also stupid
work. Still it does the job - I am just looking for a better way.

Solutions I can think of:

* Centralize the list of supported archs in the seccomp packages. By
  either creating an empty libseccomp-dev for the archs where seccomp
  is not supported, or by creating a "libseccomp-dev-dummy" for these.
  In the latter case package maintainers would have to do a one-time
  change of the build dependency into "libseccomp-dev |
  libseccomp-dev-dummy" and can focus on other issues then.

* Add an always-satisfyable alternative clause, like
  "Build-Depends: libseccomp-dev | base-files".

  Yuck.

* Introduce a statement for relaxed build dependencies. In other words,
  a new "Build-Depends-Try:" or "Build-Recommends:" that will be tried
  to be satisfied, but failure to do so will emit a warning at most.

Honestly, the last one has a lot of charm since it means a one-time
effort only. That effort however is huge and includes convincing
several people to implement it.



The second is questioning whether seccomp is something feasible on the
big scale. The domain of a seccomp filter set is the application, and
the way more and more libraries might be linked in during development,
the more syscalls have to be whitelisted, defeating the idea of
seccomp. So in consequence you'll either create a lot of small programs
or several threads with different sets of whitelisted syscalls. In
either way a lot of IPC is needed which is time-consuming and
error-prone, in implementation, execution, and debugging.

Christoph


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Christoph Biedl]

Paul Gevers wrote...

> Hi Christoph,
> 
> On 19-07-2019 17:18, Christoph Biedl wrote:
> > tl;dr: The file program in unstable is now built with seccomp support
> > enabled, expect breakage in some rather uncommon use cases.
> 
> This probably warrants an entry in the bullseye release-notes. Should we
> already forward your original mail to the BTS for that, or do you care
> to provide better text once you learn better what breaks (and what not)?

For the time being I'd suggest to just take a note somewhere and
re-visit the situation in a year from now. Too many more changes might
come, hopefully for the better. So as of now, writing elaborated texts
about the situation might be wasted time.

Christoph


signature.asc
Description: PGP signature

Re: file(1) now with seccomp support enabled

[Paul Gevers]

Hi Christoph,

On 19-07-2019 17:18, Christoph Biedl wrote:
> tl;dr: The file program in unstable is now built with seccomp support
> enabled, expect breakage in some rather uncommon use cases.

This probably warrants an entry in the bullseye release-notes. Should we
already forward your original mail to the BTS for that, or do you care
to provide better text once you learn better what breaks (and what not)?

Paul



signature.asc
Description: OpenPGP digital signature

Re: file(1) now with seccomp support enabled

[Russ Allbery]

Christoph Biedl  writes:

> tl;dr: The file program in unstable is now built with seccomp support
> enabled, expect breakage in some rather uncommon use cases.

Thank you very much for doing this!  Here's hoping this sets a trend.  It
will provide so much defense in depth against malicious files.

-- 
Russ Allbery (r...@debian.org)   

file(1) now with seccomp support enabled

[Christoph Biedl]

tl;dr: The file program in unstable is now built with seccomp support
enabled, expect breakage in some rather uncommon use cases.

Hello,

Upstream of the file package added seccomp support a while ago, and
probably everyone with even a small concern about security will agree
the file program, often being used on dubious or even doubtless
malicious input, should use seccomp to make the attack surface smaller.
However I refrained from enabling this feature back then just weeks
before the buster freeze, in restrospect: indeed the right decision.
Now this early moment in the bullseye development cycle is a good time,
so there's version 1:5.37-2, accepted in unstable a few moments ago.

This however comes with a price: Some features are no longer available.
For example, inspecting the content of compressed files (disabled by
default, command-line parameters -z and -Z) is now supported for a few
compressions only: gzip (and friends, see libz), bzip2, lzma, xz.
Decompressing other formats requires invocation of external programs
which will lead to a program abort (SIGSYS).

Also, when running in LD_PRELOAD environments, that extra library may
use blacklisted syscalls. One example is fakeroot which caused breakage
in debhelper (#931985, already fixed). In both cases you should see a
log message in the kernel log then.

There is a workaround for such situations which is disabling seccomp,
command line parameter --no-sandbox.

But I have no idea about the impact this will cause. Checking all
packages that (install-)depend on file for usage of these parameters
turned out to be a fairly though job. Probably I've killed
codesearch.d.n a few times, the term "file" is just very generic :)
Some 53 binary packages have a dependency on the file package, two of
them (cloud-utils, cracklib2) are very likely affected and will receive
an extra bug report.

Overall, I'm just asking to keep an eye on possible breakage, also
check the kernel log. If you encounter one and can imagine a better
solution than simply disabling seccomp in that case, let me know via
the BTS.


Finally, a clarification: Applications that link libmagic instead of
calling the file executable are not affected by any of this. But the
respective program authors might consider enabling seccomp on their
own, for the above reason.

Cheers,
Christoph


signature.asc
Description: PGP signature