Re: [Tails-dev] What is *not* erased (after shutdown) with PAX_MEMORY_SANITIZE enabled?

[intrigeri]

Hi everybody!

Harlan Lieberman-Berg:
> Thanks for weighing in, PaX Team.  (And thank you for the awesome you
> and spender do on kernel security!)

+1 :)

> To summarize, it seems that we have a couple different options to choose
> from:

> * Switch to a dedicated microkernel that does a memory wipe, and kexec
>   into it.  This could be something custom, or an enhancement of the
>   preexisting solution.

>   Advantages are that it will (probably) give us the most clean wipe, as
>   we can reduce the amount of space the kernel takes up and drop all
>   functionality that's not absolutely needed.  On the negative side, it
>   will require continuing to support a separate codeline that's not
>   going to be reused elsewhere, limiting testing and development
>   effort.  It also requires us to reenable kexec functionality, which
>   exposes a risk of code injection unless we get signed kexec support.

> * Rely on PAX_MEMORY_SANITIZE.  This either takes the form of enhancing
>   cleaning memory on shutdown, or kexec'ing into the same version of the
>   kernel that's already running to rely on the buddy allocator clearing
>   everything again.

>   Definite pro in that it reduces the amount of code maintained
>   downstream by the Tails team to ~zero.  Cons are increased reliance on
>   more complex functionality in the kernel, and potentially relying on a
>   somewhat undocumented and unplanned functionality in the kernel.  (It
>   seems unlikely that it'll change any time without us noticing, but
>   it's possible.)

> * Do it in userspace.  Add functionality into the initramfs as
>   necessary to wipe memory, and simply run an abbreviated shutdown.

>   This lets us not have to deal with the potential for kexec's attack
>   surface, and is probably some medium amount of code between the above
>   two options.  Downside is that it potentially is less reliable in
>   terms of clearing memory than the other options, and is probably
>   slower time-to-first-bit-erased than the other options.

> Does that seem like a fair summary to everyone?

It does seem like it to me.

> If so, which path seems the best for us to move forward on?

I prefer the two first options (mostly because the third one is
essentially what we're already doing, and are trying to
improve/replace).

Not sure which one of those two yet, but the next thing to do in both
cases is the same: make it possible to allow kexec without disabling
GRKERNSEC_KMEM entirely.

I don't know what configuration interface would be best: move kexec
disabling out of GRKERNSEC_KMEM, to another kernel build-time config
setting? Leave it as part of GRKERNSEC_KMEM, but add a sysctl to allow
re-enabling kexec at runtime? pipacs, spender: what do you think?

Cheers,
-- 
intrigeri
___
Tails-dev mailing list
Tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev
To unsubscribe from this list, send an empty email to 
tails-dev-unsubscr...@boum.org.

Re: [Tails-dev] What is *not* erased (after shutdown) with PAX_MEMORY_SANITIZE enabled?

[Harlan Lieberman-Berg]

Harlan Lieberman-Berg  writes:
> It also requires us to reenable kexec functionality, which exposes a
> risk of code injection unless we get signed kexec support.

I just checked the kernel, and it seems that signed kexec functionality
was mainlined in 3.17.  So, strike that from the list of problems with
all the kexec dependent solutions.

-- 
Harlan Lieberman-Berg
~hlieberman
___
Tails-dev mailing list
Tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev
To unsubscribe from this list, send an empty email to 
tails-dev-unsubscr...@boum.org.

Re: [Tails-dev] What is *not* erased (after shutdown) with PAX_MEMORY_SANITIZE enabled?

[Harlan Lieberman-Berg]

Hello everyone!

Thanks for weighing in, PaX Team.  (And thank you for the awesome you
and spender do on kernel security!)

To summarize, it seems that we have a couple different options to choose
from:

* Switch to a dedicated microkernel that does a memory wipe, and kexec
  into it.  This could be something custom, or an enhancement of the
  preexisting solution.

  Advantages are that it will (probably) give us the most clean wipe, as
  we can reduce the amount of space the kernel takes up and drop all
  functionality that's not absolutely needed.  On the negative side, it
  will require continuing to support a separate codeline that's not
  going to be reused elsewhere, limiting testing and development
  effort.  It also requires us to reenable kexec functionality, which
  exposes a risk of code injection unless we get signed kexec support.

* Rely on PAX_MEMORY_SANITIZE.  This either takes the form of enhancing
  cleaning memory on shutdown, or kexec'ing into the same version of the
  kernel that's already running to rely on the buddy allocator clearing
  everything again.

  Definite pro in that it reduces the amount of code maintained
  downstream by the Tails team to ~zero.  Cons are increased reliance on
  more complex functionality in the kernel, and potentially relying on a
  somewhat undocumented and unplanned functionality in the kernel.  (It
  seems unlikely that it'll change any time without us noticing, but
  it's possible.)

* Do it in userspace.  Add functionality into the initramfs as
  necessary to wipe memory, and simply run an abbreviated shutdown.

  This lets us not have to deal with the potential for kexec's attack
  surface, and is probably some medium amount of code between the above
  two options.  Downside is that it potentially is less reliable in
  terms of clearing memory than the other options, and is probably
  slower time-to-first-bit-erased than the other options.

Does that seem like a fair summary to everyone?  If so, which path seems
the best for us to move forward on?

-- 
Harlan Lieberman-Berg
~hlieberman
___
Tails-dev mailing list
Tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev
To unsubscribe from this list, send an empty email to 
tails-dev-unsubscr...@boum.org.

Re: [Tails-dev] What is *not* erased (after shutdown) with PAX_MEMORY_SANITIZE enabled?

[PaX Team]

On 2 Jan 2017 at 15:27, intrigeri wrote:

Hi all,

first of all, i've put spender on CC as an interested party
(especially for the kexec vs. kmem hardening you're discussing
in another thread).

> We're considering dropping our current implementation, and relying on
> PAX_MEMORY_SANITIZE instead; I know that PAX_MEMORY_SANITIZE was not
> designed for this use case, but who knows:  it might be good enough :)

actually this isn't the first time this case has been raised and
cleaning memory on shutdown has been on my todo list ever since,
just not at a high enough priority.

> I understand that the result won't achieve perfect protection, but our
> current implementation doesn't either. I'd like to see numbers, but
> it's hard for me to compare experimentally both approaches: our QA
> about this relies on filling memory with a known pattern from
> userspace, which works fine to measure the efficiency of our current
> implementation, but of course PAX_MEMORY_SANITIZE will erase this
> known pattern from memory… So for now I'll stick to the theoretical
> level, and experimental measurements will come later.
> 
> 
> If I got it right, with PAX_MEMORY_SANITIZE enabled, then:
> 
>  * during the lifetime of a system, any memory allocated to
>a userspace program that terminates is erased;

correct but don't forget that 'allocated to userland' means only
anonymous memory pages that are mapped into userland memory,
everything else (page cache holding file content, dentry/inode/etc
metadata) is subject to different rules, see next.

>  * during the lifetime of a system, any kernel memory that's
>explicitly freed, e.g. with kfree(), is erased;

well, 'free' is a bit abstract concept to reason about at this
level ;).

the kernel has more than one way to allocate (and thus free) memory
and SANITIZE doesn't (directly) apply to all of them. in practice
most memory is managed by the buddy allocator and a few layers that
build on top of it (slab, vmalloc, etc). the first (and original)
layer where SANITIZE kicks in is the buddy allocator where memory
is managed in groups of pages. this becomes important for the layers
built on top of it since they can very well allocate/free memory at
their level without every freeing back to the buddy allocator. for
this reason a while ago Mathias Krause added support for sanitizing
at the slab layer as well (slab manages C level objects, you can
consider it as the kernel's equivalent of malloc) but for performance
reasons not all slabs are sanitized by default (for tails you'll want
to enable all of it i guess, see the config help on the boot time
command line option or just patch the code to change the default
which is probably safer for your case).

>  * on system shutdown, all processes are killed, and thus their memory
>is erased.

yes if (presumably :) you meant a clean shutdown (and subject to
the above details). i'm just pointing it out because in the panic
case you mentioned a clean shutdown may not be possible.

> So, what remains after system shutdown boils down to:
> 
>  * kernel memory allocated and then freed, during the lifetime of the
>system, through means that are not covered by PAX_MEMORY_SANITIZE:
>is there any such thing? I'm interested in a (possibly incomplete)
>list of these memory areas or allocation methods.

while the buddy allocator acts as the lowest layer of memory manager
for the kernel (so it sees everything as far as C code is concerned),
there's also boot time memory allocated that is kept around and i'm
pretty sure that some of it is never freed. there can also be
arch-specific memory reservations that may need special handling
(hugetlbfs, CMA, etc).

>  * kernel memory, that was still in use during shutdown, and that the
>kernel does not explicitly free: again, is there any such thing?

as hinted at above, there's lots of memory that is not process memory
and that will be kept around as long as something uses that memory (or
even when not). shutdown in linux isn't about undoing everything that
happened before, it's simply just another function to execute that
happens to never return. this leaves some memory in-use. just so you
can get an idea, look at /proc/slabinfo from your shutdown script and
count all the still allocated objects... now not all of them have
sensitive information that you care about, but some definitely do.

with all that said, let me comment on your current approach:

> Current Tails implementation
> 
> 
> Our current protection against this scenario is: when I unplug my
> Tails USB stick, an emergency shutdown procedure is triggered, that
> kexec's another kernel with some special command line parameter, and
> then the initramfs erases all the memory that's available from
> userspace.

i'd like to interject here and point out a less than obvious sideeffect
of SANITIZE (not really documented because the effect isn't relevant
for the original purpose of SANITIZE): when the kernel