Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Tue, Feb 20, 2018 at 9:16 AM, Igor Stoppawrote: > > > On 13/02/18 20:10, Laura Abbott wrote: >> On 02/13/2018 07:20 AM, Igor Stoppa wrote: >>> Why alterations of page properties are not considered a risk and the >>> physmap is? >>> And how would it be easier (i suppose) to attack the latter? >> >> Alterations are certainly a risk but with the physmap the >> mapping is already there. Find the address and you have >> access vs. needing to actually modify the properties >> then do the access. I could also be complete off base >> on my threat model here so please correct me if I'm >> wrong. > > It's difficult for me to comment on this without knowing *how* the > attack would be performed, in your model. > > Ex: my expectation is that the attacked has R/W access to kernel data > and has knowledge of the location of static variables. > > This is not just a guess, but a real-life scenario, found in attacks > that, among other things, are capable of disabling SELinux, to proceed > toward gaining full root capability. > > At that point, I think that variables which are allocated dynamically, > in vmalloc address space, are harder to locate, because of the virtual > mapping and the randomness of the address chosen (this I have not > confirmed yet, but I suppose there is some randomness in picking the > address to assign to a certain allocation request to vmalloc, otherwise, > it could be added). Machine-to-machine runtime variation certainly affects the mapping location, but for early boot allocations, these become surprisingly deterministic, especially across similar hardware/memory layouts (both the virtmap and physmap locations). However, using CONFIG_RANDOMIZE_MEMORY makes it MUCH more difficult. (Note that RANDOMIZE_BASE on arm64 effectively includes RANDOMIZE_MEMORY, as it uses the entropy for multiple base offsets, including the physmap, IIRC.) >> I think your other summaries are good points though >> and should go in the cover letter. > > Ok, I'm just afraid it risks becoming a lengthy dissertation :-) It's rare to have anyone say "your commit log is too long". :) -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Tue, Feb 20, 2018 at 9:16 AM, Igor Stoppa wrote: > > > On 13/02/18 20:10, Laura Abbott wrote: >> On 02/13/2018 07:20 AM, Igor Stoppa wrote: >>> Why alterations of page properties are not considered a risk and the >>> physmap is? >>> And how would it be easier (i suppose) to attack the latter? >> >> Alterations are certainly a risk but with the physmap the >> mapping is already there. Find the address and you have >> access vs. needing to actually modify the properties >> then do the access. I could also be complete off base >> on my threat model here so please correct me if I'm >> wrong. > > It's difficult for me to comment on this without knowing *how* the > attack would be performed, in your model. > > Ex: my expectation is that the attacked has R/W access to kernel data > and has knowledge of the location of static variables. > > This is not just a guess, but a real-life scenario, found in attacks > that, among other things, are capable of disabling SELinux, to proceed > toward gaining full root capability. > > At that point, I think that variables which are allocated dynamically, > in vmalloc address space, are harder to locate, because of the virtual > mapping and the randomness of the address chosen (this I have not > confirmed yet, but I suppose there is some randomness in picking the > address to assign to a certain allocation request to vmalloc, otherwise, > it could be added). Machine-to-machine runtime variation certainly affects the mapping location, but for early boot allocations, these become surprisingly deterministic, especially across similar hardware/memory layouts (both the virtmap and physmap locations). However, using CONFIG_RANDOMIZE_MEMORY makes it MUCH more difficult. (Note that RANDOMIZE_BASE on arm64 effectively includes RANDOMIZE_MEMORY, as it uses the entropy for multiple base offsets, including the physmap, IIRC.) >> I think your other summaries are good points though >> and should go in the cover letter. > > Ok, I'm just afraid it risks becoming a lengthy dissertation :-) It's rare to have anyone say "your commit log is too long". :) -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Tue, Feb 20, 2018 at 8:28 AM, Igor Stoppawrote: > > > On 14/02/18 21:29, Kees Cook wrote: >> On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: > > [...] > >>> Kernel code should be fine, if it isn't that is a bug that should be >>> fixed. Modules yes are not fully protected. The conclusion from past >> >> I think that's a pretty serious problem: we can't have aliases with >> mismatched permissions; this degrades a deterministic protection >> (read-only) to a probabilistic protection (knowing where the alias of >> a target is mapped). Having an attack be "needs some info leaks" >> instead of "need execution control to change perms" is a much lower >> bar, IMO. > > Why "need execution control to change permission"? > Or, iow, what does it mean exactly? > ROP/JOP? Data-oriented control flow hijack? Right, I mean, if an attacker has already gained execute control, they can just call the needed functions to change memory permissions. But that isn't needed if there is a mismatch between physmap and virtmap: i.e. they can write to the physmap without needing to change perms first. > One can argue that this sort of R/W activity probably does require some > form of execution control, but AFAIK, the only way to to prevent it, is > to have CFI - btw, is there any standardization in that sense? I meant that I don't want a difference in protection between physmap and virtmap. I'd like to be able to reason the smae about the exposures in either. > So, from my (pessimistic?) perspective, the best that can be hoped for, > is to make it much harder to figure out where the data is located. > > Virtual mapping has this side effect, compared to linear mapping. Right, this is good, for sure. No complaints there at all. It's why I think pmalloc and arm64 physmap perms are separate issues. -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Tue, Feb 20, 2018 at 8:28 AM, Igor Stoppa wrote: > > > On 14/02/18 21:29, Kees Cook wrote: >> On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: > > [...] > >>> Kernel code should be fine, if it isn't that is a bug that should be >>> fixed. Modules yes are not fully protected. The conclusion from past >> >> I think that's a pretty serious problem: we can't have aliases with >> mismatched permissions; this degrades a deterministic protection >> (read-only) to a probabilistic protection (knowing where the alias of >> a target is mapped). Having an attack be "needs some info leaks" >> instead of "need execution control to change perms" is a much lower >> bar, IMO. > > Why "need execution control to change permission"? > Or, iow, what does it mean exactly? > ROP/JOP? Data-oriented control flow hijack? Right, I mean, if an attacker has already gained execute control, they can just call the needed functions to change memory permissions. But that isn't needed if there is a mismatch between physmap and virtmap: i.e. they can write to the physmap without needing to change perms first. > One can argue that this sort of R/W activity probably does require some > form of execution control, but AFAIK, the only way to to prevent it, is > to have CFI - btw, is there any standardization in that sense? I meant that I don't want a difference in protection between physmap and virtmap. I'd like to be able to reason the smae about the exposures in either. > So, from my (pessimistic?) perspective, the best that can be hoped for, > is to make it much harder to figure out where the data is located. > > Virtual mapping has this side effect, compared to linear mapping. Right, this is good, for sure. No complaints there at all. It's why I think pmalloc and arm64 physmap perms are separate issues. -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 13/02/18 20:10, Laura Abbott wrote: > On 02/13/2018 07:20 AM, Igor Stoppa wrote: >> Why alterations of page properties are not considered a risk and the physmap >> is? >> And how would it be easier (i suppose) to attack the latter? > > Alterations are certainly a risk but with the physmap the > mapping is already there. Find the address and you have > access vs. needing to actually modify the properties > then do the access. I could also be complete off base > on my threat model here so please correct me if I'm > wrong. It's difficult for me to comment on this without knowing *how* the attack would be performed, in your model. Ex: my expectation is that the attacked has R/W access to kernel data and has knowledge of the location of static variables. This is not just a guess, but a real-life scenario, found in attacks that, among other things, are capable of disabling SELinux, to proceed toward gaining full root capability. At that point, I think that variables which are allocated dynamically, in vmalloc address space, are harder to locate, because of the virtual mapping and the randomness of the address chosen (this I have not confirmed yet, but I suppose there is some randomness in picking the address to assign to a certain allocation request to vmalloc, otherwise, it could be added). > I think your other summaries are good points though > and should go in the cover letter. Ok, I'm just afraid it risks becoming a lengthy dissertation :-) -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 13/02/18 20:10, Laura Abbott wrote: > On 02/13/2018 07:20 AM, Igor Stoppa wrote: >> Why alterations of page properties are not considered a risk and the physmap >> is? >> And how would it be easier (i suppose) to attack the latter? > > Alterations are certainly a risk but with the physmap the > mapping is already there. Find the address and you have > access vs. needing to actually modify the properties > then do the access. I could also be complete off base > on my threat model here so please correct me if I'm > wrong. It's difficult for me to comment on this without knowing *how* the attack would be performed, in your model. Ex: my expectation is that the attacked has R/W access to kernel data and has knowledge of the location of static variables. This is not just a guess, but a real-life scenario, found in attacks that, among other things, are capable of disabling SELinux, to proceed toward gaining full root capability. At that point, I think that variables which are allocated dynamically, in vmalloc address space, are harder to locate, because of the virtual mapping and the randomness of the address chosen (this I have not confirmed yet, but I suppose there is some randomness in picking the address to assign to a certain allocation request to vmalloc, otherwise, it could be added). > I think your other summaries are good points though > and should go in the cover letter. Ok, I'm just afraid it risks becoming a lengthy dissertation :-) -- igor
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 14/02/18 21:29, Kees Cook wrote: > On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbottwrote: [...] >> Kernel code should be fine, if it isn't that is a bug that should be >> fixed. Modules yes are not fully protected. The conclusion from past > > I think that's a pretty serious problem: we can't have aliases with > mismatched permissions; this degrades a deterministic protection > (read-only) to a probabilistic protection (knowing where the alias of > a target is mapped). Having an attack be "needs some info leaks" > instead of "need execution control to change perms" is a much lower > bar, IMO. Why "need execution control to change permission"? Or, iow, what does it mean exactly? ROP/JOP? Data-oriented control flow hijack? Unless I misunderstand the meaning of "need execution control", I think that "need write capability to arbitrary data address" should be sufficient, albeit uncomfortable to use. OTOH, "need read/write capability from/to arbitrary data address" would be enough, I think, assuming that one knows the offset where to write to - but that information could be inferred, for example, by scanning the memory for known patterns. IMHO the attack surface is so vast that it's not unreasonable to expect that it will be possible to fish out means to perform arbitrary R/W into kernel address space. Ex: some more recent/less tested driver. One can argue that this sort of R/W activity probably does require some form of execution control, but AFAIK, the only way to to prevent it, is to have CFI - btw, is there any standardization in that sense? So, from my (pessimistic?) perspective, the best that can be hoped for, is to make it much harder to figure out where the data is located. Virtual mapping has this side effect, compared to linear mapping. But, once easier attack targets are removed, I suspect the page mapping will become the next target. -- igor
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 14/02/18 21:29, Kees Cook wrote: > On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: [...] >> Kernel code should be fine, if it isn't that is a bug that should be >> fixed. Modules yes are not fully protected. The conclusion from past > > I think that's a pretty serious problem: we can't have aliases with > mismatched permissions; this degrades a deterministic protection > (read-only) to a probabilistic protection (knowing where the alias of > a target is mapped). Having an attack be "needs some info leaks" > instead of "need execution control to change perms" is a much lower > bar, IMO. Why "need execution control to change permission"? Or, iow, what does it mean exactly? ROP/JOP? Data-oriented control flow hijack? Unless I misunderstand the meaning of "need execution control", I think that "need write capability to arbitrary data address" should be sufficient, albeit uncomfortable to use. OTOH, "need read/write capability from/to arbitrary data address" would be enough, I think, assuming that one knows the offset where to write to - but that information could be inferred, for example, by scanning the memory for known patterns. IMHO the attack surface is so vast that it's not unreasonable to expect that it will be possible to fish out means to perform arbitrary R/W into kernel address space. Ex: some more recent/less tested driver. One can argue that this sort of R/W activity probably does require some form of execution control, but AFAIK, the only way to to prevent it, is to have CFI - btw, is there any standardization in that sense? So, from my (pessimistic?) perspective, the best that can be hoped for, is to make it much harder to figure out where the data is located. Virtual mapping has this side effect, compared to linear mapping. But, once easier attack targets are removed, I suspect the page mapping will become the next target. -- igor
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 2:13 PM, Tycho Andersenwrote: > On Wed, Feb 14, 2018 at 11:48:38AM -0800, Kees Cook wrote: >> On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: >> > fixed. Modules yes are not fully protected. The conclusion from past >> > experience has been that we cannot safely break down larger page sizes >> > at runtime like x86 does. We could theoretically >> > add support for fixing up the alias if PAGE_POISONING is enabled but >> > I don't know who would actually use that in production. Performance >> > is very poor at that point. >> >> XPFO forces 4K pages on the physmap[1] for similar reasons. I have no >> doubt about performance changes, but I'd be curious to see real >> numbers. Did anyone do benchmarks on just the huge/4K change? (Without >> also the XPFO overhead?) >> >> If this, XPFO, and PAGE_POISONING all need it, I think we have to >> start a closer investigation. :) > > I haven't but it shouldn't be too hard. What benchmarks are you > thinking? Unless I'm looking at some specific micro benchmark, I tend to default to looking at kernel build benchmarks but that gets pretty noisy. Laura regularly uses hackbench, IIRC. I'm not finding the pastebin I had for that, though. I wonder if we need a benchmark subdirectory in tools/testing/, so we could collect some of these common tools? All benchmarks are terrible, but at least we'd have the same terrible benchmarks. :) -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 2:13 PM, Tycho Andersen wrote: > On Wed, Feb 14, 2018 at 11:48:38AM -0800, Kees Cook wrote: >> On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: >> > fixed. Modules yes are not fully protected. The conclusion from past >> > experience has been that we cannot safely break down larger page sizes >> > at runtime like x86 does. We could theoretically >> > add support for fixing up the alias if PAGE_POISONING is enabled but >> > I don't know who would actually use that in production. Performance >> > is very poor at that point. >> >> XPFO forces 4K pages on the physmap[1] for similar reasons. I have no >> doubt about performance changes, but I'd be curious to see real >> numbers. Did anyone do benchmarks on just the huge/4K change? (Without >> also the XPFO overhead?) >> >> If this, XPFO, and PAGE_POISONING all need it, I think we have to >> start a closer investigation. :) > > I haven't but it shouldn't be too hard. What benchmarks are you > thinking? Unless I'm looking at some specific micro benchmark, I tend to default to looking at kernel build benchmarks but that gets pretty noisy. Laura regularly uses hackbench, IIRC. I'm not finding the pastebin I had for that, though. I wonder if we need a benchmark subdirectory in tools/testing/, so we could collect some of these common tools? All benchmarks are terrible, but at least we'd have the same terrible benchmarks. :) -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:48:38AM -0800, Kees Cook wrote: > On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbottwrote: > > fixed. Modules yes are not fully protected. The conclusion from past > > experience has been that we cannot safely break down larger page sizes > > at runtime like x86 does. We could theoretically > > add support for fixing up the alias if PAGE_POISONING is enabled but > > I don't know who would actually use that in production. Performance > > is very poor at that point. > > XPFO forces 4K pages on the physmap[1] for similar reasons. I have no > doubt about performance changes, but I'd be curious to see real > numbers. Did anyone do benchmarks on just the huge/4K change? (Without > also the XPFO overhead?) > > If this, XPFO, and PAGE_POISONING all need it, I think we have to > start a closer investigation. :) I haven't but it shouldn't be too hard. What benchmarks are you thinking? Tycho
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:48:38AM -0800, Kees Cook wrote: > On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: > > fixed. Modules yes are not fully protected. The conclusion from past > > experience has been that we cannot safely break down larger page sizes > > at runtime like x86 does. We could theoretically > > add support for fixing up the alias if PAGE_POISONING is enabled but > > I don't know who would actually use that in production. Performance > > is very poor at that point. > > XPFO forces 4K pages on the physmap[1] for similar reasons. I have no > doubt about performance changes, but I'd be curious to see real > numbers. Did anyone do benchmarks on just the huge/4K change? (Without > also the XPFO overhead?) > > If this, XPFO, and PAGE_POISONING all need it, I think we have to > start a closer investigation. :) I haven't but it shouldn't be too hard. What benchmarks are you thinking? Tycho
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 02/14/2018 11:28 AM, Ard Biesheuvel wrote: On 14 February 2018 at 19:06, Laura Abbottwrote: On 02/13/2018 01:43 PM, Kees Cook wrote: On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING does use 4K pages which could be adjusted at runtime. So yes, you are right we would have physmap exposure on arm64 as well. Errr, so that means even modules and kernel code are writable via the arm64 physmap? That seems extraordinarily bad. :( -Kees (adding linux-arm-kernel and changing the subject) Kernel code should be fine, if it isn't that is a bug that should be fixed. We take care to ensure that the linear alias of the core kernel's .text and .rodata segments are mapped read-only. When we first moved the kernel out of the linear region, we did not map it there at all anymore, but that broke hibernation so we had to put something back. Modules yes are not fully protected. The conclusion from past experience has been that we cannot safely break down larger page sizes at runtime like x86 does. We could theoretically add support for fixing up the alias if PAGE_POISONING is enabled but I don't know who would actually use that in production. Performance is very poor at that point. As long as the linear alias of the module is mapped down to pages, we should be able to tweak the permissions. I take it that PAGE_POISONING does more than just that? Page poisoning does exactly that. The argument I was trying to make was that if nobody really uses page poisoning except for debugging it might not be worth it to fix up the alias. Thinking a bit more, this is a terrible argument for many reasons so yes I agree that we can just fix up the alias if PAGE_POISONING (or other features) are enabled. Thanks, Laura
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 02/14/2018 11:28 AM, Ard Biesheuvel wrote: On 14 February 2018 at 19:06, Laura Abbott wrote: On 02/13/2018 01:43 PM, Kees Cook wrote: On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING does use 4K pages which could be adjusted at runtime. So yes, you are right we would have physmap exposure on arm64 as well. Errr, so that means even modules and kernel code are writable via the arm64 physmap? That seems extraordinarily bad. :( -Kees (adding linux-arm-kernel and changing the subject) Kernel code should be fine, if it isn't that is a bug that should be fixed. We take care to ensure that the linear alias of the core kernel's .text and .rodata segments are mapped read-only. When we first moved the kernel out of the linear region, we did not map it there at all anymore, but that broke hibernation so we had to put something back. Modules yes are not fully protected. The conclusion from past experience has been that we cannot safely break down larger page sizes at runtime like x86 does. We could theoretically add support for fixing up the alias if PAGE_POISONING is enabled but I don't know who would actually use that in production. Performance is very poor at that point. As long as the linear alias of the module is mapped down to pages, we should be able to tweak the permissions. I take it that PAGE_POISONING does more than just that? Page poisoning does exactly that. The argument I was trying to make was that if nobody really uses page poisoning except for debugging it might not be worth it to fix up the alias. Thinking a bit more, this is a terrible argument for many reasons so yes I agree that we can just fix up the alias if PAGE_POISONING (or other features) are enabled. Thanks, Laura
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbottwrote: > fixed. Modules yes are not fully protected. The conclusion from past > experience has been that we cannot safely break down larger page sizes > at runtime like x86 does. We could theoretically > add support for fixing up the alias if PAGE_POISONING is enabled but > I don't know who would actually use that in production. Performance > is very poor at that point. XPFO forces 4K pages on the physmap[1] for similar reasons. I have no doubt about performance changes, but I'd be curious to see real numbers. Did anyone do benchmarks on just the huge/4K change? (Without also the XPFO overhead?) If this, XPFO, and PAGE_POISONING all need it, I think we have to start a closer investigation. :) -Kees [1] http://www.openwall.com/lists/kernel-hardening/2017/09/07/13 -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: > fixed. Modules yes are not fully protected. The conclusion from past > experience has been that we cannot safely break down larger page sizes > at runtime like x86 does. We could theoretically > add support for fixing up the alias if PAGE_POISONING is enabled but > I don't know who would actually use that in production. Performance > is very poor at that point. XPFO forces 4K pages on the physmap[1] for similar reasons. I have no doubt about performance changes, but I'd be curious to see real numbers. Did anyone do benchmarks on just the huge/4K change? (Without also the XPFO overhead?) If this, XPFO, and PAGE_POISONING all need it, I think we have to start a closer investigation. :) -Kees [1] http://www.openwall.com/lists/kernel-hardening/2017/09/07/13 -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:29 AM, Kees Cookwrote: > Why does using finer granularity on the physmap degrade performance? I > assume TLB pressure, but what is heavily using that area? (I must not > be understanding what physmap actually gets used for -- I thought it > was just a convenience to have a 1:1 virt/phys map for some lookups?) Jann has sorted me out: it's that physmap isn't an _alias_ for the buddy allocator memory areas; it's used directly. -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:29 AM, Kees Cook wrote: > Why does using finer granularity on the physmap degrade performance? I > assume TLB pressure, but what is heavily using that area? (I must not > be understanding what physmap actually gets used for -- I thought it > was just a convenience to have a 1:1 virt/phys map for some lookups?) Jann has sorted me out: it's that physmap isn't an _alias_ for the buddy allocator memory areas; it's used directly. -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbottwrote: > On 02/13/2018 01:43 PM, Kees Cook wrote: >> >> On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: >>> >>> No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger >>> page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING >>> does use 4K pages which could be adjusted at runtime. So yes, you are >>> right we would have physmap exposure on arm64 as well. >> >> >> Errr, so that means even modules and kernel code are writable via the >> arm64 physmap? That seems extraordinarily bad. :( >> >> -Kees >> > > (adding linux-arm-kernel and changing the subject) > > Kernel code should be fine, if it isn't that is a bug that should be > fixed. Modules yes are not fully protected. The conclusion from past I think that's a pretty serious problem: we can't have aliases with mismatched permissions; this degrades a deterministic protection (read-only) to a probabilistic protection (knowing where the alias of a target is mapped). Having an attack be "needs some info leaks" instead of "need execution control to change perms" is a much lower bar, IMO. > experience has been that we cannot safely break down larger page sizes > at runtime like x86 does. We could theoretically > add support for fixing up the alias if PAGE_POISONING is enabled but > I don't know who would actually use that in production. Performance > is very poor at that point. Why does using finer granularity on the physmap degrade performance? I assume TLB pressure, but what is heavily using that area? (I must not be understanding what physmap actually gets used for -- I thought it was just a convenience to have a 1:1 virt/phys map for some lookups?) -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On Wed, Feb 14, 2018 at 11:06 AM, Laura Abbott wrote: > On 02/13/2018 01:43 PM, Kees Cook wrote: >> >> On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: >>> >>> No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger >>> page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING >>> does use 4K pages which could be adjusted at runtime. So yes, you are >>> right we would have physmap exposure on arm64 as well. >> >> >> Errr, so that means even modules and kernel code are writable via the >> arm64 physmap? That seems extraordinarily bad. :( >> >> -Kees >> > > (adding linux-arm-kernel and changing the subject) > > Kernel code should be fine, if it isn't that is a bug that should be > fixed. Modules yes are not fully protected. The conclusion from past I think that's a pretty serious problem: we can't have aliases with mismatched permissions; this degrades a deterministic protection (read-only) to a probabilistic protection (knowing where the alias of a target is mapped). Having an attack be "needs some info leaks" instead of "need execution control to change perms" is a much lower bar, IMO. > experience has been that we cannot safely break down larger page sizes > at runtime like x86 does. We could theoretically > add support for fixing up the alias if PAGE_POISONING is enabled but > I don't know who would actually use that in production. Performance > is very poor at that point. Why does using finer granularity on the physmap degrade performance? I assume TLB pressure, but what is heavily using that area? (I must not be understanding what physmap actually gets used for -- I thought it was just a convenience to have a 1:1 virt/phys map for some lookups?) -Kees -- Kees Cook Pixel Security
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 14 February 2018 at 19:06, Laura Abbottwrote: > On 02/13/2018 01:43 PM, Kees Cook wrote: >> >> On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: >>> >>> No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger >>> page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING >>> does use 4K pages which could be adjusted at runtime. So yes, you are >>> right we would have physmap exposure on arm64 as well. >> >> >> Errr, so that means even modules and kernel code are writable via the >> arm64 physmap? That seems extraordinarily bad. :( >> >> -Kees >> > > (adding linux-arm-kernel and changing the subject) > > Kernel code should be fine, if it isn't that is a bug that should be > fixed. We take care to ensure that the linear alias of the core kernel's .text and .rodata segments are mapped read-only. When we first moved the kernel out of the linear region, we did not map it there at all anymore, but that broke hibernation so we had to put something back. > Modules yes are not fully protected. The conclusion from past > experience has been that we cannot safely break down larger page sizes > at runtime like x86 does. We could theoretically > add support for fixing up the alias if PAGE_POISONING is enabled but > I don't know who would actually use that in production. Performance > is very poor at that point. > As long as the linear alias of the module is mapped down to pages, we should be able to tweak the permissions. I take it that PAGE_POISONING does more than just that?
Re: arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 14 February 2018 at 19:06, Laura Abbott wrote: > On 02/13/2018 01:43 PM, Kees Cook wrote: >> >> On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: >>> >>> No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger >>> page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING >>> does use 4K pages which could be adjusted at runtime. So yes, you are >>> right we would have physmap exposure on arm64 as well. >> >> >> Errr, so that means even modules and kernel code are writable via the >> arm64 physmap? That seems extraordinarily bad. :( >> >> -Kees >> > > (adding linux-arm-kernel and changing the subject) > > Kernel code should be fine, if it isn't that is a bug that should be > fixed. We take care to ensure that the linear alias of the core kernel's .text and .rodata segments are mapped read-only. When we first moved the kernel out of the linear region, we did not map it there at all anymore, but that broke hibernation so we had to put something back. > Modules yes are not fully protected. The conclusion from past > experience has been that we cannot safely break down larger page sizes > at runtime like x86 does. We could theoretically > add support for fixing up the alias if PAGE_POISONING is enabled but > I don't know who would actually use that in production. Performance > is very poor at that point. > As long as the linear alias of the module is mapped down to pages, we should be able to tweak the permissions. I take it that PAGE_POISONING does more than just that?
arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 02/13/2018 01:43 PM, Kees Cook wrote: On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbottwrote: No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING does use 4K pages which could be adjusted at runtime. So yes, you are right we would have physmap exposure on arm64 as well. Errr, so that means even modules and kernel code are writable via the arm64 physmap? That seems extraordinarily bad. :( -Kees (adding linux-arm-kernel and changing the subject) Kernel code should be fine, if it isn't that is a bug that should be fixed. Modules yes are not fully protected. The conclusion from past experience has been that we cannot safely break down larger page sizes at runtime like x86 does. We could theoretically add support for fixing up the alias if PAGE_POISONING is enabled but I don't know who would actually use that in production. Performance is very poor at that point. Thanks, Laura
arm64 physmap (was Re: [kernel-hardening] [PATCH 4/6] Protectable Memory)
On 02/13/2018 01:43 PM, Kees Cook wrote: On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING does use 4K pages which could be adjusted at runtime. So yes, you are right we would have physmap exposure on arm64 as well. Errr, so that means even modules and kernel code are writable via the arm64 physmap? That seems extraordinarily bad. :( -Kees (adding linux-arm-kernel and changing the subject) Kernel code should be fine, if it isn't that is a bug that should be fixed. Modules yes are not fully protected. The conclusion from past experience has been that we cannot safely break down larger page sizes at runtime like x86 does. We could theoretically add support for fixing up the alias if PAGE_POISONING is enabled but I don't know who would actually use that in production. Performance is very poor at that point. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbottwrote: > No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger > page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING > does use 4K pages which could be adjusted at runtime. So yes, you are > right we would have physmap exposure on arm64 as well. Errr, so that means even modules and kernel code are writable via the arm64 physmap? That seems extraordinarily bad. :( -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Tue, Feb 13, 2018 at 8:09 AM, Laura Abbott wrote: > No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger > page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING > does use 4K pages which could be adjusted at runtime. So yes, you are > right we would have physmap exposure on arm64 as well. Errr, so that means even modules and kernel code are writable via the arm64 physmap? That seems extraordinarily bad. :( -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 02/13/2018 07:20 AM, Igor Stoppa wrote: Why alterations of page properties are not considered a risk and the physmap is? And how would it be easier (i suppose) to attack the latter? Alterations are certainly a risk but with the physmap the mapping is already there. Find the address and you have access vs. needing to actually modify the properties then do the access. I could also be complete off base on my threat model here so please correct me if I'm wrong. I think your other summaries are good points though and should go in the cover letter. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 02/13/2018 07:20 AM, Igor Stoppa wrote: Why alterations of page properties are not considered a risk and the physmap is? And how would it be easier (i suppose) to attack the latter? Alterations are certainly a risk but with the physmap the mapping is already there. Find the address and you have access vs. needing to actually modify the properties then do the access. I could also be complete off base on my threat model here so please correct me if I'm wrong. I think your other summaries are good points though and should go in the cover letter. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 02/12/2018 07:39 PM, Jann Horn wrote: On Tue, Feb 13, 2018 at 2:25 AM, Kees Cookwrote: On Mon, Feb 12, 2018 at 4:40 PM, Laura Abbott wrote: On 02/12/2018 03:27 PM, Kees Cook wrote: On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa wrote: On 04/02/18 00:29, Boris Lukashev wrote: On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: [...] What you are suggesting, if I have understood it correctly, is that, when the pool is protected, the addresses already given out, will become traps that get resolved through a lookup table that is built based on the content of each allocation. That seems to generate a lot of overhead, not to mention the fact that it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... I would agree: let's get basic functionality in first. Both verification and the physmap part can be done separately, IMO. Skipping over physmap leaves a pretty big area of exposure that could be difficult to solve later. I appreciate this might block basic functionality but I don't think we should just gloss over it without at least some idea of what we would do. What's our exposure on physmap for other regions? e.g. things that are executable, or made read-only later (like __ro_after_init)? I just checked on a system with a 4.9 kernel, and there seems to be no physical memory that is mapped as writable in the init PGD and executable elsewhere. Ah, I think I missed something. At least on X86, set_memory_ro, set_memory_rw, set_memory_nx and set_memory_x all use change_page_attr_clear/change_page_attr_set, which use change_page_attr_set_clr, which calls __change_page_attr_set_clr() with a second parameter "checkalias" that is set to 1 unless the bit being changed is the NX bit, and that parameter causes the invocation of cpa_process_alias(), which will, for mapped ranges, also change the attributes of physmap ranges. set_memory_ro() and so on are also used by the module loading code. But in the ARM64 code, I don't see anything similar. Does anyone with a better understanding of ARM64 want to check whether I missed something? Or maybe, with a recent kernel, check whether executable module pages show up with a second writable mapping in the "kernel_page_tables" file in debugfs? No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING does use 4K pages which could be adjusted at runtime. So yes, you are right we would have physmap exposure on arm64 as well. To the original question, it does sound like we are actually okay with the physmap. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 02/12/2018 07:39 PM, Jann Horn wrote: On Tue, Feb 13, 2018 at 2:25 AM, Kees Cook wrote: On Mon, Feb 12, 2018 at 4:40 PM, Laura Abbott wrote: On 02/12/2018 03:27 PM, Kees Cook wrote: On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa wrote: On 04/02/18 00:29, Boris Lukashev wrote: On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: [...] What you are suggesting, if I have understood it correctly, is that, when the pool is protected, the addresses already given out, will become traps that get resolved through a lookup table that is built based on the content of each allocation. That seems to generate a lot of overhead, not to mention the fact that it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... I would agree: let's get basic functionality in first. Both verification and the physmap part can be done separately, IMO. Skipping over physmap leaves a pretty big area of exposure that could be difficult to solve later. I appreciate this might block basic functionality but I don't think we should just gloss over it without at least some idea of what we would do. What's our exposure on physmap for other regions? e.g. things that are executable, or made read-only later (like __ro_after_init)? I just checked on a system with a 4.9 kernel, and there seems to be no physical memory that is mapped as writable in the init PGD and executable elsewhere. Ah, I think I missed something. At least on X86, set_memory_ro, set_memory_rw, set_memory_nx and set_memory_x all use change_page_attr_clear/change_page_attr_set, which use change_page_attr_set_clr, which calls __change_page_attr_set_clr() with a second parameter "checkalias" that is set to 1 unless the bit being changed is the NX bit, and that parameter causes the invocation of cpa_process_alias(), which will, for mapped ranges, also change the attributes of physmap ranges. set_memory_ro() and so on are also used by the module loading code. But in the ARM64 code, I don't see anything similar. Does anyone with a better understanding of ARM64 want to check whether I missed something? Or maybe, with a recent kernel, check whether executable module pages show up with a second writable mapping in the "kernel_page_tables" file in debugfs? No, arm64 doesn't fixup the aliases, mostly because arm64 uses larger page sizes which can't be broken down at runtime. CONFIG_PAGE_POISONING does use 4K pages which could be adjusted at runtime. So yes, you are right we would have physmap exposure on arm64 as well. To the original question, it does sound like we are actually okay with the physmap. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Tue, Feb 13, 2018 at 2:25 AM, Kees Cookwrote: > On Mon, Feb 12, 2018 at 4:40 PM, Laura Abbott wrote: >> On 02/12/2018 03:27 PM, Kees Cook wrote: >>> >>> On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa >>> wrote: On 04/02/18 00:29, Boris Lukashev wrote: > > On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa > wrote: [...] >> What you are suggesting, if I have understood it correctly, is that, >> when the pool is protected, the addresses already given out, will >> become >> traps that get resolved through a lookup table that is built based on >> the content of each allocation. >> >> That seems to generate a lot of overhead, not to mention the fact that >> it might not play very well with the MMU. > > > That is effectively what i'm suggesting - as a form of protection for > consumers against direct reads of data which may have been corrupted > by some irrelevant means. In the context of pmalloc, it would probably > be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... >>> >>> >>> I would agree: let's get basic functionality in first. Both >>> verification and the physmap part can be done separately, IMO. >> >> >> Skipping over physmap leaves a pretty big area of exposure that could >> be difficult to solve later. I appreciate this might block basic >> functionality but I don't think we should just gloss over it without >> at least some idea of what we would do. > > What's our exposure on physmap for other regions? e.g. things that are > executable, or made read-only later (like __ro_after_init)? I just checked on a system with a 4.9 kernel, and there seems to be no physical memory that is mapped as writable in the init PGD and executable elsewhere. Ah, I think I missed something. At least on X86, set_memory_ro, set_memory_rw, set_memory_nx and set_memory_x all use change_page_attr_clear/change_page_attr_set, which use change_page_attr_set_clr, which calls __change_page_attr_set_clr() with a second parameter "checkalias" that is set to 1 unless the bit being changed is the NX bit, and that parameter causes the invocation of cpa_process_alias(), which will, for mapped ranges, also change the attributes of physmap ranges. set_memory_ro() and so on are also used by the module loading code. But in the ARM64 code, I don't see anything similar. Does anyone with a better understanding of ARM64 want to check whether I missed something? Or maybe, with a recent kernel, check whether executable module pages show up with a second writable mapping in the "kernel_page_tables" file in debugfs?
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Tue, Feb 13, 2018 at 2:25 AM, Kees Cook wrote: > On Mon, Feb 12, 2018 at 4:40 PM, Laura Abbott wrote: >> On 02/12/2018 03:27 PM, Kees Cook wrote: >>> >>> On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa >>> wrote: On 04/02/18 00:29, Boris Lukashev wrote: > > On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa > wrote: [...] >> What you are suggesting, if I have understood it correctly, is that, >> when the pool is protected, the addresses already given out, will >> become >> traps that get resolved through a lookup table that is built based on >> the content of each allocation. >> >> That seems to generate a lot of overhead, not to mention the fact that >> it might not play very well with the MMU. > > > That is effectively what i'm suggesting - as a form of protection for > consumers against direct reads of data which may have been corrupted > by some irrelevant means. In the context of pmalloc, it would probably > be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... >>> >>> >>> I would agree: let's get basic functionality in first. Both >>> verification and the physmap part can be done separately, IMO. >> >> >> Skipping over physmap leaves a pretty big area of exposure that could >> be difficult to solve later. I appreciate this might block basic >> functionality but I don't think we should just gloss over it without >> at least some idea of what we would do. > > What's our exposure on physmap for other regions? e.g. things that are > executable, or made read-only later (like __ro_after_init)? I just checked on a system with a 4.9 kernel, and there seems to be no physical memory that is mapped as writable in the init PGD and executable elsewhere. Ah, I think I missed something. At least on X86, set_memory_ro, set_memory_rw, set_memory_nx and set_memory_x all use change_page_attr_clear/change_page_attr_set, which use change_page_attr_set_clr, which calls __change_page_attr_set_clr() with a second parameter "checkalias" that is set to 1 unless the bit being changed is the NX bit, and that parameter causes the invocation of cpa_process_alias(), which will, for mapped ranges, also change the attributes of physmap ranges. set_memory_ro() and so on are also used by the module loading code. But in the ARM64 code, I don't see anything similar. Does anyone with a better understanding of ARM64 want to check whether I missed something? Or maybe, with a recent kernel, check whether executable module pages show up with a second writable mapping in the "kernel_page_tables" file in debugfs?
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Mon, Feb 12, 2018 at 4:40 PM, Laura Abbottwrote: > On 02/12/2018 03:27 PM, Kees Cook wrote: >> >> On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa >> wrote: >>> >>> On 04/02/18 00:29, Boris Lukashev wrote: On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: >>> >>> >>> [...] >>> > What you are suggesting, if I have understood it correctly, is that, > when the pool is protected, the addresses already given out, will > become > traps that get resolved through a lookup table that is built based on > the content of each allocation. > > That seems to generate a lot of overhead, not to mention the fact that > it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool >>> >>> ok, that seems more like an extension though. >>> >>> ATM I am having problems gaining traction to get even the basic merged >>> :-) >>> >>> I would consider this as a possibility for future work, unless it is >>> said that it's necessary for pmalloc to be accepted ... >> >> >> I would agree: let's get basic functionality in first. Both >> verification and the physmap part can be done separately, IMO. > > > Skipping over physmap leaves a pretty big area of exposure that could > be difficult to solve later. I appreciate this might block basic > functionality but I don't think we should just gloss over it without > at least some idea of what we would do. What's our exposure on physmap for other regions? e.g. things that are executable, or made read-only later (like __ro_after_init)? -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Mon, Feb 12, 2018 at 4:40 PM, Laura Abbott wrote: > On 02/12/2018 03:27 PM, Kees Cook wrote: >> >> On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa >> wrote: >>> >>> On 04/02/18 00:29, Boris Lukashev wrote: On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: >>> >>> >>> [...] >>> > What you are suggesting, if I have understood it correctly, is that, > when the pool is protected, the addresses already given out, will > become > traps that get resolved through a lookup table that is built based on > the content of each allocation. > > That seems to generate a lot of overhead, not to mention the fact that > it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool >>> >>> ok, that seems more like an extension though. >>> >>> ATM I am having problems gaining traction to get even the basic merged >>> :-) >>> >>> I would consider this as a possibility for future work, unless it is >>> said that it's necessary for pmalloc to be accepted ... >> >> >> I would agree: let's get basic functionality in first. Both >> verification and the physmap part can be done separately, IMO. > > > Skipping over physmap leaves a pretty big area of exposure that could > be difficult to solve later. I appreciate this might block basic > functionality but I don't think we should just gloss over it without > at least some idea of what we would do. What's our exposure on physmap for other regions? e.g. things that are executable, or made read-only later (like __ro_after_init)? -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 02/12/2018 03:27 PM, Kees Cook wrote: On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppawrote: On 04/02/18 00:29, Boris Lukashev wrote: On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: [...] What you are suggesting, if I have understood it correctly, is that, when the pool is protected, the addresses already given out, will become traps that get resolved through a lookup table that is built based on the content of each allocation. That seems to generate a lot of overhead, not to mention the fact that it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... I would agree: let's get basic functionality in first. Both verification and the physmap part can be done separately, IMO. Skipping over physmap leaves a pretty big area of exposure that could be difficult to solve later. I appreciate this might block basic functionality but I don't think we should just gloss over it without at least some idea of what we would do. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 02/12/2018 03:27 PM, Kees Cook wrote: On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa wrote: On 04/02/18 00:29, Boris Lukashev wrote: On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: [...] What you are suggesting, if I have understood it correctly, is that, when the pool is protected, the addresses already given out, will become traps that get resolved through a lookup table that is built based on the content of each allocation. That seems to generate a lot of overhead, not to mention the fact that it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... I would agree: let's get basic functionality in first. Both verification and the physmap part can be done separately, IMO. Skipping over physmap leaves a pretty big area of exposure that could be difficult to solve later. I appreciate this might block basic functionality but I don't think we should just gloss over it without at least some idea of what we would do. Thanks, Laura
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppawrote: > On 04/02/18 00:29, Boris Lukashev wrote: >> On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: > > [...] > >>> What you are suggesting, if I have understood it correctly, is that, >>> when the pool is protected, the addresses already given out, will become >>> traps that get resolved through a lookup table that is built based on >>> the content of each allocation. >>> >>> That seems to generate a lot of overhead, not to mention the fact that >>> it might not play very well with the MMU. >> >> That is effectively what i'm suggesting - as a form of protection for >> consumers against direct reads of data which may have been corrupted >> by some irrelevant means. In the context of pmalloc, it would probably >> be a separate type of ro+verified pool > ok, that seems more like an extension though. > > ATM I am having problems gaining traction to get even the basic merged :-) > > I would consider this as a possibility for future work, unless it is > said that it's necessary for pmalloc to be accepted ... I would agree: let's get basic functionality in first. Both verification and the physmap part can be done separately, IMO. -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sun, Feb 4, 2018 at 7:05 AM, Igor Stoppa wrote: > On 04/02/18 00:29, Boris Lukashev wrote: >> On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: > > [...] > >>> What you are suggesting, if I have understood it correctly, is that, >>> when the pool is protected, the addresses already given out, will become >>> traps that get resolved through a lookup table that is built based on >>> the content of each allocation. >>> >>> That seems to generate a lot of overhead, not to mention the fact that >>> it might not play very well with the MMU. >> >> That is effectively what i'm suggesting - as a form of protection for >> consumers against direct reads of data which may have been corrupted >> by some irrelevant means. In the context of pmalloc, it would probably >> be a separate type of ro+verified pool > ok, that seems more like an extension though. > > ATM I am having problems gaining traction to get even the basic merged :-) > > I would consider this as a possibility for future work, unless it is > said that it's necessary for pmalloc to be accepted ... I would agree: let's get basic functionality in first. Both verification and the physmap part can be done separately, IMO. -Kees -- Kees Cook Pixel Security
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 05/02/18 17:40, Christopher Lameter wrote: > On Sat, 3 Feb 2018, Igor Stoppa wrote: > >>> We could even do this in a more thorough way. Can we use a ring 1 / 2 >>> distinction to create a hardened OS core that policies the rest of >>> the ever expanding kernel with all its modules and this and that feature? >> >> What would be the differentiating criteria? Furthermore, what are the >> chances >> of invalidating the entire concept, because there is already an >> hypervisor using >> the higher level features? >> That is what you are proposing, if I understand correctly. > > Were there not 4 rings as well as methods by the processor vendors to > virtualize them as well? I think you are talking x86, mostly. On ARM there are ELx and they are often (typically?) already used. For x86 I cannot comment. >>> I think that will long term be a better approach and allow more than the >>> current hardening approaches can get you. It seems that we are willing to >>> tolerate significant performance regressions now. So lets use the >>> protection mechanisms that the hardware offers. >> >> I would rather *not* propose significant performance regression :-P > > But we already have implemented significant kernel hardening which causes > performance regressions. Using hardware capabilities allows the processor > vendor to further optimize these mechanisms whereas the software > preventative measures are eating up more and more performance as the pile > them on. Plus these are methods that can be worked around. Restrictions > implemented in a higher ring can be enforced and are much better than > just "hardening" (which is making life difficult for the hackers and > throwing away performannce for the average user). What you are proposing requires major restructuring of the memory management - at the very least - provided that it doesn't cause the conflicts I mentioned above. Even after you do that, the system will still be working with memory pages, there will be still a need to segregate data within certain pages, or pay the penalty of handling exceptions, when data with different permissions coexist within the same page. The way the pmalloc API is designed is meant to facilitate the segregation and to actually improve performance, by grouping types of data with same scope and permission. WRT the implementation, there is a minimal exposure to the memory provider, both for allocation and release. Same goes for the protection mechanism. It's a single call to the function which makes pages read only. It would be trivial to swap it out with a call to whatever framework you want to come up with, for implementing ring/EL based protection. >From this perspective, you can easily provide patches that implement what you are proposing, against pmalloc, if you really think that it's the way to go. I'll be happy to use them, if they provide improved performance and same or better protection. The way I designed pmalloc was really to be able to switch to some alternate memory provider and/or protection mechanism, should a better one arise. But it can be done in a separate step, I think, since you are not proposing to just change pmalloc, you are proposing to re-design how the overall kernel memory hardening works (including executable pages, const data, __ro_after_init, etc.) -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 05/02/18 17:40, Christopher Lameter wrote: > On Sat, 3 Feb 2018, Igor Stoppa wrote: > >>> We could even do this in a more thorough way. Can we use a ring 1 / 2 >>> distinction to create a hardened OS core that policies the rest of >>> the ever expanding kernel with all its modules and this and that feature? >> >> What would be the differentiating criteria? Furthermore, what are the >> chances >> of invalidating the entire concept, because there is already an >> hypervisor using >> the higher level features? >> That is what you are proposing, if I understand correctly. > > Were there not 4 rings as well as methods by the processor vendors to > virtualize them as well? I think you are talking x86, mostly. On ARM there are ELx and they are often (typically?) already used. For x86 I cannot comment. >>> I think that will long term be a better approach and allow more than the >>> current hardening approaches can get you. It seems that we are willing to >>> tolerate significant performance regressions now. So lets use the >>> protection mechanisms that the hardware offers. >> >> I would rather *not* propose significant performance regression :-P > > But we already have implemented significant kernel hardening which causes > performance regressions. Using hardware capabilities allows the processor > vendor to further optimize these mechanisms whereas the software > preventative measures are eating up more and more performance as the pile > them on. Plus these are methods that can be worked around. Restrictions > implemented in a higher ring can be enforced and are much better than > just "hardening" (which is making life difficult for the hackers and > throwing away performannce for the average user). What you are proposing requires major restructuring of the memory management - at the very least - provided that it doesn't cause the conflicts I mentioned above. Even after you do that, the system will still be working with memory pages, there will be still a need to segregate data within certain pages, or pay the penalty of handling exceptions, when data with different permissions coexist within the same page. The way the pmalloc API is designed is meant to facilitate the segregation and to actually improve performance, by grouping types of data with same scope and permission. WRT the implementation, there is a minimal exposure to the memory provider, both for allocation and release. Same goes for the protection mechanism. It's a single call to the function which makes pages read only. It would be trivial to swap it out with a call to whatever framework you want to come up with, for implementing ring/EL based protection. >From this perspective, you can easily provide patches that implement what you are proposing, against pmalloc, if you really think that it's the way to go. I'll be happy to use them, if they provide improved performance and same or better protection. The way I designed pmalloc was really to be able to switch to some alternate memory provider and/or protection mechanism, should a better one arise. But it can be done in a separate step, I think, since you are not proposing to just change pmalloc, you are proposing to re-design how the overall kernel memory hardening works (including executable pages, const data, __ro_after_init, etc.) -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sat, 3 Feb 2018, Igor Stoppa wrote: > > We could even do this in a more thorough way. Can we use a ring 1 / 2 > > distinction to create a hardened OS core that policies the rest of > > the ever expanding kernel with all its modules and this and that feature? > > What would be the differentiating criteria? Furthermore, what are the > chances > of invalidating the entire concept, because there is already an > hypervisor using > the higher level features? > That is what you are proposing, if I understand correctly. Were there not 4 rings as well as methods by the processor vendors to virtualize them as well? > > I think that will long term be a better approach and allow more than the > > current hardening approaches can get you. It seems that we are willing to > > tolerate significant performance regressions now. So lets use the > > protection mechanisms that the hardware offers. > > I would rather *not* propose significant performance regression :-P But we already have implemented significant kernel hardening which causes performance regressions. Using hardware capabilities allows the processor vendor to further optimize these mechanisms whereas the software preventative measures are eating up more and more performance as the pile them on. Plus these are methods that can be worked around. Restrictions implemented in a higher ring can be enforced and are much better than just "hardening" (which is making life difficult for the hackers and throwing away performannce for the average user).
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sat, 3 Feb 2018, Igor Stoppa wrote: > > We could even do this in a more thorough way. Can we use a ring 1 / 2 > > distinction to create a hardened OS core that policies the rest of > > the ever expanding kernel with all its modules and this and that feature? > > What would be the differentiating criteria? Furthermore, what are the > chances > of invalidating the entire concept, because there is already an > hypervisor using > the higher level features? > That is what you are proposing, if I understand correctly. Were there not 4 rings as well as methods by the processor vendors to virtualize them as well? > > I think that will long term be a better approach and allow more than the > > current hardening approaches can get you. It seems that we are willing to > > tolerate significant performance regressions now. So lets use the > > protection mechanisms that the hardware offers. > > I would rather *not* propose significant performance regression :-P But we already have implemented significant kernel hardening which causes performance regressions. Using hardware capabilities allows the processor vendor to further optimize these mechanisms whereas the software preventative measures are eating up more and more performance as the pile them on. Plus these are methods that can be worked around. Restrictions implemented in a higher ring can be enforced and are much better than just "hardening" (which is making life difficult for the hackers and throwing away performannce for the average user).
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 04/02/18 00:29, Boris Lukashev wrote: > On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppawrote: [...] >> What you are suggesting, if I have understood it correctly, is that, >> when the pool is protected, the addresses already given out, will become >> traps that get resolved through a lookup table that is built based on >> the content of each allocation. >> >> That seems to generate a lot of overhead, not to mention the fact that >> it might not play very well with the MMU. > > That is effectively what i'm suggesting - as a form of protection for > consumers against direct reads of data which may have been corrupted > by some irrelevant means. In the context of pmalloc, it would probably > be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 04/02/18 00:29, Boris Lukashev wrote: > On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: [...] >> What you are suggesting, if I have understood it correctly, is that, >> when the pool is protected, the addresses already given out, will become >> traps that get resolved through a lookup table that is built based on >> the content of each allocation. >> >> That seems to generate a lot of overhead, not to mention the fact that >> it might not play very well with the MMU. > > That is effectively what i'm suggesting - as a form of protection for > consumers against direct reads of data which may have been corrupted > by some irrelevant means. In the context of pmalloc, it would probably > be a separate type of ro+verified pool ok, that seems more like an extension though. ATM I am having problems gaining traction to get even the basic merged :-) I would consider this as a possibility for future work, unless it is said that it's necessary for pmalloc to be accepted ... -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppawrote: > > > On 03/02/18 22:12, Boris Lukashev wrote: > >> Regarding the notion of validated protected memory, is there a method >> by which the resulting checksum could be used in a lookup >> table/function to resolve the location of the protected data? > > What I have in mind is a checksum at page/vmap_area level, so there > would be no 1:1 mapping between a specific allocation and the checksum. > > An extreme case would be the one where an allocation crosses one or more > page boundaries, while the checksum refers to a (partially) overlapping > memory area. > > Code accessing a pool could perform one (relatively expensive) > validation. But still something that would require a more sophisticated > attack, to subvert. > >> Effectively a hash table of protected allocations, with a benefit of >> dedup since any data matching the same key would be the same data >> (multiple identical cred structs being pushed around). Should leave >> the resolver address/csum in recent memory to check against, right? > > I see where you are trying to land, but I do not see how it would work > without a further intermediate step. > > pmalloc dishes out virtual memory addresses, when called. > > It doesn't know what the user of the allocation will put in it. > The user, otoh, has the direct address of the memory it got. > > What you are suggesting, if I have understood it correctly, is that, > when the pool is protected, the addresses already given out, will become > traps that get resolved through a lookup table that is built based on > the content of each allocation. > > That seems to generate a lot of overhead, not to mention the fact that > it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool which consumers would explicitly opt into. Say there's a maintenance cycle on a and it wants to make sure that the instructions it read in are what they should have been before running them, those consumers might well take the penalty if it keeps from doing . If such a resolver could be implemented in a manner which doesnt break all the things (including acceptable performance for at least a significant number of workloads), it might be useful as a general tool for handing out memory to userspace, even in rw, as it provides execution context in which other requirements can be forcibly resolved, preventing unauthorized access to pages the consumer shouldn't get in a very generic way. Spectre comes to mind as a potential class of issues to be addressed this way, since speculative load could be prevented if the resolution were to fail. > > If I misunderstood, then I'd need a step by step description of what > happens, because it's not clear to me how else the data would be > accessed if not through the address that was obtained when pmalloc was > invoked. > > -- > igor -- Boris Lukashev Systems Architect Semper Victus
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sat, Feb 3, 2018 at 3:32 PM, Igor Stoppa wrote: > > > On 03/02/18 22:12, Boris Lukashev wrote: > >> Regarding the notion of validated protected memory, is there a method >> by which the resulting checksum could be used in a lookup >> table/function to resolve the location of the protected data? > > What I have in mind is a checksum at page/vmap_area level, so there > would be no 1:1 mapping between a specific allocation and the checksum. > > An extreme case would be the one where an allocation crosses one or more > page boundaries, while the checksum refers to a (partially) overlapping > memory area. > > Code accessing a pool could perform one (relatively expensive) > validation. But still something that would require a more sophisticated > attack, to subvert. > >> Effectively a hash table of protected allocations, with a benefit of >> dedup since any data matching the same key would be the same data >> (multiple identical cred structs being pushed around). Should leave >> the resolver address/csum in recent memory to check against, right? > > I see where you are trying to land, but I do not see how it would work > without a further intermediate step. > > pmalloc dishes out virtual memory addresses, when called. > > It doesn't know what the user of the allocation will put in it. > The user, otoh, has the direct address of the memory it got. > > What you are suggesting, if I have understood it correctly, is that, > when the pool is protected, the addresses already given out, will become > traps that get resolved through a lookup table that is built based on > the content of each allocation. > > That seems to generate a lot of overhead, not to mention the fact that > it might not play very well with the MMU. That is effectively what i'm suggesting - as a form of protection for consumers against direct reads of data which may have been corrupted by some irrelevant means. In the context of pmalloc, it would probably be a separate type of ro+verified pool which consumers would explicitly opt into. Say there's a maintenance cycle on a and it wants to make sure that the instructions it read in are what they should have been before running them, those consumers might well take the penalty if it keeps from doing . If such a resolver could be implemented in a manner which doesnt break all the things (including acceptable performance for at least a significant number of workloads), it might be useful as a general tool for handing out memory to userspace, even in rw, as it provides execution context in which other requirements can be forcibly resolved, preventing unauthorized access to pages the consumer shouldn't get in a very generic way. Spectre comes to mind as a potential class of issues to be addressed this way, since speculative load could be prevented if the resolution were to fail. > > If I misunderstood, then I'd need a step by step description of what > happens, because it's not clear to me how else the data would be > accessed if not through the address that was obtained when pmalloc was > invoked. > > -- > igor -- Boris Lukashev Systems Architect Semper Victus
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 03/02/18 22:12, Boris Lukashev wrote: > Regarding the notion of validated protected memory, is there a method > by which the resulting checksum could be used in a lookup > table/function to resolve the location of the protected data? What I have in mind is a checksum at page/vmap_area level, so there would be no 1:1 mapping between a specific allocation and the checksum. An extreme case would be the one where an allocation crosses one or more page boundaries, while the checksum refers to a (partially) overlapping memory area. Code accessing a pool could perform one (relatively expensive) validation. But still something that would require a more sophisticated attack, to subvert. > Effectively a hash table of protected allocations, with a benefit of > dedup since any data matching the same key would be the same data > (multiple identical cred structs being pushed around). Should leave > the resolver address/csum in recent memory to check against, right? I see where you are trying to land, but I do not see how it would work without a further intermediate step. pmalloc dishes out virtual memory addresses, when called. It doesn't know what the user of the allocation will put in it. The user, otoh, has the direct address of the memory it got. What you are suggesting, if I have understood it correctly, is that, when the pool is protected, the addresses already given out, will become traps that get resolved through a lookup table that is built based on the content of each allocation. That seems to generate a lot of overhead, not to mention the fact that it might not play very well with the MMU. If I misunderstood, then I'd need a step by step description of what happens, because it's not clear to me how else the data would be accessed if not through the address that was obtained when pmalloc was invoked. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 03/02/18 22:12, Boris Lukashev wrote: > Regarding the notion of validated protected memory, is there a method > by which the resulting checksum could be used in a lookup > table/function to resolve the location of the protected data? What I have in mind is a checksum at page/vmap_area level, so there would be no 1:1 mapping between a specific allocation and the checksum. An extreme case would be the one where an allocation crosses one or more page boundaries, while the checksum refers to a (partially) overlapping memory area. Code accessing a pool could perform one (relatively expensive) validation. But still something that would require a more sophisticated attack, to subvert. > Effectively a hash table of protected allocations, with a benefit of > dedup since any data matching the same key would be the same data > (multiple identical cred structs being pushed around). Should leave > the resolver address/csum in recent memory to check against, right? I see where you are trying to land, but I do not see how it would work without a further intermediate step. pmalloc dishes out virtual memory addresses, when called. It doesn't know what the user of the allocation will put in it. The user, otoh, has the direct address of the memory it got. What you are suggesting, if I have understood it correctly, is that, when the pool is protected, the addresses already given out, will become traps that get resolved through a lookup table that is built based on the content of each allocation. That seems to generate a lot of overhead, not to mention the fact that it might not play very well with the MMU. If I misunderstood, then I'd need a step by step description of what happens, because it's not clear to me how else the data would be accessed if not through the address that was obtained when pmalloc was invoked. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sat, Feb 3, 2018 at 2:57 PM, Igor Stoppawrote: >>> On Thu, 25 Jan 2018, Matthew Wilcox wrote: > It's worth having a discussion about whether we want the pmalloc API or whether we want a slab-based API. > I'd love to have some feedback specifically about the API. > > I have also some idea about userspace and how to extend the pmalloc > concept to it: > > http://www.openwall.com/lists/kernel-hardening/2018/01/30/20 > > I'll be AFK intermittently for about 2 weeks, so i might not be able to > reply immediately, but from my perspective this would be just the > beginning of a broader hardening of both kernel and userspace that I'd > like to pursue. > > -- > igor Regarding the notion of validated protected memory, is there a method by which the resulting checksum could be used in a lookup table/function to resolve the location of the protected data? Effectively a hash table of protected allocations, with a benefit of dedup since any data matching the same key would be the same data (multiple identical cred structs being pushed around). Should leave the resolver address/csum in recent memory to check against, right? -- Boris Lukashev Systems Architect Semper Victus
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Sat, Feb 3, 2018 at 2:57 PM, Igor Stoppa wrote: >>> On Thu, 25 Jan 2018, Matthew Wilcox wrote: > It's worth having a discussion about whether we want the pmalloc API or whether we want a slab-based API. > I'd love to have some feedback specifically about the API. > > I have also some idea about userspace and how to extend the pmalloc > concept to it: > > http://www.openwall.com/lists/kernel-hardening/2018/01/30/20 > > I'll be AFK intermittently for about 2 weeks, so i might not be able to > reply immediately, but from my perspective this would be just the > beginning of a broader hardening of both kernel and userspace that I'd > like to pursue. > > -- > igor Regarding the notion of validated protected memory, is there a method by which the resulting checksum could be used in a lookup table/function to resolve the location of the protected data? Effectively a hash table of protected allocations, with a benefit of dedup since any data matching the same key would be the same data (multiple identical cred structs being pushed around). Should leave the resolver address/csum in recent memory to check against, right? -- Boris Lukashev Systems Architect Semper Victus
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
>> On Thu, 25 Jan 2018, Matthew Wilcox wrote: >>> It's worth having a discussion about whether we want the pmalloc API >>> or whether we want a slab-based API. I'd love to have some feedback specifically about the API. I have also some idea about userspace and how to extend the pmalloc concept to it: http://www.openwall.com/lists/kernel-hardening/2018/01/30/20 I'll be AFK intermittently for about 2 weeks, so i might not be able to reply immediately, but from my perspective this would be just the beginning of a broader hardening of both kernel and userspace that I'd like to pursue. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
>> On Thu, 25 Jan 2018, Matthew Wilcox wrote: >>> It's worth having a discussion about whether we want the pmalloc API >>> or whether we want a slab-based API. I'd love to have some feedback specifically about the API. I have also some idea about userspace and how to extend the pmalloc concept to it: http://www.openwall.com/lists/kernel-hardening/2018/01/30/20 I'll be AFK intermittently for about 2 weeks, so i might not be able to reply immediately, but from my perspective this would be just the beginning of a broader hardening of both kernel and userspace that I'd like to pursue. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
+Boris Lukashev On 02/02/18 20:39, Christopher Lameter wrote: > On Thu, 25 Jan 2018, Matthew Wilcox wrote: > >> It's worth having a discussion about whether we want the pmalloc API >> or whether we want a slab-based API. We can have a separate discussion >> about an API to remove pages from the physmap. > > We could even do this in a more thorough way. Can we use a ring 1 / 2 > distinction to create a hardened OS core that policies the rest of > the ever expanding kernel with all its modules and this and that feature? What would be the differentiating criteria? Furthermore, what are the chances of invalidating the entire concept, because there is already an hypervisor using the higher level features? That is what you are proposing, if I understand correctly. But more on this below ... > I think that will long term be a better approach and allow more than the > current hardening approaches can get you. It seems that we are willing to > tolerate significant performance regressions now. So lets use the > protection mechanisms that the hardware offers. I would rather *not* propose significant performance regression :-P There might be some one-off case or anyway rare event which is penalized, but my preference goes to not introducing any significant performance penalty, during regular use. After all, the lower the penalty, the wider the (potential) adoption. More in detail: there are 2 major cases for wanting some form of read-only protection. 1) extra ward against accidental corruption The kernel provides many debugging tools and they can detect lots of errors during development, but they require time and knowledge to use them, which are not always available. Furthermore, it is objectively true that not all the code has the same level of maturity, especially when non-upstream code is used in some custom product. It's not my main goal, but it would be nice if that case too could be addressed by the protection. Corruption *can* happen. Having live guards against it, will definitely help spotting bugs or, at the very least, crash/reboot a device before it can cause permanent data corruption. Protection against accidental corruption should be used as widely as possible, therefore it cannot have an high price tag, in terms of lost performance. Otherwise, there's the risk that it will be just a debug feature, more like lockdep or ubsan. 2) protection against malicious attacks This is harder, of course, but what is realistically to be expected? If an attacker can gain full control of the kernel, the only way to do damage control is to have HW and/or higher privilege SW that can somehow limit the reach of the attacker. To make it work for real, it should be mandated that either these extra HW/SW means can tell apart legitimate kernel activity from rogue actions, or they operate so independently from the kernel that a compromise kernel cannot use any API to influence them. The consensus seems to be to put aside (for now) this concern and instead focus on what is a typical scenario: - some bug is found that allows to read/write kernel memory - some other bug is found, which leaks the address of a well known variable, effectively revealing the randomized offset of each symbol placed in linear memory, once their relative location is known. What is described above is a toolkit that effectively can allow - with patience - to attack anything that is writable by the kernel. Including page tables and permissions. However the typical attack is more like: "let's flip some bit(s)". Which is where __ro_after_init has its purpose to exist. My proposal is to extend the same sort of protection also to variables allocated dynamically. * make the pages read only, once the data is initialized * use vmalloc to prevent that exfiltrating the address of an unrelated variable can easily give away the location of the real target, because of the individual page mapping vs linear mapping. Boris Lukashev proposed additional hardening, when accessing a certain variable, in the form of hash/checksum, but I could not come up with an implementation that did not have too much overhead. Re-considering this, one option would be to have a function "pool_validate()" - probably expensive - that could be invoked by a piece of code before using the data from the pool. Not perfect, because it would not be atomic, but it could be used once, at the beginning of a function, without adding overhead to each access to the pool that the function would perform. An attacker would have to time the attack so that the corruption of the data wold happen after the pool is validated and before the data is read from it. Possible, but way tricker than the current unprotected situation. What I am trying to say, is that even after having multi-ring implementation (which would be more dependent on HW features), there would be still the problem of validating the legitimacy of the use of the API that such implementation would expose. I'd rather try to
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
+Boris Lukashev On 02/02/18 20:39, Christopher Lameter wrote: > On Thu, 25 Jan 2018, Matthew Wilcox wrote: > >> It's worth having a discussion about whether we want the pmalloc API >> or whether we want a slab-based API. We can have a separate discussion >> about an API to remove pages from the physmap. > > We could even do this in a more thorough way. Can we use a ring 1 / 2 > distinction to create a hardened OS core that policies the rest of > the ever expanding kernel with all its modules and this and that feature? What would be the differentiating criteria? Furthermore, what are the chances of invalidating the entire concept, because there is already an hypervisor using the higher level features? That is what you are proposing, if I understand correctly. But more on this below ... > I think that will long term be a better approach and allow more than the > current hardening approaches can get you. It seems that we are willing to > tolerate significant performance regressions now. So lets use the > protection mechanisms that the hardware offers. I would rather *not* propose significant performance regression :-P There might be some one-off case or anyway rare event which is penalized, but my preference goes to not introducing any significant performance penalty, during regular use. After all, the lower the penalty, the wider the (potential) adoption. More in detail: there are 2 major cases for wanting some form of read-only protection. 1) extra ward against accidental corruption The kernel provides many debugging tools and they can detect lots of errors during development, but they require time and knowledge to use them, which are not always available. Furthermore, it is objectively true that not all the code has the same level of maturity, especially when non-upstream code is used in some custom product. It's not my main goal, but it would be nice if that case too could be addressed by the protection. Corruption *can* happen. Having live guards against it, will definitely help spotting bugs or, at the very least, crash/reboot a device before it can cause permanent data corruption. Protection against accidental corruption should be used as widely as possible, therefore it cannot have an high price tag, in terms of lost performance. Otherwise, there's the risk that it will be just a debug feature, more like lockdep or ubsan. 2) protection against malicious attacks This is harder, of course, but what is realistically to be expected? If an attacker can gain full control of the kernel, the only way to do damage control is to have HW and/or higher privilege SW that can somehow limit the reach of the attacker. To make it work for real, it should be mandated that either these extra HW/SW means can tell apart legitimate kernel activity from rogue actions, or they operate so independently from the kernel that a compromise kernel cannot use any API to influence them. The consensus seems to be to put aside (for now) this concern and instead focus on what is a typical scenario: - some bug is found that allows to read/write kernel memory - some other bug is found, which leaks the address of a well known variable, effectively revealing the randomized offset of each symbol placed in linear memory, once their relative location is known. What is described above is a toolkit that effectively can allow - with patience - to attack anything that is writable by the kernel. Including page tables and permissions. However the typical attack is more like: "let's flip some bit(s)". Which is where __ro_after_init has its purpose to exist. My proposal is to extend the same sort of protection also to variables allocated dynamically. * make the pages read only, once the data is initialized * use vmalloc to prevent that exfiltrating the address of an unrelated variable can easily give away the location of the real target, because of the individual page mapping vs linear mapping. Boris Lukashev proposed additional hardening, when accessing a certain variable, in the form of hash/checksum, but I could not come up with an implementation that did not have too much overhead. Re-considering this, one option would be to have a function "pool_validate()" - probably expensive - that could be invoked by a piece of code before using the data from the pool. Not perfect, because it would not be atomic, but it could be used once, at the beginning of a function, without adding overhead to each access to the pool that the function would perform. An attacker would have to time the attack so that the corruption of the data wold happen after the pool is validated and before the data is read from it. Possible, but way tricker than the current unprotected situation. What I am trying to say, is that even after having multi-ring implementation (which would be more dependent on HW features), there would be still the problem of validating the legitimacy of the use of the API that such implementation would expose. I'd rather try to
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Thu, 25 Jan 2018, Matthew Wilcox wrote: > It's worth having a discussion about whether we want the pmalloc API > or whether we want a slab-based API. We can have a separate discussion > about an API to remove pages from the physmap. We could even do this in a more thorough way. Can we use a ring 1 / 2 distinction to create a hardened OS core that policies the rest of the ever expanding kernel with all its modules and this and that feature? I think that will long term be a better approach and allow more than the current hardening approaches can get you. It seems that we are willing to tolerate significant performance regressions now. So lets use the protection mechanisms that the hardware offers.
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Thu, 25 Jan 2018, Matthew Wilcox wrote: > It's worth having a discussion about whether we want the pmalloc API > or whether we want a slab-based API. We can have a separate discussion > about an API to remove pages from the physmap. We could even do this in a more thorough way. Can we use a ring 1 / 2 distinction to create a hardened OS core that policies the rest of the ever expanding kernel with all its modules and this and that feature? I think that will long term be a better approach and allow more than the current hardening approaches can get you. It seems that we are willing to tolerate significant performance regressions now. So lets use the protection mechanisms that the hardware offers.
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 26/01/18 18:36, Boris Lukashev wrote: > I like the idea of making the verification call optional for consumers > allowing for fast/slow+hard paths depending on their needs. > Cant see any additional vectors for abuse (other than the original > ones effecting out-of-band modification) introduced by having > verify/normal callers, but i've not had enough coffee yet. Any access > races or things like that come to mind for anyone? Well, the devil is in the details. In this case, the question is how to perform the verification in a way that is sufficiently robust against races. After thinking about it for a while, I doubt it can be done reliably. It might work for some small data types, but the typical use case I have found myself dealing with, is protecting data structures. That also brings up a separate problem: what would be the size of data to hash? At one extreme there is a page, but it's probably too much, so what is the correct size? it cannot be smaller than a specific allocation, however that would imply looking for the hash related to the data being accessed, with extra overhead. And the data being accessed might be a field in a struct, for which we would not have any hash. There would be a hash only for the containing struct that was allocated ... Overall, it seems a good idea in theory, but when I think about its implementation, it seems like the overhead is so big that it would discourage its use for almost any practical purpose. If one really wants to be paranoid could, otoh have redundancy in a different pool. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 26/01/18 18:36, Boris Lukashev wrote: > I like the idea of making the verification call optional for consumers > allowing for fast/slow+hard paths depending on their needs. > Cant see any additional vectors for abuse (other than the original > ones effecting out-of-band modification) introduced by having > verify/normal callers, but i've not had enough coffee yet. Any access > races or things like that come to mind for anyone? Well, the devil is in the details. In this case, the question is how to perform the verification in a way that is sufficiently robust against races. After thinking about it for a while, I doubt it can be done reliably. It might work for some small data types, but the typical use case I have found myself dealing with, is protecting data structures. That also brings up a separate problem: what would be the size of data to hash? At one extreme there is a page, but it's probably too much, so what is the correct size? it cannot be smaller than a specific allocation, however that would imply looking for the hash related to the data being accessed, with extra overhead. And the data being accessed might be a field in a struct, for which we would not have any hash. There would be a hash only for the containing struct that was allocated ... Overall, it seems a good idea in theory, but when I think about its implementation, it seems like the overhead is so big that it would discourage its use for almost any practical purpose. If one really wants to be paranoid could, otoh have redundancy in a different pool. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Fri, Jan 26, 2018 at 7:28 AM, Igor Stoppawrote: > On 25/01/18 17:38, Jerome Glisse wrote: >> On Thu, Jan 25, 2018 at 10:14:28AM -0500, Boris Lukashev wrote: >>> On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppa wrote: >> >> [...] >> >>> DMA/physmap access coupled with a knowledge of which virtual mappings >>> are in the physical space should be enough for an attacker to bypass >>> the gating mechanism this work imposes. Not trivial, but not >>> impossible. Since there's no way to prevent that sort of access in >>> current hardware (especially something like a NIC or GPU working >>> independently of the CPU altogether) > > [...] > >> I am not saying that this can not happen but that we are trying our best >> to avoid it. > > How about an opt-in verification, similar to what proposed by Boris > Lukashev? > > When reading back the data, one could access the pointer directly and > bypass the verification, or could use a function that explicitly checks > the integrity of the data. > > Starting from an unprotected kmalloc allocation, even just turning the > data into R/O is an improvement, but if one can afford the overhead of > performing the verification, why not? > I like the idea of making the verification call optional for consumers allowing for fast/slow+hard paths depending on their needs. Cant see any additional vectors for abuse (other than the original ones effecting out-of-band modification) introduced by having verify/normal callers, but i've not had enough coffee yet. Any access races or things like that come to mind for anyone? Shouldn't happen with a write-once allocation, but again, lacking coffee. > It would still be better if the service was provided by the library, > instead than implemented by individual users, I think. > > -- > igor -Boris
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Fri, Jan 26, 2018 at 7:28 AM, Igor Stoppa wrote: > On 25/01/18 17:38, Jerome Glisse wrote: >> On Thu, Jan 25, 2018 at 10:14:28AM -0500, Boris Lukashev wrote: >>> On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppa wrote: >> >> [...] >> >>> DMA/physmap access coupled with a knowledge of which virtual mappings >>> are in the physical space should be enough for an attacker to bypass >>> the gating mechanism this work imposes. Not trivial, but not >>> impossible. Since there's no way to prevent that sort of access in >>> current hardware (especially something like a NIC or GPU working >>> independently of the CPU altogether) > > [...] > >> I am not saying that this can not happen but that we are trying our best >> to avoid it. > > How about an opt-in verification, similar to what proposed by Boris > Lukashev? > > When reading back the data, one could access the pointer directly and > bypass the verification, or could use a function that explicitly checks > the integrity of the data. > > Starting from an unprotected kmalloc allocation, even just turning the > data into R/O is an improvement, but if one can afford the overhead of > performing the verification, why not? > I like the idea of making the verification call optional for consumers allowing for fast/slow+hard paths depending on their needs. Cant see any additional vectors for abuse (other than the original ones effecting out-of-band modification) introduced by having verify/normal callers, but i've not had enough coffee yet. Any access races or things like that come to mind for anyone? Shouldn't happen with a write-once allocation, but again, lacking coffee. > It would still be better if the service was provided by the library, > instead than implemented by individual users, I think. > > -- > igor -Boris
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 25/01/18 17:38, Jerome Glisse wrote: > On Thu, Jan 25, 2018 at 10:14:28AM -0500, Boris Lukashev wrote: >> On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppawrote: > > [...] > >> DMA/physmap access coupled with a knowledge of which virtual mappings >> are in the physical space should be enough for an attacker to bypass >> the gating mechanism this work imposes. Not trivial, but not >> impossible. Since there's no way to prevent that sort of access in >> current hardware (especially something like a NIC or GPU working >> independently of the CPU altogether) [...] > I am not saying that this can not happen but that we are trying our best > to avoid it. How about an opt-in verification, similar to what proposed by Boris Lukashev? When reading back the data, one could access the pointer directly and bypass the verification, or could use a function that explicitly checks the integrity of the data. Starting from an unprotected kmalloc allocation, even just turning the data into R/O is an improvement, but if one can afford the overhead of performing the verification, why not? It would still be better if the service was provided by the library, instead than implemented by individual users, I think. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 25/01/18 17:38, Jerome Glisse wrote: > On Thu, Jan 25, 2018 at 10:14:28AM -0500, Boris Lukashev wrote: >> On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppa wrote: > > [...] > >> DMA/physmap access coupled with a knowledge of which virtual mappings >> are in the physical space should be enough for an attacker to bypass >> the gating mechanism this work imposes. Not trivial, but not >> impossible. Since there's no way to prevent that sort of access in >> current hardware (especially something like a NIC or GPU working >> independently of the CPU altogether) [...] > I am not saying that this can not happen but that we are trying our best > to avoid it. How about an opt-in verification, similar to what proposed by Boris Lukashev? When reading back the data, one could access the pointer directly and bypass the verification, or could use a function that explicitly checks the integrity of the data. Starting from an unprotected kmalloc allocation, even just turning the data into R/O is an improvement, but if one can afford the overhead of performing the verification, why not? It would still be better if the service was provided by the library, instead than implemented by individual users, I think. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 26/01/18 07:35, Matthew Wilcox wrote: > On Wed, Jan 24, 2018 at 08:10:53PM +0100, Jann Horn wrote: >> I'm not entirely convinced by the approach of marking small parts of >> kernel memory as readonly for hardening. > > It depends how significant the data stored in there are. For example, > storing function pointers in read-only memory provides significant > hardening. > >> You're allocating with vmalloc(), which, as far as I know, establishes >> a second mapping in the vmalloc area for pages that are already mapped >> as RW through the physmap. AFAICS, later, when you're trying to make >> pages readonly, you're only changing the protections on the second >> mapping in the vmalloc area, therefore leaving the memory writable >> through the physmap. Is that correct? If so, please either document >> the reasoning why this is okay or change it. > > Yes, this is still vulnerable to attacks through the physmap. That's also > true for marking structs as const. We should probably fix that at some > point, but at least they're not vulnerable to heap overruns by small > amounts ... you have to be able to overrun some other array by terabytes. Actually, I think there is something to say in favor of using a vmalloc based approach, precisely because of the physmap :-P If I understood correctly, the physmap is primarily meant to speed up access to physical memory through the TLB. In particular, for kmalloc based allocations. Which means that, to perform a physmap-based attack to a kmalloced allocation, one needs to know: - the address of the target variable in the kmalloc range - the randomized offset of the kernel - the location of the physmap But, for a vmalloc based allocation, there is one extra hoop: since the mapping is really per page, now the attacker has actually to walk the page table, to figure out where to poke in the physmap. One more thought about physmap: does it map also code? Because, if it does, and one wants to use it for an attack, isn't it easier to look for some security test and replace a bne with be or equivalent? > It's worth having a discussion about whether we want the pmalloc API > or whether we want a slab-based API. pmalloc is meant to be useful where the attack surface is made up of lots of small allocations - my first use case was the SE Linux policy DB, where there is a variety of elements being allocated, in large amount. To the point where having ready made caches would be wasteful. Then there is the issue I already mentioned about arm/arm64 which would require to break down large mappings, which seems to be against current policy, as described in my previous mail: http://www.openwall.com/lists/kernel-hardening/2018/01/24/11 I do not know exactly what you have in mind wrt slab, but my impression is that it will most likely gravitate toward the pmalloc implementation. It will need: - "pools" or anyway some means to lock only a certain group of pages, related to a specific kernel user - (mostly) lockless allocation - a way to manage granularity (or order of allocation) Most of this is already provided by genalloc, which is what I ended up almost re-implementing, before being pointed to it :-) I only had to add the tracking of end of allocations, which is what the patch 1/6 does - as side note, is anybody maintaining it? I could not find an entry in MAINTAINERS As I mentioned above, using vmalloc adds even an extra layer of protection. The major downside is the increased TLB use, however this is not so relevant for the volumes of data that I had to deal with so far: only few 4K pages. But you might have in mind something else. I'd be interested to know what and what would be an obstacle in using pmalloc. Maybe it can be solved. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On 26/01/18 07:35, Matthew Wilcox wrote: > On Wed, Jan 24, 2018 at 08:10:53PM +0100, Jann Horn wrote: >> I'm not entirely convinced by the approach of marking small parts of >> kernel memory as readonly for hardening. > > It depends how significant the data stored in there are. For example, > storing function pointers in read-only memory provides significant > hardening. > >> You're allocating with vmalloc(), which, as far as I know, establishes >> a second mapping in the vmalloc area for pages that are already mapped >> as RW through the physmap. AFAICS, later, when you're trying to make >> pages readonly, you're only changing the protections on the second >> mapping in the vmalloc area, therefore leaving the memory writable >> through the physmap. Is that correct? If so, please either document >> the reasoning why this is okay or change it. > > Yes, this is still vulnerable to attacks through the physmap. That's also > true for marking structs as const. We should probably fix that at some > point, but at least they're not vulnerable to heap overruns by small > amounts ... you have to be able to overrun some other array by terabytes. Actually, I think there is something to say in favor of using a vmalloc based approach, precisely because of the physmap :-P If I understood correctly, the physmap is primarily meant to speed up access to physical memory through the TLB. In particular, for kmalloc based allocations. Which means that, to perform a physmap-based attack to a kmalloced allocation, one needs to know: - the address of the target variable in the kmalloc range - the randomized offset of the kernel - the location of the physmap But, for a vmalloc based allocation, there is one extra hoop: since the mapping is really per page, now the attacker has actually to walk the page table, to figure out where to poke in the physmap. One more thought about physmap: does it map also code? Because, if it does, and one wants to use it for an attack, isn't it easier to look for some security test and replace a bne with be or equivalent? > It's worth having a discussion about whether we want the pmalloc API > or whether we want a slab-based API. pmalloc is meant to be useful where the attack surface is made up of lots of small allocations - my first use case was the SE Linux policy DB, where there is a variety of elements being allocated, in large amount. To the point where having ready made caches would be wasteful. Then there is the issue I already mentioned about arm/arm64 which would require to break down large mappings, which seems to be against current policy, as described in my previous mail: http://www.openwall.com/lists/kernel-hardening/2018/01/24/11 I do not know exactly what you have in mind wrt slab, but my impression is that it will most likely gravitate toward the pmalloc implementation. It will need: - "pools" or anyway some means to lock only a certain group of pages, related to a specific kernel user - (mostly) lockless allocation - a way to manage granularity (or order of allocation) Most of this is already provided by genalloc, which is what I ended up almost re-implementing, before being pointed to it :-) I only had to add the tracking of end of allocations, which is what the patch 1/6 does - as side note, is anybody maintaining it? I could not find an entry in MAINTAINERS As I mentioned above, using vmalloc adds even an extra layer of protection. The major downside is the increased TLB use, however this is not so relevant for the volumes of data that I had to deal with so far: only few 4K pages. But you might have in mind something else. I'd be interested to know what and what would be an obstacle in using pmalloc. Maybe it can be solved. -- igor
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Wed, Jan 24, 2018 at 08:10:53PM +0100, Jann Horn wrote: > I'm not entirely convinced by the approach of marking small parts of > kernel memory as readonly for hardening. It depends how significant the data stored in there are. For example, storing function pointers in read-only memory provides significant hardening. > You're allocating with vmalloc(), which, as far as I know, establishes > a second mapping in the vmalloc area for pages that are already mapped > as RW through the physmap. AFAICS, later, when you're trying to make > pages readonly, you're only changing the protections on the second > mapping in the vmalloc area, therefore leaving the memory writable > through the physmap. Is that correct? If so, please either document > the reasoning why this is okay or change it. Yes, this is still vulnerable to attacks through the physmap. That's also true for marking structs as const. We should probably fix that at some point, but at least they're not vulnerable to heap overruns by small amounts ... you have to be able to overrun some other array by terabytes. It's worth having a discussion about whether we want the pmalloc API or whether we want a slab-based API. We can have a separate discussion about an API to remove pages from the physmap.
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Wed, Jan 24, 2018 at 08:10:53PM +0100, Jann Horn wrote: > I'm not entirely convinced by the approach of marking small parts of > kernel memory as readonly for hardening. It depends how significant the data stored in there are. For example, storing function pointers in read-only memory provides significant hardening. > You're allocating with vmalloc(), which, as far as I know, establishes > a second mapping in the vmalloc area for pages that are already mapped > as RW through the physmap. AFAICS, later, when you're trying to make > pages readonly, you're only changing the protections on the second > mapping in the vmalloc area, therefore leaving the memory writable > through the physmap. Is that correct? If so, please either document > the reasoning why this is okay or change it. Yes, this is still vulnerable to attacks through the physmap. That's also true for marking structs as const. We should probably fix that at some point, but at least they're not vulnerable to heap overruns by small amounts ... you have to be able to overrun some other array by terabytes. It's worth having a discussion about whether we want the pmalloc API or whether we want a slab-based API. We can have a separate discussion about an API to remove pages from the physmap.
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Thu, Jan 25, 2018 at 10:14:28AM -0500, Boris Lukashev wrote: > On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppawrote: [...] > DMA/physmap access coupled with a knowledge of which virtual mappings > are in the physical space should be enough for an attacker to bypass > the gating mechanism this work imposes. Not trivial, but not > impossible. Since there's no way to prevent that sort of access in > current hardware (especially something like a NIC or GPU working > independently of the CPU altogether) I am not saying this is impossible but this is unlikely they are several mecanisms. First you have IOMMU it has been defaulted to on by OEM for last few years (it use to be enabled only on server for virtualization). Which means that a given device only can access memory that is mapped to it through the IOMMU page table (usualy each device get their own distinct IOMMU page table). Then on device like GPU you have an MMU (no GPU without an MMU for the last 10 years or more). The MMU is under the control of the kernel driver of the GPU and for the open source driver we try hard to make sure it can not be abuse and circumvent by userspace ie we restrict userspace process to only access memory they own. I am not saying that this can not happen but that we are trying our best to avoid it. Cheers, Jérôme
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Thu, Jan 25, 2018 at 10:14:28AM -0500, Boris Lukashev wrote: > On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppa wrote: [...] > DMA/physmap access coupled with a knowledge of which virtual mappings > are in the physical space should be enough for an attacker to bypass > the gating mechanism this work imposes. Not trivial, but not > impossible. Since there's no way to prevent that sort of access in > current hardware (especially something like a NIC or GPU working > independently of the CPU altogether) I am not saying this is impossible but this is unlikely they are several mecanisms. First you have IOMMU it has been defaulted to on by OEM for last few years (it use to be enabled only on server for virtualization). Which means that a given device only can access memory that is mapped to it through the IOMMU page table (usualy each device get their own distinct IOMMU page table). Then on device like GPU you have an MMU (no GPU without an MMU for the last 10 years or more). The MMU is under the control of the kernel driver of the GPU and for the open source driver we try hard to make sure it can not be abuse and circumvent by userspace ie we restrict userspace process to only access memory they own. I am not saying that this can not happen but that we are trying our best to avoid it. Cheers, Jérôme
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppawrote: > > Hi, > > thanks for the review. My reply below. > > On 24/01/18 21:10, Jann Horn wrote: > > > I'm not entirely convinced by the approach of marking small parts of > > kernel memory as readonly for hardening. > > Because of the physmap you mention later? > > Regarding small parts vs big parts (what is big enough?) I did propose > the use of a custom zone at the very beginning, however I met 2 objections: > > 1. It's not a special case and there was no will to reserve another zone >This might be mitigated by aliasing with a zone that is already >defined, but not in use. For example DMA or DMA32. >But it looks like a good way to replicate the confusion that is page >struct. Anyway, I found the next objection more convincing. > > 2. What would be the size of this zone? It would become something that >is really application specific. At the very least it should become a >command line parameter. A distro would have to allocate a lot of >memory for it, because it cannot really know upfront what its users >will do. But, most likely, the vast majority of users would never >need that much. > > If you have some idea of how to address these objections without using > vmalloc, or at least without using the same page provider that vmalloc > is using now, I'd be interested to hear it. > > Besides the double mapping problem, the major benefit I can see from > having a contiguous area is that it simplifies the hardened user copy > verification, because there is a fixed range to test for overlap. > > > Comments on some details are inline. > > thank you > > >> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h > >> index 1e5d8c3..116d280 100644 > >> --- a/include/linux/vmalloc.h > >> +++ b/include/linux/vmalloc.h > >> @@ -20,6 +20,7 @@ struct notifier_block;/* in notifier.h */ > >> #define VM_UNINITIALIZED 0x0020 /* vm_struct is not fully > >> initialized */ > >> #define VM_NO_GUARD0x0040 /* don't add guard page */ > >> #define VM_KASAN 0x0080 /* has allocated kasan > >> shadow memory */ > >> +#define VM_PMALLOC 0x0100 /* pmalloc area - see docs > >> */ > > > > Is "see docs" specific enough to actually guide the reader to the > > right documentation? > > The doc file is named pmalloc.txt, but I can be more explicit. > > >> +#define pmalloc_attr_init(data, attr_name) \ > >> +do { \ > >> + sysfs_attr_init(>attr_##attr_name.attr); \ > >> + data->attr_##attr_name.attr.name = #attr_name; \ > >> + data->attr_##attr_name.attr.mode = VERIFY_OCTAL_PERMISSIONS(0444); > >> \ > >> + data->attr_##attr_name.show = pmalloc_pool_show_##attr_name; \ > >> +} while (0) > > > > Is there a good reason for making all these files mode 0444 (as > > opposed to setting them to 0400 and then allowing userspace to make > > them accessible if desired)? /proc/slabinfo contains vaguely similar > > data and is mode 0400 (or mode 0600, depending on the kernel config) > > AFAICS. > > ok, you do have a point, so far I have been mostly focusing on the > > "drop-in replacement for kmalloc" aspect. > > >> +void *pmalloc(struct gen_pool *pool, size_t size, gfp_t gfp) > >> +{ > > [...] > >> + /* Expand pool */ > >> + chunk_size = roundup(size, PAGE_SIZE); > >> + chunk = vmalloc(chunk_size); > > > > You're allocating with vmalloc(), which, as far as I know, establishes > > a second mapping in the vmalloc area for pages that are already mapped > > as RW through the physmap. AFAICS, later, when you're trying to make > > pages readonly, you're only changing the protections on the second > > mapping in the vmalloc area, therefore leaving the memory writable > > through the physmap. Is that correct? If so, please either document > > the reasoning why this is okay or change it. > > About why vmalloc as backend for pmalloc, please refer to this: > > http://www.openwall.com/lists/kernel-hardening/2018/01/24/11 > > I tried to give a short summary of what took me toward vmalloc. > vmalloc is also a convenient way of obtaining arbitrarily (within > reason) large amounts of virtually contiguous memory. > > Your objection is toward the unprotected access, through the alternate > mapping, rather than to the idea of having pools that can be protected > individually, right? > > In the mail I linked, I explained that I could not use kmalloc because > of the problem of splitting huge pages on ARM. > > kmalloc does require the physmap, for performance reason. > > However, vmalloc is already doing mapping of individual pages, because > it must ensure that they are virtually contiguous, so would it be > possible to have vmalloc _always_ outside of the physmap? > > If I have understood correctly, the actual extension of physmap is > highly architecture and platform dependant, so it might be (but I have > not checked)
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Thu, Jan 25, 2018 at 6:59 AM, Igor Stoppa wrote: > > Hi, > > thanks for the review. My reply below. > > On 24/01/18 21:10, Jann Horn wrote: > > > I'm not entirely convinced by the approach of marking small parts of > > kernel memory as readonly for hardening. > > Because of the physmap you mention later? > > Regarding small parts vs big parts (what is big enough?) I did propose > the use of a custom zone at the very beginning, however I met 2 objections: > > 1. It's not a special case and there was no will to reserve another zone >This might be mitigated by aliasing with a zone that is already >defined, but not in use. For example DMA or DMA32. >But it looks like a good way to replicate the confusion that is page >struct. Anyway, I found the next objection more convincing. > > 2. What would be the size of this zone? It would become something that >is really application specific. At the very least it should become a >command line parameter. A distro would have to allocate a lot of >memory for it, because it cannot really know upfront what its users >will do. But, most likely, the vast majority of users would never >need that much. > > If you have some idea of how to address these objections without using > vmalloc, or at least without using the same page provider that vmalloc > is using now, I'd be interested to hear it. > > Besides the double mapping problem, the major benefit I can see from > having a contiguous area is that it simplifies the hardened user copy > verification, because there is a fixed range to test for overlap. > > > Comments on some details are inline. > > thank you > > >> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h > >> index 1e5d8c3..116d280 100644 > >> --- a/include/linux/vmalloc.h > >> +++ b/include/linux/vmalloc.h > >> @@ -20,6 +20,7 @@ struct notifier_block;/* in notifier.h */ > >> #define VM_UNINITIALIZED 0x0020 /* vm_struct is not fully > >> initialized */ > >> #define VM_NO_GUARD0x0040 /* don't add guard page */ > >> #define VM_KASAN 0x0080 /* has allocated kasan > >> shadow memory */ > >> +#define VM_PMALLOC 0x0100 /* pmalloc area - see docs > >> */ > > > > Is "see docs" specific enough to actually guide the reader to the > > right documentation? > > The doc file is named pmalloc.txt, but I can be more explicit. > > >> +#define pmalloc_attr_init(data, attr_name) \ > >> +do { \ > >> + sysfs_attr_init(>attr_##attr_name.attr); \ > >> + data->attr_##attr_name.attr.name = #attr_name; \ > >> + data->attr_##attr_name.attr.mode = VERIFY_OCTAL_PERMISSIONS(0444); > >> \ > >> + data->attr_##attr_name.show = pmalloc_pool_show_##attr_name; \ > >> +} while (0) > > > > Is there a good reason for making all these files mode 0444 (as > > opposed to setting them to 0400 and then allowing userspace to make > > them accessible if desired)? /proc/slabinfo contains vaguely similar > > data and is mode 0400 (or mode 0600, depending on the kernel config) > > AFAICS. > > ok, you do have a point, so far I have been mostly focusing on the > > "drop-in replacement for kmalloc" aspect. > > >> +void *pmalloc(struct gen_pool *pool, size_t size, gfp_t gfp) > >> +{ > > [...] > >> + /* Expand pool */ > >> + chunk_size = roundup(size, PAGE_SIZE); > >> + chunk = vmalloc(chunk_size); > > > > You're allocating with vmalloc(), which, as far as I know, establishes > > a second mapping in the vmalloc area for pages that are already mapped > > as RW through the physmap. AFAICS, later, when you're trying to make > > pages readonly, you're only changing the protections on the second > > mapping in the vmalloc area, therefore leaving the memory writable > > through the physmap. Is that correct? If so, please either document > > the reasoning why this is okay or change it. > > About why vmalloc as backend for pmalloc, please refer to this: > > http://www.openwall.com/lists/kernel-hardening/2018/01/24/11 > > I tried to give a short summary of what took me toward vmalloc. > vmalloc is also a convenient way of obtaining arbitrarily (within > reason) large amounts of virtually contiguous memory. > > Your objection is toward the unprotected access, through the alternate > mapping, rather than to the idea of having pools that can be protected > individually, right? > > In the mail I linked, I explained that I could not use kmalloc because > of the problem of splitting huge pages on ARM. > > kmalloc does require the physmap, for performance reason. > > However, vmalloc is already doing mapping of individual pages, because > it must ensure that they are virtually contiguous, so would it be > possible to have vmalloc _always_ outside of the physmap? > > If I have understood correctly, the actual extension of physmap is > highly architecture and platform dependant, so it might be (but I have > not checked) that in some cases (like
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
Hi, thanks for the review. My reply below. On 24/01/18 21:10, Jann Horn wrote: > I'm not entirely convinced by the approach of marking small parts of > kernel memory as readonly for hardening. Because of the physmap you mention later? Regarding small parts vs big parts (what is big enough?) I did propose the use of a custom zone at the very beginning, however I met 2 objections: 1. It's not a special case and there was no will to reserve another zone This might be mitigated by aliasing with a zone that is already defined, but not in use. For example DMA or DMA32. But it looks like a good way to replicate the confusion that is page struct. Anyway, I found the next objection more convincing. 2. What would be the size of this zone? It would become something that is really application specific. At the very least it should become a command line parameter. A distro would have to allocate a lot of memory for it, because it cannot really know upfront what its users will do. But, most likely, the vast majority of users would never need that much. If you have some idea of how to address these objections without using vmalloc, or at least without using the same page provider that vmalloc is using now, I'd be interested to hear it. Besides the double mapping problem, the major benefit I can see from having a contiguous area is that it simplifies the hardened user copy verification, because there is a fixed range to test for overlap. > Comments on some details are inline. thank you >> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h >> index 1e5d8c3..116d280 100644 >> --- a/include/linux/vmalloc.h >> +++ b/include/linux/vmalloc.h >> @@ -20,6 +20,7 @@ struct notifier_block;/* in notifier.h */ >> #define VM_UNINITIALIZED 0x0020 /* vm_struct is not fully >> initialized */ >> #define VM_NO_GUARD0x0040 /* don't add guard page */ >> #define VM_KASAN 0x0080 /* has allocated kasan >> shadow memory */ >> +#define VM_PMALLOC 0x0100 /* pmalloc area - see docs */ > > Is "see docs" specific enough to actually guide the reader to the > right documentation? The doc file is named pmalloc.txt, but I can be more explicit. >> +#define pmalloc_attr_init(data, attr_name) \ >> +do { \ >> + sysfs_attr_init(>attr_##attr_name.attr); \ >> + data->attr_##attr_name.attr.name = #attr_name; \ >> + data->attr_##attr_name.attr.mode = VERIFY_OCTAL_PERMISSIONS(0444); \ >> + data->attr_##attr_name.show = pmalloc_pool_show_##attr_name; \ >> +} while (0) > > Is there a good reason for making all these files mode 0444 (as > opposed to setting them to 0400 and then allowing userspace to make > them accessible if desired)? /proc/slabinfo contains vaguely similar > data and is mode 0400 (or mode 0600, depending on the kernel config) > AFAICS. ok, you do have a point, so far I have been mostly focusing on the "drop-in replacement for kmalloc" aspect. >> +void *pmalloc(struct gen_pool *pool, size_t size, gfp_t gfp) >> +{ > [...] >> + /* Expand pool */ >> + chunk_size = roundup(size, PAGE_SIZE); >> + chunk = vmalloc(chunk_size); > > You're allocating with vmalloc(), which, as far as I know, establishes > a second mapping in the vmalloc area for pages that are already mapped > as RW through the physmap. AFAICS, later, when you're trying to make > pages readonly, you're only changing the protections on the second > mapping in the vmalloc area, therefore leaving the memory writable > through the physmap. Is that correct? If so, please either document > the reasoning why this is okay or change it. About why vmalloc as backend for pmalloc, please refer to this: http://www.openwall.com/lists/kernel-hardening/2018/01/24/11 I tried to give a short summary of what took me toward vmalloc. vmalloc is also a convenient way of obtaining arbitrarily (within reason) large amounts of virtually contiguous memory. Your objection is toward the unprotected access, through the alternate mapping, rather than to the idea of having pools that can be protected individually, right? In the mail I linked, I explained that I could not use kmalloc because of the problem of splitting huge pages on ARM. kmalloc does require the physmap, for performance reason. However, vmalloc is already doing mapping of individual pages, because it must ensure that they are virtually contiguous, so would it be possible to have vmalloc _always_ outside of the physmap? If I have understood correctly, the actual extension of physmap is highly architecture and platform dependant, so it might be (but I have not checked) that in some cases (like some 32bit systems) vmalloc is typically outside of physmap, but probably that is not the case on 64bit? Also, I need to understand how physmap works against vmalloc vs how it works against kernel text and const/__ro_after_init sections. Can they also be accessed
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
Hi, thanks for the review. My reply below. On 24/01/18 21:10, Jann Horn wrote: > I'm not entirely convinced by the approach of marking small parts of > kernel memory as readonly for hardening. Because of the physmap you mention later? Regarding small parts vs big parts (what is big enough?) I did propose the use of a custom zone at the very beginning, however I met 2 objections: 1. It's not a special case and there was no will to reserve another zone This might be mitigated by aliasing with a zone that is already defined, but not in use. For example DMA or DMA32. But it looks like a good way to replicate the confusion that is page struct. Anyway, I found the next objection more convincing. 2. What would be the size of this zone? It would become something that is really application specific. At the very least it should become a command line parameter. A distro would have to allocate a lot of memory for it, because it cannot really know upfront what its users will do. But, most likely, the vast majority of users would never need that much. If you have some idea of how to address these objections without using vmalloc, or at least without using the same page provider that vmalloc is using now, I'd be interested to hear it. Besides the double mapping problem, the major benefit I can see from having a contiguous area is that it simplifies the hardened user copy verification, because there is a fixed range to test for overlap. > Comments on some details are inline. thank you >> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h >> index 1e5d8c3..116d280 100644 >> --- a/include/linux/vmalloc.h >> +++ b/include/linux/vmalloc.h >> @@ -20,6 +20,7 @@ struct notifier_block;/* in notifier.h */ >> #define VM_UNINITIALIZED 0x0020 /* vm_struct is not fully >> initialized */ >> #define VM_NO_GUARD0x0040 /* don't add guard page */ >> #define VM_KASAN 0x0080 /* has allocated kasan >> shadow memory */ >> +#define VM_PMALLOC 0x0100 /* pmalloc area - see docs */ > > Is "see docs" specific enough to actually guide the reader to the > right documentation? The doc file is named pmalloc.txt, but I can be more explicit. >> +#define pmalloc_attr_init(data, attr_name) \ >> +do { \ >> + sysfs_attr_init(>attr_##attr_name.attr); \ >> + data->attr_##attr_name.attr.name = #attr_name; \ >> + data->attr_##attr_name.attr.mode = VERIFY_OCTAL_PERMISSIONS(0444); \ >> + data->attr_##attr_name.show = pmalloc_pool_show_##attr_name; \ >> +} while (0) > > Is there a good reason for making all these files mode 0444 (as > opposed to setting them to 0400 and then allowing userspace to make > them accessible if desired)? /proc/slabinfo contains vaguely similar > data and is mode 0400 (or mode 0600, depending on the kernel config) > AFAICS. ok, you do have a point, so far I have been mostly focusing on the "drop-in replacement for kmalloc" aspect. >> +void *pmalloc(struct gen_pool *pool, size_t size, gfp_t gfp) >> +{ > [...] >> + /* Expand pool */ >> + chunk_size = roundup(size, PAGE_SIZE); >> + chunk = vmalloc(chunk_size); > > You're allocating with vmalloc(), which, as far as I know, establishes > a second mapping in the vmalloc area for pages that are already mapped > as RW through the physmap. AFAICS, later, when you're trying to make > pages readonly, you're only changing the protections on the second > mapping in the vmalloc area, therefore leaving the memory writable > through the physmap. Is that correct? If so, please either document > the reasoning why this is okay or change it. About why vmalloc as backend for pmalloc, please refer to this: http://www.openwall.com/lists/kernel-hardening/2018/01/24/11 I tried to give a short summary of what took me toward vmalloc. vmalloc is also a convenient way of obtaining arbitrarily (within reason) large amounts of virtually contiguous memory. Your objection is toward the unprotected access, through the alternate mapping, rather than to the idea of having pools that can be protected individually, right? In the mail I linked, I explained that I could not use kmalloc because of the problem of splitting huge pages on ARM. kmalloc does require the physmap, for performance reason. However, vmalloc is already doing mapping of individual pages, because it must ensure that they are virtually contiguous, so would it be possible to have vmalloc _always_ outside of the physmap? If I have understood correctly, the actual extension of physmap is highly architecture and platform dependant, so it might be (but I have not checked) that in some cases (like some 32bit systems) vmalloc is typically outside of physmap, but probably that is not the case on 64bit? Also, I need to understand how physmap works against vmalloc vs how it works against kernel text and const/__ro_after_init sections. Can they also be accessed
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Wed, Jan 24, 2018 at 6:56 PM, Igor Stoppawrote: > The MMU available in many systems running Linux can often provide R/O > protection to the memory pages it handles. > > However, the MMU-based protection works efficiently only when said pages > contain exclusively data that will not need further modifications. > > Statically allocated variables can be segregated into a dedicated > section, but this does not sit very well with dynamically allocated > ones. > > Dynamic allocation does not provide, currently, any means for grouping > variables in memory pages that would contain exclusively data suitable > for conversion to read only access mode. > > The allocator here provided (pmalloc - protectable memory allocator) > introduces the concept of pools of protectable memory. > > A module can request a pool and then refer any allocation request to the > pool handler it has received. > > Once all the chunks of memory associated to a specific pool are > initialized, the pool can be protected. I'm not entirely convinced by the approach of marking small parts of kernel memory as readonly for hardening. Comments on some details are inline. > diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h > index 1e5d8c3..116d280 100644 > --- a/include/linux/vmalloc.h > +++ b/include/linux/vmalloc.h > @@ -20,6 +20,7 @@ struct notifier_block;/* in notifier.h */ > #define VM_UNINITIALIZED 0x0020 /* vm_struct is not fully > initialized */ > #define VM_NO_GUARD0x0040 /* don't add guard page */ > #define VM_KASAN 0x0080 /* has allocated kasan shadow > memory */ > +#define VM_PMALLOC 0x0100 /* pmalloc area - see docs */ Is "see docs" specific enough to actually guide the reader to the right documentation? > +#define pmalloc_attr_init(data, attr_name) \ > +do { \ > + sysfs_attr_init(>attr_##attr_name.attr); \ > + data->attr_##attr_name.attr.name = #attr_name; \ > + data->attr_##attr_name.attr.mode = VERIFY_OCTAL_PERMISSIONS(0444); \ > + data->attr_##attr_name.show = pmalloc_pool_show_##attr_name; \ > +} while (0) Is there a good reason for making all these files mode 0444 (as opposed to setting them to 0400 and then allowing userspace to make them accessible if desired)? /proc/slabinfo contains vaguely similar data and is mode 0400 (or mode 0600, depending on the kernel config) AFAICS. > +void *pmalloc(struct gen_pool *pool, size_t size, gfp_t gfp) > +{ [...] > + /* Expand pool */ > + chunk_size = roundup(size, PAGE_SIZE); > + chunk = vmalloc(chunk_size); You're allocating with vmalloc(), which, as far as I know, establishes a second mapping in the vmalloc area for pages that are already mapped as RW through the physmap. AFAICS, later, when you're trying to make pages readonly, you're only changing the protections on the second mapping in the vmalloc area, therefore leaving the memory writable through the physmap. Is that correct? If so, please either document the reasoning why this is okay or change it.
Re: [kernel-hardening] [PATCH 4/6] Protectable Memory
On Wed, Jan 24, 2018 at 6:56 PM, Igor Stoppa wrote: > The MMU available in many systems running Linux can often provide R/O > protection to the memory pages it handles. > > However, the MMU-based protection works efficiently only when said pages > contain exclusively data that will not need further modifications. > > Statically allocated variables can be segregated into a dedicated > section, but this does not sit very well with dynamically allocated > ones. > > Dynamic allocation does not provide, currently, any means for grouping > variables in memory pages that would contain exclusively data suitable > for conversion to read only access mode. > > The allocator here provided (pmalloc - protectable memory allocator) > introduces the concept of pools of protectable memory. > > A module can request a pool and then refer any allocation request to the > pool handler it has received. > > Once all the chunks of memory associated to a specific pool are > initialized, the pool can be protected. I'm not entirely convinced by the approach of marking small parts of kernel memory as readonly for hardening. Comments on some details are inline. > diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h > index 1e5d8c3..116d280 100644 > --- a/include/linux/vmalloc.h > +++ b/include/linux/vmalloc.h > @@ -20,6 +20,7 @@ struct notifier_block;/* in notifier.h */ > #define VM_UNINITIALIZED 0x0020 /* vm_struct is not fully > initialized */ > #define VM_NO_GUARD0x0040 /* don't add guard page */ > #define VM_KASAN 0x0080 /* has allocated kasan shadow > memory */ > +#define VM_PMALLOC 0x0100 /* pmalloc area - see docs */ Is "see docs" specific enough to actually guide the reader to the right documentation? > +#define pmalloc_attr_init(data, attr_name) \ > +do { \ > + sysfs_attr_init(>attr_##attr_name.attr); \ > + data->attr_##attr_name.attr.name = #attr_name; \ > + data->attr_##attr_name.attr.mode = VERIFY_OCTAL_PERMISSIONS(0444); \ > + data->attr_##attr_name.show = pmalloc_pool_show_##attr_name; \ > +} while (0) Is there a good reason for making all these files mode 0444 (as opposed to setting them to 0400 and then allowing userspace to make them accessible if desired)? /proc/slabinfo contains vaguely similar data and is mode 0400 (or mode 0600, depending on the kernel config) AFAICS. > +void *pmalloc(struct gen_pool *pool, size_t size, gfp_t gfp) > +{ [...] > + /* Expand pool */ > + chunk_size = roundup(size, PAGE_SIZE); > + chunk = vmalloc(chunk_size); You're allocating with vmalloc(), which, as far as I know, establishes a second mapping in the vmalloc area for pages that are already mapped as RW through the physmap. AFAICS, later, when you're trying to make pages readonly, you're only changing the protections on the second mapping in the vmalloc area, therefore leaving the memory writable through the physmap. Is that correct? If so, please either document the reasoning why this is okay or change it.