Re: [kernel-hardening] Re: [PATCH] fork: make whole stack_canary random

On Mon, 2016-10-31 at 22:10 +0100, Florian Weimer wrote: > * Daniel Micay: >  > > > It makes a lot of sense on x86_64 where it means the canary is > > > still 56 bits. Also, you want -fstack-check for protecting again > > > stack overflows rather than s

Re: [kernel-hardening] Re: [PATCH] fork: make whole stack_canary random

> It makes a lot of sense on x86_64 where it means the canary is still > 56 > bits. Also, you want -fstack-check for protecting again stack > overflows > rather than stack *buffer* overflow. SSP won't really help you in that > regard. Sadly, while -fstack-check now works well in GCC 6 with little

Re: [kernel-hardening] Re: [PATCH] fork: make whole stack_canary random

> It makes a lot of sense on x86_64 where it means the canary is still > 56 > bits. Also, you want -fstack-check for protecting again stack > overflows > rather than stack *buffer* overflow. SSP won't really help you in that > regard. Sadly, while -fstack-check now works well in GCC 6 with little

Re: [kernel-hardening] Re: [PATCH] fork: make whole stack_canary random

On Mon, 2016-10-31 at 21:45 +0100, Florian Weimer wrote: > * Jann Horn: > > > On Mon, Oct 31, 2016 at 09:04:02AM -0700, Kees Cook wrote: > > > On Mon, Oct 31, 2016 at 7:04 AM, Jann Horn wrote: > > > > On machines with sizeof(unsigned long)==8, this ensures that the > > > > more >

Re: [kernel-hardening] Re: [PATCH] fork: make whole stack_canary random

On Mon, 2016-10-31 at 21:45 +0100, Florian Weimer wrote: > * Jann Horn: > > > On Mon, Oct 31, 2016 at 09:04:02AM -0700, Kees Cook wrote: > > > On Mon, Oct 31, 2016 at 7:04 AM, Jann Horn wrote: > > > > On machines with sizeof(unsigned long)==8, this ensures that the > > > > more > > > >

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Wed, 2016-10-19 at 07:26 -0300, Arnaldo Carvalho de Melo wrote: > > But self profiling JITs would be useful for non-developers, on Android > (anywhere, really), and for that it would require being able to at > least, well, self profile, using sys_perf_event_open() by a normal > process,

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Wed, 2016-10-19 at 07:26 -0300, Arnaldo Carvalho de Melo wrote: > > But self profiling JITs would be useful for non-developers, on Android > (anywhere, really), and for that it would require being able to at > least, well, self profile, using sys_perf_event_open() by a normal > process,

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Wed, 2016-10-19 at 10:41 +0100, Mark Rutland wrote: > On Mon, Oct 17, 2016 at 10:54:33AM -0400, Daniel Micay wrote: > > On Mon, 2016-10-17 at 14:44 +0100, Mark Rutland wrote: > > > It's also my understanding that for Android, perf_event_paranoid > > > is > >

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Wed, 2016-10-19 at 10:41 +0100, Mark Rutland wrote: > On Mon, Oct 17, 2016 at 10:54:33AM -0400, Daniel Micay wrote: > > On Mon, 2016-10-17 at 14:44 +0100, Mark Rutland wrote: > > > It's also my understanding that for Android, perf_event_paranoid > > > is > >

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Tue, 2016-10-18 at 13:48 -0700, Kees Cook wrote: > On Mon, Oct 17, 2016 at 6:44 AM, Mark Rutland > wrote: > > Hi, > > > > Attempt to revive discussions below... > > > > On Wed, Jul 27, 2016 at 07:45:46AM -0700, Jeff Vander Stoep wrote: > > > When

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Tue, 2016-10-18 at 13:48 -0700, Kees Cook wrote: > On Mon, Oct 17, 2016 at 6:44 AM, Mark Rutland > wrote: > > Hi, > > > > Attempt to revive discussions below... > > > > On Wed, Jul 27, 2016 at 07:45:46AM -0700, Jeff Vander Stoep wrote: > > > When kernel.perf_event_paranoid is set to 3 (or

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Mon, 2016-10-17 at 14:44 +0100, Mark Rutland wrote: > Hi, > > Attempt to revive discussions below... > > On Wed, Jul 27, 2016 at 07:45:46AM -0700, Jeff Vander Stoep wrote: > > When kernel.perf_event_paranoid is set to 3 (or greater), disallow > > all access to performance events by users

Re: [kernel-hardening] [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Mon, 2016-10-17 at 14:44 +0100, Mark Rutland wrote: > Hi, > > Attempt to revive discussions below... > > On Wed, Jul 27, 2016 at 07:45:46AM -0700, Jeff Vander Stoep wrote: > > When kernel.perf_event_paranoid is set to 3 (or greater), disallow > > all access to performance events by users

Re: [kernel-hardening] Re: [PATCH 2/2] arm: apply more __ro_after_init

On Wed, 2016-08-10 at 10:43 +0100, Russell King - ARM Linux wrote: > On Fri, Jun 03, 2016 at 11:40:24AM -0700, Kees Cook wrote: > > > > @@ -1309,16 +1309,11 @@ void __init arm_mm_memblock_reserve(void) > >   * Any other function or debugging method which may touch any > > device _will_ > >   *

Re: [kernel-hardening] Re: [PATCH 2/2] arm: apply more __ro_after_init

On Wed, 2016-08-10 at 10:43 +0100, Russell King - ARM Linux wrote: > On Fri, Jun 03, 2016 at 11:40:24AM -0700, Kees Cook wrote: > > > > @@ -1309,16 +1309,11 @@ void __init arm_mm_memblock_reserve(void) > >   * Any other function or debugging method which may touch any > > device _will_ > >   *

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> My claim was not that the mainline code was impressively perfect, but > rather that the vendor code was worse, countering a prior claim > otherwise. Hence, reality. You're arguing with a straw man. I was responding to a comment about out-of-tree code, not generic architecture perf drivers vs.

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> My claim was not that the mainline code was impressively perfect, but > rather that the vendor code was worse, countering a prior claim > otherwise. Hence, reality. You're arguing with a straw man. I was responding to a comment about out-of-tree code, not generic architecture perf drivers vs.

Re: [kernel-hardening] [PATCH] [RFC] Introduce mmap randomization

On Thu, 2016-08-04 at 16:55 +, Roberts, William C wrote: > > > > -Original Message- > > From: Daniel Micay [mailto:danielmi...@gmail.com] > > Sent: Thursday, August 4, 2016 9:53 AM > > To: kernel-harden...@lists.openwall.com; ja...@lakedaemon.net; >

Re: [kernel-hardening] [PATCH] [RFC] Introduce mmap randomization

On Thu, 2016-08-04 at 16:55 +, Roberts, William C wrote: > > > > -Original Message- > > From: Daniel Micay [mailto:danielmi...@gmail.com] > > Sent: Thursday, August 4, 2016 9:53 AM > > To: kernel-harden...@lists.openwall.com; ja...@lakedaemon.net; >

Re: [kernel-hardening] [PATCH] [RFC] Introduce mmap randomization

On Tue, 2016-07-26 at 11:22 -0700, william.c.robe...@intel.com wrote: > The recent get_random_long() change in get_random_range() and then the > subsequent patches Jason put out, all stemmed from my tinkering > with the concept of randomizing mmap. > > Any feedback would be greatly appreciated,

Re: [kernel-hardening] [PATCH] [RFC] Introduce mmap randomization

On Tue, 2016-07-26 at 11:22 -0700, william.c.robe...@intel.com wrote: > The recent get_random_long() change in get_random_range() and then the > subsequent patches Jason put out, all stemmed from my tinkering > with the concept of randomizing mmap. > > Any feedback would be greatly appreciated,

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Thu, 2016-08-04 at 17:10 +0100, Mark Rutland wrote: > On Thu, Aug 04, 2016 at 11:44:28AM -0400, Daniel Micay wrote: > > > > Qualcomm's drivers might be lower quality than core kernel code, but > > they're way above the baseline set by mainline kernel drivers... > >

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Thu, 2016-08-04 at 17:10 +0100, Mark Rutland wrote: > On Thu, Aug 04, 2016 at 11:44:28AM -0400, Daniel Micay wrote: > > > > Qualcomm's drivers might be lower quality than core kernel code, but > > they're way above the baseline set by mainline kernel drivers... > >

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Thu, 2016-08-04 at 16:11 +0200, Peter Zijlstra wrote: > On Thu, Aug 04, 2016 at 09:45:23AM -0400, Daniel Micay wrote: > > > > Qualcomm's perf driver is out-of-tree along with most of their other > > drivers.  > > > So you're asking us to maim upstream

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Thu, 2016-08-04 at 16:11 +0200, Peter Zijlstra wrote: > On Thu, Aug 04, 2016 at 09:45:23AM -0400, Daniel Micay wrote: > > > > Qualcomm's perf driver is out-of-tree along with most of their other > > drivers.  > > > So you're asking us to maim upstream

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Thu, 2016-08-04 at 11:28 +0100, Mark Rutland wrote: > On Wed, Aug 03, 2016 at 03:36:16PM -0400, Daniel Micay wrote: > > > > There's a lot of architecture and vendor specific perf events code > > and > > lots of bleeding edge features. On Android, a lot of the perf

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Thu, 2016-08-04 at 11:28 +0100, Mark Rutland wrote: > On Wed, Aug 03, 2016 at 03:36:16PM -0400, Daniel Micay wrote: > > > > There's a lot of architecture and vendor specific perf events code > > and > > lots of bleeding edge features. On Android, a lot of the perf

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> One of the strengths of linux is applications of features the authors > of > the software had not imagined.  Your proposals seem to be trying to > put > the world a tiny little box where if someone had not imagined and > preapproved a use of a feature it should not happen.   Let's please > avoid

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> One of the strengths of linux is applications of features the authors > of > the software had not imagined.  Your proposals seem to be trying to > put > the world a tiny little box where if someone had not imagined and > preapproved a use of a feature it should not happen.   Let's please > avoid

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

Having this in Yama would also make it probable that there would be a security-centric default. It would end up wiping out unprivileged perf events access on distributions using Yama for ptrace_scope unless they make the explicit decision to disable it. Having the perf subsystem extend the

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

Having this in Yama would also make it probable that there would be a security-centric default. It would end up wiping out unprivileged perf events access on distributions using Yama for ptrace_scope unless they make the explicit decision to disable it. Having the perf subsystem extend the

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> The default has no impact on the "it's too coarse and limiting" > negative property  > of this patch, which is the show-stopper aspect. Please fix that > aspect instead of  > trying to argue around it. Disabling perf events in the kernel configuration is even more limiting, and is currently the

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> The default has no impact on the "it's too coarse and limiting" > negative property  > of this patch, which is the show-stopper aspect. Please fix that > aspect instead of  > trying to argue around it. Disabling perf events in the kernel configuration is even more limiting, and is currently the

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> > So the problem I have with this is that it will completely inhibit > > development of things like JITs that self-profile to re-compile > > frequently used code. > > Or reimplement strace with sys_perf_event_open(), speeding it up > greatly > by not using ptrace (see 'perf trace', one such

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

> > So the problem I have with this is that it will completely inhibit > > development of things like JITs that self-profile to re-compile > > frequently used code. > > Or reimplement strace with sys_perf_event_open(), speeding it up > greatly > by not using ptrace (see 'perf trace', one such

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Tue, 2016-08-02 at 11:52 +0200, Peter Zijlstra wrote: > On Wed, Jul 27, 2016 at 07:45:46AM -0700, Jeff Vander Stoep wrote: > > > > When kernel.perf_event_paranoid is set to 3 (or greater), disallow > > all access to performance events by users without CAP_SYS_ADMIN. > > > > This new level of

Re: [kernel-hardening] Re: [PATCH 1/2] security, perf: allow further restriction of perf_event_open

On Tue, 2016-08-02 at 11:52 +0200, Peter Zijlstra wrote: > On Wed, Jul 27, 2016 at 07:45:46AM -0700, Jeff Vander Stoep wrote: > > > > When kernel.perf_event_paranoid is set to 3 (or greater), disallow > > all access to performance events by users without CAP_SYS_ADMIN. > > > > This new level of

Re: [kernel-hardening] Re: [PATCH] [RFC] Introduce mmap randomization

> > It's very hard to quantify the benefits of fine-grained > > randomization, > > ?  N = # of possible addresses.  The bigger N is, the more chances the > attacker will trip up before finding what they were looking for. If the attacker is forcing the creation of many objects with a function

Re: [kernel-hardening] Re: [PATCH] [RFC] Introduce mmap randomization

> > It's very hard to quantify the benefits of fine-grained > > randomization, > > ?  N = # of possible addresses.  The bigger N is, the more chances the > attacker will trip up before finding what they were looking for. If the attacker is forcing the creation of many objects with a function

Re: [kernel-hardening] Re: [PATCH] [RFC] Introduce mmap randomization

> > In the Project Zero Stagefright post > > (http://googleprojectzero.blogspot.com/2015/09/stagefrightened.html) > > , > > we see that the linear allocation of memory combined with the low > > number of bits in the initial mmap offset resulted in a much more > > predictable layout which aided the

Re: [kernel-hardening] Re: [PATCH] [RFC] Introduce mmap randomization

> > In the Project Zero Stagefright post > > (http://googleprojectzero.blogspot.com/2015/09/stagefrightened.html) > > , > > we see that the linear allocation of memory combined with the low > > number of bits in the initial mmap offset resulted in a much more > > predictable layout which aided the

Re: [kernel-hardening] Re: [PATCH v2 02/11] mm: Hardened usercopy

> I'd like it to dump stack and be fatal to the process involved, but > yeah, I guess BUG() would work. Creating an infrastructure for > handling security-related Oopses can be done separately from this > (and > I'd like to see that added, since it's a nice bit of configurable > reactivity to

Re: [kernel-hardening] Re: [PATCH v2 02/11] mm: Hardened usercopy

> I'd like it to dump stack and be fatal to the process involved, but > yeah, I guess BUG() would work. Creating an infrastructure for > handling security-related Oopses can be done separately from this > (and > I'd like to see that added, since it's a nice bit of configurable > reactivity to

Re: [kernel-hardening] Re: [PATCH v2 02/11] mm: Hardened usercopy

> This could be a BUG, but I'd rather not panic the entire kernel. It seems unlikely that it will panic without panic_on_oops and that's an explicit opt-in to taking down the system on kernel logic errors exactly like this. In grsecurity, it calls the kernel exploit handling logic (panic if root,

Re: [kernel-hardening] Re: [PATCH v2 02/11] mm: Hardened usercopy

> This could be a BUG, but I'd rather not panic the entire kernel. It seems unlikely that it will panic without panic_on_oops and that's an explicit opt-in to taking down the system on kernel logic errors exactly like this. In grsecurity, it calls the kernel exploit handling logic (panic if root,

Re: [kernel-hardening] Re: [PATCH v2 2/3] Mark functions with the __nocapture attribute

On Tue, 2016-07-12 at 15:08 -0400, Kees Cook wrote: > On Mon, Jul 4, 2016 at 7:42 PM, Emese Revfy > wrote: > > > > The nocapture gcc attribute can be on functions only. > > The attribute takes one or more unsigned integer constants as > > parameters > > that specify the

Re: [kernel-hardening] Re: [PATCH v2 2/3] Mark functions with the __nocapture attribute

On Tue, 2016-07-12 at 15:08 -0400, Kees Cook wrote: > On Mon, Jul 4, 2016 at 7:42 PM, Emese Revfy > wrote: > > > > The nocapture gcc attribute can be on functions only. > > The attribute takes one or more unsigned integer constants as > > parameters > > that specify the function argument(s) of

Re: [kernel-hardening] [PATCH 2/2] security,perf: Allow further restriction of perf_event_open

On Fri, 2016-06-17 at 17:00 -0300, Arnaldo Carvalho de Melo wrote: > Em Fri, Jun 17, 2016 at 12:16:47PM -0400, Daniel Micay escreveu: > > On Fri, 2016-06-17 at 08:54 +0200, Peter Zijlstra wrote: > > > This Changelog is completely devoid of information. _WHY_ a

Re: [kernel-hardening] [PATCH 2/2] security,perf: Allow further restriction of perf_event_open

On Fri, 2016-06-17 at 17:00 -0300, Arnaldo Carvalho de Melo wrote: > Em Fri, Jun 17, 2016 at 12:16:47PM -0400, Daniel Micay escreveu: > > On Fri, 2016-06-17 at 08:54 +0200, Peter Zijlstra wrote: > > > This Changelog is completely devoid of information. _WHY_ a

Re: [kernel-hardening] [PATCH 2/2] security,perf: Allow further restriction of perf_event_open

On Fri, 2016-06-17 at 08:54 +0200, Peter Zijlstra wrote: > On Thu, Jun 16, 2016 at 03:27:55PM -0700, Kees Cook wrote: > > Hi guys, > > > > This patch wasn't originally CCed to you (I'm fixing that now). > > Would > > you consider taking this into the perf tree?  > > No. > > > It's been in

Re: [kernel-hardening] [PATCH 2/2] security,perf: Allow further restriction of perf_event_open

On Fri, 2016-06-17 at 08:54 +0200, Peter Zijlstra wrote: > On Thu, Jun 16, 2016 at 03:27:55PM -0700, Kees Cook wrote: > > Hi guys, > > > > This patch wasn't originally CCed to you (I'm fixing that now). > > Would > > you consider taking this into the perf tree?  > > No. > > > It's been in

Re: [kernel-hardening] Re: [PATCH 2/2] security,perf: Allow further restriction of perf_event_open

> As a debian user, is this a good place to complain? Because it does > get > it the way. It would be relevant to whether or not it should be set to 3 by default in the kernel without explicit configuration, but there's no proposal to do that. Debian has to pick a trade-off beyond security and a

Re: [kernel-hardening] Re: [PATCH 2/2] security,perf: Allow further restriction of perf_event_open

> As a debian user, is this a good place to complain? Because it does > get > it the way. It would be relevant to whether or not it should be set to 3 by default in the kernel without explicit configuration, but there's no proposal to do that. Debian has to pick a trade-off beyond security and a

Re: [kernel-hardening] Re: [PATCH v4 0/4] SROP Mitigation: Sigreturn Cookies

> Then there's an unanswered question: is this patch acceptable given > that it's an ABI break?  Security fixes are sometimes an exception to > the "no ABI breaks" rule, but it's by no means an automatic exception. > > --Andy It seems this could be worked around in general. Processes can have a

Re: [kernel-hardening] Re: [PATCH v4 0/4] SROP Mitigation: Sigreturn Cookies

> Then there's an unanswered question: is this patch acceptable given > that it's an ABI break?  Security fixes are sometimes an exception to > the "no ABI breaks" rule, but it's by no means an automatic exception. > > --Andy It seems this could be worked around in general. Processes can have a

Re: [kernel-hardening] Re: [PATCH 0/2] sysctl: allow CLONE_NEWUSER to be disabled

> This feature is already implemented by two distros, and likely wanted > by others. We cannot ignore that. Date point: Arch Linux won't be enabling CONFIG_USERNS until there's a way to disable unprivileged user namespaces. The kernel maintainers are unwilling to carry long-term out-of-tree

Re: [kernel-hardening] Re: [PATCH 0/2] sysctl: allow CLONE_NEWUSER to be disabled

> This feature is already implemented by two distros, and likely wanted > by others. We cannot ignore that. Date point: Arch Linux won't be enabling CONFIG_USERNS until there's a way to disable unprivileged user namespaces. The kernel maintainers are unwilling to carry long-term out-of-tree

Re: [kernel-hardening] Re: [RFC][PATCH 0/7] Sanitization of slabs based on grsecurity/PaX

> I am not sure what the point of this patchset is. We have a similar > effect > to sanitization already in the allocators through two mechanisms: The rationale was covered earlier. Are you responding to that or did you not see it? > 1. Slab poisoning > 2. Allocation with GFP_ZERO > > I do not

Re: [kernel-hardening] Re: [RFC][PATCH 0/7] Sanitization of slabs based on grsecurity/PaX

> I am not sure what the point of this patchset is. We have a similar > effect > to sanitization already in the allocators through two mechanisms: The rationale was covered earlier. Are you responding to that or did you not see it? > 1. Slab poisoning > 2. Allocation with GFP_ZERO > > I do not

Re: [PATCH v2 00/13] MADV_FREE support

On 05/12/15 06:10 AM, Pavel Machek wrote: > On Wed 2015-11-04 10:25:54, Minchan Kim wrote: >> MADV_FREE is on linux-next so long time. The reason was two, I think. >> >> 1. MADV_FREE code on reclaim path was really mess. > > Could you explain what MADV_FREE does? > > Comment in code says 'free

Re: [PATCH v2 00/13] MADV_FREE support

On 05/12/15 06:10 AM, Pavel Machek wrote: > On Wed 2015-11-04 10:25:54, Minchan Kim wrote: >> MADV_FREE is on linux-next so long time. The reason was two, I think. >> >> 1. MADV_FREE code on reclaim path was really mess. > > Could you explain what MADV_FREE does? > > Comment in code says 'free

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

On 13/11/15 02:03 AM, Minchan Kim wrote: > On Fri, Nov 13, 2015 at 01:45:52AM -0500, Daniel Micay wrote: >>> And now I am thinking if we use access bit, we could implment MADV_FREE_UNDO >>> easily when we need it. Maybe, that's what you want. Right? >> >> Ye

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

On 13/11/15 02:03 AM, Minchan Kim wrote: > On Fri, Nov 13, 2015 at 01:45:52AM -0500, Daniel Micay wrote: >>> And now I am thinking if we use access bit, we could implment MADV_FREE_UNDO >>> easily when we need it. Maybe, that's what you want. Right? >> >> Ye

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

> And now I am thinking if we use access bit, we could implment MADV_FREE_UNDO > easily when we need it. Maybe, that's what you want. Right? Yes, but why the access bit instead of the dirty bit for that? It could always be made more strict (i.e. access bit) in the future, while going the other

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

On 13/11/15 01:15 AM, Minchan Kim wrote: > On Thu, Nov 12, 2015 at 12:21:30AM -0500, Daniel Micay wrote: >>> I also think that the kernel should commit to either zeroing the page >>> or leaving it unchanged in response to MADV_FREE (even if the decision >>> of whi

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

> And now I am thinking if we use access bit, we could implment MADV_FREE_UNDO > easily when we need it. Maybe, that's what you want. Right? Yes, but why the access bit instead of the dirty bit for that? It could always be made more strict (i.e. access bit) in the future, while going the other

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

On 13/11/15 01:15 AM, Minchan Kim wrote: > On Thu, Nov 12, 2015 at 12:21:30AM -0500, Daniel Micay wrote: >>> I also think that the kernel should commit to either zeroing the page >>> or leaving it unchanged in response to MADV_FREE (even if the decision >>> of whi

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

> I also think that the kernel should commit to either zeroing the page > or leaving it unchanged in response to MADV_FREE (even if the decision > of which to do is made later on). I think that your patch series does > this, but only after a few of the patches are applied (the swap entry >

Re: [PATCH v3 01/17] mm: support madvise(MADV_FREE)

> I also think that the kernel should commit to either zeroing the page > or leaving it unchanged in response to MADV_FREE (even if the decision > of which to do is made later on). I think that your patch series does > this, but only after a few of the patches are applied (the swap entry >

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> active:clean active:dirty*, sigh. signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> I posted a patch doing exactly iovec madvise. Doesn't support MADV_FREE yet > though, but should be easy to do it. > > http://marc.info/?l=linux-mm=144615663522661=2 I think that would be a great way to deal with this. It keeps the nice property of still being able to drop pages in allocations

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> active:clean active:dirty*, sigh. signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> I posted a patch doing exactly iovec madvise. Doesn't support MADV_FREE yet > though, but should be easy to do it. > > http://marc.info/?l=linux-mm=144615663522661=2 I think that would be a great way to deal with this. It keeps the nice property of still being able to drop pages in allocations

Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

> From a user perspective, it doesn't depend on swap. It's just slower > without swap because it does what MADV_DONTNEED does. The current > implementation can be dropped in where MADV_DONTNEED was previously used. It just wouldn't replace existing layers of purging logic until that edge case is

Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

>>> It probably makes sense to stop thinking about them as anonymous pages >>> entirely at this point when it comes to aging. They're really not. The >>> LRU lists are split to differentiate access patterns and cost of page >>> stealing (and restoring). From that angle, MADV_FREE pages really have

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> With enough pages at once, though, munmap would be fine, too. That implies lots of page faults and zeroing though. The zeroing alone is a major performance issue. There are separate issues with munmap since it ends up resulting in a lot more virtual memory fragmentation. It would help if the

Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

> Even if we're wrong about the aging of those MADV_FREE pages, their > contents are invalidated; they can be discarded freely, and restoring > them is a mere GFP_ZERO allocation. All other anonymous pages have to > be written to disk, and potentially be read back. > > [ Arguably, MADV_FREE pages

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> That's comparable to Android's pinning / unpinning API for ashmem and I > think it makes sense if it's faster. It's different than the MADV_FREE > API though, because the new allocations that are handed out won't have > the usual lazy commit which MADV_FREE provides. Pages in an allocation >

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> Compared to MADV_DONTNEED, MADV_FREE's lazy memory free is a huge win to > reduce > page fault. But there is one issue remaining, the TLB flush. Both > MADV_DONTNEED > and MADV_FREE do TLB flush. TLB flush overhead is quite big in contemporary > multi-thread applications. In our production

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> Compared to MADV_DONTNEED, MADV_FREE's lazy memory free is a huge win to > reduce > page fault. But there is one issue remaining, the TLB flush. Both > MADV_DONTNEED > and MADV_FREE do TLB flush. TLB flush overhead is quite big in contemporary > multi-thread applications. In our production

Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

> Even if we're wrong about the aging of those MADV_FREE pages, their > contents are invalidated; they can be discarded freely, and restoring > them is a mere GFP_ZERO allocation. All other anonymous pages have to > be written to disk, and potentially be read back. > > [ Arguably, MADV_FREE pages

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> With enough pages at once, though, munmap would be fine, too. That implies lots of page faults and zeroing though. The zeroing alone is a major performance issue. There are separate issues with munmap since it ends up resulting in a lot more virtual memory fragmentation. It would help if the

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> That's comparable to Android's pinning / unpinning API for ashmem and I > think it makes sense if it's faster. It's different than the MADV_FREE > API though, because the new allocations that are handed out won't have > the usual lazy commit which MADV_FREE provides. Pages in an allocation >

Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

> From a user perspective, it doesn't depend on swap. It's just slower > without swap because it does what MADV_DONTNEED does. The current > implementation can be dropped in where MADV_DONTNEED was previously used. It just wouldn't replace existing layers of purging logic until that edge case is

Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

>>> It probably makes sense to stop thinking about them as anonymous pages >>> entirely at this point when it comes to aging. They're really not. The >>> LRU lists are split to differentiate access patterns and cost of page >>> stealing (and restoring). From that angle, MADV_FREE pages really have

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

On 04/11/15 12:53 AM, Daniel Micay wrote: >> In the common case it will be passed many pages by the allocator. There >> will still be a layer of purging logic on top of MADV_FREE but it can be >> much thinner than the current workarounds for MADV_DONTNEED. So the >&g

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> In the common case it will be passed many pages by the allocator. There > will still be a layer of purging logic on top of MADV_FREE but it can be > much thinner than the current workarounds for MADV_DONTNEED. So the > allocator would still be coalescing dirty ranges and only purging when > the

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> Does this set the write protect bit? > > What happens on architectures without hardware dirty tracking? It's supposed to avoid needing page faults when the data is accessed again, but it can just be implemented via page faults on architectures without a way to check for access or writes.

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> In the common case it will be passed many pages by the allocator. There > will still be a layer of purging logic on top of MADV_FREE but it can be > much thinner than the current workarounds for MADV_DONTNEED. So the > allocator would still be coalescing dirty ranges and only purging when > the

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

> Does this set the write protect bit? > > What happens on architectures without hardware dirty tracking? It's supposed to avoid needing page faults when the data is accessed again, but it can just be implemented via page faults on architectures without a way to check for access or writes.

Re: [PATCH v2 01/13] mm: support madvise(MADV_FREE)

On 04/11/15 12:53 AM, Daniel Micay wrote: >> In the common case it will be passed many pages by the allocator. There >> will still be a layer of purging logic on top of MADV_FREE but it can be >> much thinner than the current workarounds for MADV_DONTNEED. So the >&g

Re: [PATCH 0/8] MADV_FREE support

e of userland people who want to use >>the syscall. >> >> A few month ago, Daniel Micay(jemalloc active contributor) requested me >> to make progress upstreaming but I was busy at that time so it took >> so long time for me to revist the code and finally, I clean i

Re: [PATCH 0/8] MADV_FREE support

e of userland people who want to use >>the syscall. >> >> A few month ago, Daniel Micay(jemalloc active contributor) requested me >> to make progress upstreaming but I was busy at that time so it took >> so long time for me to revist the code and finally, I clean i

[PATCH v4] mm: add mremap flag for preserving the old mapping

d fragmentating memory over time is considered unacceptable. Signed-off-by: Daniel Micay --- include/linux/huge_mm.h | 2 +- include/linux/mm.h| 6 ++ include/uapi/linux/mman.h | 1 + mm/huge_memory.c | 4 ++-- mm/memory.c | 2 +- mm/mmap.c

Re: [PATCH v3] mm: add mremap flag for preserving the old mapping

On 30/09/14 01:49 PM, Andy Lutomirski wrote: > > I think it might pay to add an explicit vm_op to authorize > duplication, especially for non-cow mappings. IOW this kind of > extension seems quite magical for anything that doesn't have the > normal COW semantics, including for plain old

Re: [PATCH v3] mm: add mremap flag for preserving the old mapping

On 30/09/14 01:49 PM, Andy Lutomirski wrote: I think it might pay to add an explicit vm_op to authorize duplication, especially for non-cow mappings. IOW this kind of extension seems quite magical for anything that doesn't have the normal COW semantics, including for plain old read-only

[PATCH v4] mm: add mremap flag for preserving the old mapping

and fragmentating memory over time is considered unacceptable. Signed-off-by: Daniel Micay danielmi...@gmail.com --- include/linux/huge_mm.h | 2 +- include/linux/mm.h| 6 ++ include/uapi/linux/mman.h | 1 + mm/huge_memory.c | 4 ++-- mm/memory.c | 2 +- mm

Re: [PATCH v3] mm: add mremap flag for preserving the old mapping

On 30/09/14 01:49 PM, Andy Lutomirski wrote: > > I think it might pay to add an explicit vm_op to authorize > duplication, especially for non-cow mappings. IOW this kind of > extension seems quite magical for anything that doesn't have the > normal COW semantics, including for plain old

Re: [PATCH v3] mm: add mremap flag for preserving the old mapping

On 30/09/14 01:53 AM, Andy Lutomirski wrote: > On Mon, Sep 29, 2014 at 9:55 PM, Daniel Micay wrote: >> This introduces the MREMAP_RETAIN flag for preserving the source mapping >> when MREMAP_MAYMOVE moves the pages to a new destination. Accesses to >> the source location

Re: [PATCH v3] mm: add mremap flag for preserving the old mapping

On 30/09/14 01:53 AM, Andy Lutomirski wrote: On Mon, Sep 29, 2014 at 9:55 PM, Daniel Micay danielmi...@gmail.com wrote: This introduces the MREMAP_RETAIN flag for preserving the source mapping when MREMAP_MAYMOVE moves the pages to a new destination. Accesses to the source location will fault

<    1   2   3   4   >