Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-15 Thread Joel Fernandes
On Mon, Oct 15, 2018 at 10:18:14AM +0200, Martin Schwidefsky wrote: > On Mon, 15 Oct 2018 09:10:53 +0200 > Christian Borntraeger wrote: > > > On 10/12/2018 03:37 AM, Joel Fernandes (Google) wrote: > > > Android needs to mremap large regions of memory during memory management > > > related

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-15 Thread Martin Schwidefsky
On Mon, 15 Oct 2018 09:10:53 +0200 Christian Borntraeger wrote: > On 10/12/2018 03:37 AM, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > > not enabled. The

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-15 Thread Christian Borntraeger
On 10/12/2018 03:37 AM, Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each > pte at a time,

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-13 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 07:25:08PM -0700, Daniel Colascione wrote: [...] > > But anyway, I think this runtime detection thing is not needed. THP is > > actually expected to be as fast as this anyway, so if that's available then > > we should already be as fast. > > Ah, I think the commit message

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-13 Thread Anton Ivanov
On 12/10/2018 22:40, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: On 10/12/18 3:48 PM, Anton Ivanov wrote: On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Daniel Colascione
On Fri, Oct 12, 2018 at 7:10 PM, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 06:54:33PM -0700, Daniel Colascione wrote: >> I wonder whether it makes sense to expose to userspace somehow whether >> mremap is "fast" for a particular architecture. If a feature relies on >> fast mremap, it might

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Daniel Colascione
I wonder whether it makes sense to expose to userspace somehow whether mremap is "fast" for a particular architecture. If a feature relies on fast mremap, it might be better for some userland component to disable that feature entirely rather than blindly use mremap and end up performing very

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Daniel Colascione
Not 32-bit ARM? On Fri, Oct 12, 2018 at 6:35 PM, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 11:18:36AM -0700, David Miller wrote: >> From: Joel Fernandes > [...] >> > Also, do we not flush the caches from any path when we munmap >> > address space? We do call do_munmap on the old mapping

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 06:54:33PM -0700, Daniel Colascione wrote: > I wonder whether it makes sense to expose to userspace somehow whether > mremap is "fast" for a particular architecture. If a feature relies on > fast mremap, it might be better for some userland component to disable > that

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 06:39:45PM -0700, Daniel Colascione wrote: > Not 32-bit ARM? Well, I didn't want to enable every possible architecture we could in a single go. Certainly arm32 can be a follow on enablement as can be other architectures. The point of this series is to upstream this feature

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 11:18:36AM -0700, David Miller wrote: > From: Joel Fernandes [...] > > Also, do we not flush the caches from any path when we munmap > > address space? We do call do_munmap on the old mapping from mremap > > after moving to the new one. > > Sparc makes sure that shared

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: > > On 10/12/18 3:48 PM, Anton Ivanov wrote: > > On 12/10/2018 15:37, Kirill A. Shutemov wrote: > > > On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: > > > > On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: > > > > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 09:57:19AM -0700, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 04:19:46PM +0300, Kirill A. Shutemov wrote: > > On Fri, Oct 12, 2018 at 05:50:46AM -0700, Joel Fernandes wrote: > > > On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > > > > On Thu, Oct

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really slow if THP is not enabled. The bottleneck is move_page_tables, which is copying each pte at a time, and can be

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 10/12/18 3:48 PM, Anton Ivanov wrote: On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: Android needs to mremap large regions of memory during memory management related

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 10/12/18 5:50 PM, Joel Fernandes wrote: On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: On 10/12/18 3:48 PM, Anton Ivanov wrote: On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernandes

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 05:58:40PM +0100, Anton Ivanov wrote: [...] > > > > > > If I read the code right, MIPS completely ignores the address > > > > > > argument so > > > > > > set_pmd_at there may not have the effect which this patch is trying > > > > > > to > > > > > > achieve. > > > > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 04:19:46PM +0300, Kirill A. Shutemov wrote: > On Fri, Oct 12, 2018 at 05:50:46AM -0700, Joel Fernandes wrote: > > On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > > > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: > > On 10/12/18 3:48 PM, Anton Ivanov wrote: > > On 12/10/2018 15:37, Kirill A. Shutemov wrote: > > > On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: > > > > On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: > > > > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: > On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > > not enabled. The

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 05:50:46AM -0700, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > > Android needs to mremap large regions of memory during memory management > > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread David Miller
From: Joel Fernandes Date: Fri, 12 Oct 2018 05:50:46 -0700 > If its an issue, then how do transparent huge pages work on Sparc? I don't > see the huge page code (move_huge_pages) during mremap doing anything special > for Sparc architecture when moving PMDs.. This is because all huge pages are

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread David Miller
From: "Kirill A. Shutemov" Date: Fri, 12 Oct 2018 14:30:56 +0300 > I looked into the code more and noticed move_pte() helper called from > move_ptes(). It changes PTE entry to suite new address. > > It is only defined in non-trivial way on Sparc. I don't know much about > Sparc and it's hard

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > @@ -239,7 +287,21 @@ unsigned long move_page_tables(struct vm_area_struct > > *vma, > > split_huge_pmd(vma, old_pmd, old_addr); >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each >

[PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-11 Thread Joel Fernandes (Google)
Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really slow if THP is not enabled. The bottleneck is move_page_tables, which is copying each pte at a time, and can be really slow across a large map. Turning on THP may not