[RFC v3 01/10] mm/damon: Use vm-independent address range concept

2020-06-09 Thread SeongJae Park
From: SeongJae Park DAMON's main idea is not limited to virtual address space. To prepare for further expansion of the support for other address spaces including physical memory, this commit modifies one of its core struct, 'struct damon_region' to use virtual memory independent address space

Re: Re: [PATCH v2] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
On Tue, 9 Jun 2020 14:56:07 +0200 Greg KH wrote: > On Tue, Jun 09, 2020 at 02:25:49PM +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit recommends the patches to replace 'blacklist' and > > 'whitelist' with the 'blocklist' and 'allowlist', beca

[PATCH v2] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit recommends the patches to replace 'blacklist' and 'whitelist' with the 'blocklist' and 'allowlist', because the new suggestions are incontrovertible, doesn't make people hurt, and more self-explanatory. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 2

Re: [PATCH] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
On Tue, 9 Jun 2020 14:18:43 +0200 SeongJae Park wrote: > From: SeongJae Park > > This commit recommends the patches to replace 'blacklist' and > 'whitelist' with the 'blocklist' and 'allowlist', because the new > suggestions are incontrovertible, doesn't make people hurt,

[PATCH] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit recommends the patches to replace 'blacklist' and 'whitelist' with the 'blocklist' and 'allowlist', because the new suggestions are incontrovertible, doesn't make people hurt, and more self-explanatory. Signed-off-by: SeongJae Park cr https://code.amazon.com

Re: Re: [RFC v11 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-06-09 Thread SeongJae Park
On Tue, 9 Jun 2020 10:47:45 +0200 David Hildenbrand wrote: > On 09.06.20 08:53, SeongJae Park wrote: > > From: SeongJae Park > > > > In many cases, users might use DAMON for simple data access aware > > memory management optimizations such as applying an operation sc

[RFC v11 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/guide.rst | 35 + Documentation/admin-guide/mm/damon/plans.rst | 26 +--- Documentation/admin-guide/mm/damon/usage.rst | 127

[RFC v11 6/8] mm/damon/selftests: Add 'schemes' debugfs tests

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v11 7/8] damon/tools: Support more human friendly 'schemes' control

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 128

[RFC v11 4/8] mm/damon/schemes: Implement a debugfs interface

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v11 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-06-09 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v11 5/8] mm/damon/schemes: Implement statistics feature

2020-06-09 Thread SeongJae Park
From: SeongJae Park To tune the DAMON-based operation schemes, knowing how many and how large regions are affected by each of the schemes will be helful. Those stats could be used for not only the tuning, but also monitoring of the working set size and the number of regions, if the scheme does

[RFC v11 0/8] Implement Data Access Monitoring-based Memory Operation Schemes

2020-06-09 Thread SeongJae Park
From: SeongJae Park DAMON[1] can be used as a primitive for data access awared memory management optimizations. For that, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v11 1/8] mm/madvise: Export do_madvise() to external GPL modules

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 1ad7522567d4

[RFC v11 2/8] mm/damon: Account age of target regions

2020-06-09 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

Re: Re: [PATCH v15 01/14] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-06-08 Thread SeongJae Park
So if no further comments are given, in the next spin, I will keep this separate, but will mention exactly what future commit needs this change, so that it can be easily squashed later, if necessary. Thanks, SeongJae Park

Re: Re: [PATCH v15 01/14] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-06-08 Thread SeongJae Park
On Mon, 8 Jun 2020 13:53:23 +0200 David Hildenbrand wrote: > On 08.06.20 13:40, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit exports 'lookup_page_ext()' to GPL modules. This will be > > used by DAMON. > > > > Signed-off-by: SeongJae

[PATCH v15 13/14] mm/damon: Add user space selftests

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v15 14/14] MAINTAINERS: Update for DAMON

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 50659d76976b..2396a9098715 100644 --- a/MAINTAINERS +++ b

[PATCH v15 12/14] mm/damon: Add kunit tests

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 635 mm/damon.c | 6 + 3 files changed, 652 insertions

[PATCH v15 10/14] tools: Add a minimal user-space tool for DAMON

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v15 09/14] mm/damon: Add tracepoints

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park Reviewed-by: Leonard

[PATCH v15 08/14] mm/damon: Add debugfs interface

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v15 07/14] mm/damon: Implement access pattern recording

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v15 06/14] mm/damon: Implement callbacks

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v15 05/14] mm/damon: Apply dynamic memory mapping changes

2020-06-08 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v15 04/14] mm/damon: Adaptively adjust regions

2020-06-08 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v15 03/14] mm/damon: Implement region based sampling

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v15 02/14] mm: Introduce Data Access MONitor (DAMON)

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v15 01/14] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-06-08 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index a3616f7a0e9e

[PATCH v15 00/14] Introduce Data Access MONitor (DAMON)

2020-06-08 Thread SeongJae Park
From: SeongJae Park Introduction DAMON is a data access monitoring framework subsystem for the Linux kernel. The core mechanisms of DAMON called 'region based sampling' and 'adaptive regions adjustment' (refer to 'mechanisms.rst' in the 11th patch of this patchset for the detail

[PATCH] docs/memory-barriers.txt/kokr: smp_mb__{before,after}_atomic(): update Documentation

2020-06-06 Thread SeongJae Park
Translate this commit to Korean: 39323c64b8a9 ("smp_mb__{before,after}_atomic(): update Documentation") Signed-off-by: SeongJae Park Reviewed-by: Yunjae Lee --- .../translations/ko_KR/memory-barriers.txt| 15 +-- 1 file changed, 9 insertions(+), 6 deletion

[PATCH 2/2] scripts/spelling: Add a few more typos

2020-06-05 Thread SeongJae Park
From: SeongJae Park This commit adds typos I found from another works. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 9 + 1 file changed, 9 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index d9cd24cf0d40..c45e9afaab2d 100644 --- a/scripts

[PATCH 0/2] Trivial fixups

2020-06-05 Thread SeongJae Park
From: SeongJae Park This patchset fixes a potential problem in idle page tracking and adds more typos in the scripts/spelling.txt. The problem and the typos were found during my DAMON[1] work. This patchset is based on next/akpm. [1] https://lore.kernel.org/linux-mm/20200602130125.20467-1-sjp

[PATCH 1/2] mm/page_idle.c: Skip offline pages

2020-06-05 Thread SeongJae Park
From: SeongJae Park 'Idle page tracking' users can pass random pfn that might be mapped to an offline page. To avoid accessing such pages, this commit modifies the 'page_idle_get_page()' to use 'pfn_to_online_page()' instead of 'pfn_valid()' and 'pfn_to_page()' combination, so that the pfn

Re: Re: [RFC v2 7/9] mm/damon: Implement callbacks for physical memory monitoring

2020-06-04 Thread SeongJae Park
On Thu, 4 Jun 2020 17:39:49 +0200 David Hildenbrand wrote: > On 04.06.20 17:23, SeongJae Park wrote: > > On Thu, 4 Jun 2020 16:58:13 +0200 David Hildenbrand > > wrote: > > > >> On 04.06.20 09:26, SeongJae Park wrote: > >>> On Wed, 3 Jun 2020 18:0

Re: Re: [RFC v2 7/9] mm/damon: Implement callbacks for physical memory monitoring

2020-06-04 Thread SeongJae Park
On Thu, 4 Jun 2020 16:58:13 +0200 David Hildenbrand wrote: > On 04.06.20 09:26, SeongJae Park wrote: > > On Wed, 3 Jun 2020 18:09:21 +0200 David Hildenbrand > > wrote: > > > >> On 03.06.20 16:11, SeongJae Park wrote: > >>> From: SeongJae Park &

Re: [RFC v9 4/8] mm/damon/schemes: Implement a debugfs interface

2020-06-04 Thread SeongJae Park
On Tue, 26 May 2020 09:56:58 +0200 SeongJae Park wrote: > From: SeongJae Park > > This commit implements a debugfs interface for the data access > monitoring oriented memory management schemes. It is supposed to be > used by administrators and/or privileged user space progr

Re: Re: [RFC v2 7/9] mm/damon: Implement callbacks for physical memory monitoring

2020-06-04 Thread SeongJae Park
On Wed, 3 Jun 2020 18:09:21 +0200 David Hildenbrand wrote: > On 03.06.20 16:11, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit implements the four callbacks (->init_target_regions, > > ->update_target_regions, ->prepare_access_check, and

[RFC v2 8/9] mm/damon/debugfs: Support physical memory monitoring

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit makes the debugfs interface to support the physical memory monitoring, in addition to the virtual memory monitoring. Users can do the physical memory monitoring by writing a special keyword, 'paddr\n' to the 'pids' debugfs file. Then, DAMON will check

[RFC v2 9/9] Docs/damon: Document physical memory monitoring support

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit adds description for the physical memory monitoring usage in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 42 ++-- 1 file changed, 29 insertions(+), 13 deletions(-) diff --git

[RFC v2 7/9] mm/damon: Implement callbacks for physical memory monitoring

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit implements the four callbacks (->init_target_regions, ->update_target_regions, ->prepare_access_check, and ->check_accesses) for the basic access monitoring of the physical memory address space. By setting the callback pointers to point those, user

[RFC v2 6/9] mm/damon: Make access check primitive configurable

2020-06-03 Thread SeongJae Park
From: SeongJae Park DAMON assumes the target region is in virtual address space and therefore uses PTE Accessed bit checking for access checking. However, as some CPU provides H/W based memory access check features that usually more accurate and light-weight than PTE Accessed bit checking, some

[RFC v2 5/9] Docs/damon: Document 'initial_regions' feature

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit documents the 'initial_regions' feature. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 34 1 file changed, 34 insertions(+) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin

[RFC v2 3/9] mm/damon: Make monitoring target regions init/update configurable

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit allows DAMON users to configure their own monitoring target regions initializer / updater. Using this, users can confine the monitoring address spaces as they want. For example, users can track only stack, heap, shared memory area, or specific file-backed area

[RFC v2 2/9] mm/damon: Clean up code using 'struct damon_addr_range'

2020-06-03 Thread SeongJae Park
From: SeongJae Park There are unnecessarily duplicated code in DAMON, that can be eliminated by using the new struct, 'damon_addr_range'. This commit cleans up the DAMON code in the way. Signed-off-by: SeongJae Park --- mm/damon-test.h | 36 ++-- mm/damon.c

[RFC v2 4/9] mm/damon/debugfs: Allow users to set initial monitoring target regions

2020-06-03 Thread SeongJae Park
From: SeongJae Park Some users would want to monitor only a part of the entire virtual memory address space. The '->init_target_regions' callback is therefore provided, but only programming interface can use it. For the reason, this commit introduces a new debugfs file, 'init_region'. Us

[RFC v2 0/9] DAMON: Support Access Monitoring of Any Address Space Including Physical Memory

2020-06-03 Thread SeongJae Park
From: SeongJae Park Currently, DAMON[1] supports only virtual memory address spaces because it utilizes PTE Accessed bits as its low-level access check primitive and ``struct vma`` as a way to address the monitoring target regions. However, the core idea of DAMON, which makes it able to provide

[RFC v2 1/9] mm/damon: Use vm-independent address range concept

2020-06-03 Thread SeongJae Park
From: SeongJae Park DAMON's main idea is not limited to virtual address space. To prepare for further expansion of the support for other address spaces including physical memory, this commit modifies one of its core struct, 'struct damon_region' to use virtual memory independent address space

Re: Re: [PATCH v14 13/15] mm/damon-test: Add a kunit test for recording setup

2020-06-03 Thread SeongJae Park
On Tue, 2 Jun 2020 13:28:03 -0700 Brendan Higgins wrote: > On Tue, Jun 2, 2020 at 6:16 AM SeongJae Park wrote: > > > > From: SeongJae Park > > > > This commit adds another unit test case for the recording setup. > > > > Signed-off-by: SeongJae Park >

[RFC v10 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/guide.rst | 35 + Documentation/admin-guide/mm/damon/usage.rst | 127 +-- 2 files changed, 152 insertions

[RFC v10 7/8] damon/tools: Support more human friendly 'schemes' control

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 128

[RFC v10 6/8] mm/damon/selftests: Add 'schemes' debugfs tests

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v10 5/8] mm/damon/schemes: Implement statistics feature

2020-06-03 Thread SeongJae Park
From: SeongJae Park To tune the DAMON-based operation schemes, knowing how many and how large regions are affected by each of the schemes will be helful. Those stats could be used for not only the tuning, but also monitoring of the working set size and the number of regions, if the scheme does

[RFC v10 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-06-03 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v10 4/8] mm/damon/schemes: Implement a debugfs interface

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v10 2/8] mm/damon: Account age of target regions

2020-06-03 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v10 1/8] mm/madvise: Export do_madvise() to external GPL modules

2020-06-03 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 1ad7522567d4

[RFC v10 0/8] Implement Data Access Monitoring-based Memory Operation Schemes

2020-06-03 Thread SeongJae Park
From: SeongJae Park DAMON[1] can be used as a primitive for data access awared memory management optimizations. For that, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

Re: Re: [PATCH v14 09/15] mm/damon: Add tracepoints

2020-06-02 Thread SeongJae Park
On Tue, 2 Jun 2020 11:35:36 -0400 Steven Rostedt wrote: > On Tue, 2 Jun 2020 15:12:49 +0200 > SeongJae Park wrote: > > > From: SeongJae Park > > > > This commit adds a tracepoint for DAMON. It traces the monitoring > > results of each region for each aggrega

[PATCH v14 15/15] MAINTAINERS: Update for DAMON

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 50659d76976b..2396a9098715 100644 --- a/MAINTAINERS +++ b

[PATCH v14 14/15] mm/damon: Add user space selftests

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v14 12/15] mm/damon: Add kunit tests

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v14 13/15] mm/damon-test: Add a kunit test for recording setup

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit adds another unit test case for the recording setup. Signed-off-by: SeongJae Park --- mm/damon-test.h | 13 + 1 file changed, 13 insertions(+) diff --git a/mm/damon-test.h b/mm/damon-test.h index cf715529ff64..5b18619efe72 100644 --- a/mm/damon

[PATCH v14 10/15] tools: Add a minimal user-space tool for DAMON

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v14 09/15] mm/damon: Add tracepoints

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park Reviewed-by: Leonard

[PATCH v14 08/15] mm/damon: Add debugfs interface

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v14 07/15] mm/damon: Implement access pattern recording

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v14 05/15] mm/damon: Apply dynamic memory mapping changes

2020-06-02 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v14 06/15] mm/damon: Implement callbacks

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v14 04/15] mm/damon: Adaptively adjust regions

2020-06-02 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v14 03/15] mm/damon: Implement region based sampling

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v14 02/15] mm: Introduce Data Access MONitor (DAMON)

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v14 01/15] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-06-02 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index a3616f7a0e9e

[PATCH v14 00/15] Introduce Data Access MONitor (DAMON)

2020-06-02 Thread SeongJae Park
From: SeongJae Park Introduction DAMON is a data access monitoring framework subsystem for the Linux kernel. The core mechanisms of DAMON called 'region based sampling' and adaptive regions adjustment' (refer to :doc:`mechanisms` for the detail) make it - accurate (The monitored

Re: Re: [PATCH v13 05/15] mm/damon: Adaptively adjust regions

2020-05-27 Thread SeongJae Park
On Wed, 27 May 2020 13:23:56 +0200 Leonard Foerster wrote: > On 2020-05-25T11:15:02+02:00 SeongJae Park wrote: > > > From: SeongJae Park > > > > At the beginning of the monitoring, DAMON constructs the initial regions > > by evenly splitting the memory mappe

Re: Re: [PATCH v13 03/15] mm: Introduce Data Access MONitor (DAMON)

2020-05-27 Thread SeongJae Park
On Wed, 27 May 2020 13:23:31 +0200 Leonard Foerster wrote: > On 2020-05-25T11:15:00+02:00 SeongJae Park wrote: > > > From: SeongJae Park > > > > This commit introduces a kernel module named DAMON. Note that this > > commit is implementing only the stub for

[RFC v9 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes

2020-05-26 Thread SeongJae Park
From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/guide.rst | 35 ++ Documentation/admin-guide/mm/damon/usage.rst | 126 +-- 2 files changed, 151 insertions

[RFC v9 7/8] damon/tools: Support more human friendly 'schemes' control

2020-05-26 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 128

[RFC v9 6/8] mm/damon/selftests: Add 'schemes' debugfs tests

2020-05-26 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v9 5/8] mm/damon/schemes: Implement statistics feature

2020-05-26 Thread SeongJae Park
From: SeongJae Park To tune the DAMON-based operation schemes, knowing how many and how large regions are affected by each of the schemes will be helful. Those stats could be used for not only the tuning, but also monitoring of the working set size and the number of regions, if the scheme does

[RFC v9 4/8] mm/damon/schemes: Implement a debugfs interface

2020-05-26 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v9 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-05-26 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v9 2/8] mm/damon: Account age of target regions

2020-05-26 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v9 1/8] mm/madvise: Export do_madvise() to external GPL modules

2020-05-26 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 80f8a1839f70

[RFC v9 0/8] Implement Data Access Monitoring-based Memory Operation Schemes

2020-05-26 Thread SeongJae Park
From: SeongJae Park DAMON[1] can be used as a primitive for data access awared memory management optimizations. For that, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[PATCH v13 15/15] MAINTAINERS: Update for DAMON

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5a5332b3591d..586513e92b65 100644 --- a/MAINTAINERS +++ b

[PATCH v13 14/15] mm/damon: Add user space selftests

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v13 13/15] mm/damon: Add kunit tests

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v13 11/15] tools: Add a minimal user-space tool for DAMON

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v13 10/15] mm/damon: Add tracepoints

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park --- include/trace/events

[PATCH v13 09/15] mm/damon: Add debugfs interface

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v13 08/15] mm/damon: Implement access pattern recording

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v13 07/15] mm/damon: Implement callbacks

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v13 06/15] mm/damon: Apply dynamic memory mapping changes

2020-05-25 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v13 05/15] mm/damon: Adaptively adjust regions

2020-05-25 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v13 04/15] mm/damon: Implement region based sampling

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v13 02/15] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index 4ade843ff588..71169b45bba9 100644 --- a/mm/page_ext.c

[PATCH v13 03/15] mm: Introduce Data Access MONitor (DAMON)

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v13 00/15] Introduce Data Access MONitor (DAMON)

2020-05-25 Thread SeongJae Park
From: SeongJae Park Introduction DAMON is a data access monitoring framework subsystem for the Linux kernel. The core mechanisms of DAMON called 'region based sampling' and adaptive regions adjustment' (refer to :doc:`mechanisms` for the detail) make it accurate, efficient

<    2   3   4   5   6   7   8   9   10   11   >