[PATCH v11 09/16] mm/damon: Implement access pattern recording

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v11 10/16] mm/damon: Add debugfs interface

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v11 11/16] mm/damon: Add tracepoints

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park --- include/trace/events

[PATCH v11 12/16] tools: Add a minimal user-space tool for DAMON

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v11 13/16] Documentation/admin-guide/mm: Add a document for DAMON

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a simple document for DAMON under `Documentation/admin-guide/mm`. Signed-off-by: SeongJae Park --- .../admin-guide/mm/data_access_monitor.rst| 428 ++ Documentation/admin-guide/mm/index.rst| 1 + 2 files changed, 429

[PATCH v11 14/16] mm/damon: Add kunit tests

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v11 15/16] mm/damon: Add user space selftests

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v11 16/16] MAINTAINERS: Update for DAMON

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5a5332b3591d..cb4a7fa3cdfe 100644 --- a/MAINTAINERS +++ b

Re: Re: Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-06 Thread SeongJae Park
aster/TEST_DIR/OS_Eval.c#L820 [3] https://github.com/LinuxPerfStudy/LEBench/blob/master/TEST_DIR/OS_Eval.c#L822 Thanks, SeongJae Park

Re: Re: Re: Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-06 Thread SeongJae Park
On Wed, 6 May 2020 07:41:51 -0700 "Paul E. McKenney" wrote: > On Wed, May 06, 2020 at 02:59:26PM +0200, SeongJae Park wrote: > > TL; DR: It was not kernel's fault, but the benchmark program. > > > > So, the problem is reproducible using the lebench[1] only. I car

Re: [PATCH v10 06/16] mm/damon: Split regions into 4 subregions if possible

2020-05-07 Thread SeongJae Park
On Tue, 5 May 2020 13:08:05 +0200 SeongJae Park wrote: > From: SeongJae Park > > Suppose that there are a very wide and cold region and a hot region, and > both regions are identified by DAMON. And then, the middle small region > inside the very wide and cold region beco

[RFC v8 0/8] Implement Data Access Monitoring-based Memory Operation Schemes

2020-05-12 Thread SeongJae Park
From: SeongJae Park DAMON[1] can be used as a primitive for data access awared memory management optimizations. That said, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v8 1/8] mm/madvise: Export do_madvise() to external GPL modules

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 80f8a1839f70

[RFC v8 2/8] mm/damon: Account age of target regions

2020-05-12 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v8 4/8] mm/damon/schemes: Implement a debugfs interface

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v8 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-05-12 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v8 6/8] mm/damon/selftests: Add 'schemes' debugfs tests

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v8 5/8] mm/damon-test: Add kunit test case for regions age accounting

2020-05-12 Thread SeongJae Park
From: SeongJae Park After merges of regions, each region should know their last shape in proper way to measure the changes from the last modification and reset the age if the changes are significant. This commit adds kunit test cases checking whether the regions are knowing their last shape

[RFC v8 7/8] damon/tools: Support more human friendly 'schemes' control

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 127

[RFC v8 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- .../admin-guide/mm/data_access_monitor.rst| 100 +- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/mm

[PATCH v12 01/16] scripts/spelling: Add a few more typos

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds typos found from DAMON patchset. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 8 1 file changed, 8 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index ffa838f3a2b5..a8214d8e8e4b 100644 --- a/scripts

[PATCH v12 00/16] Introduce Data Access MONitor (DAMON)

2020-05-18 Thread SeongJae Park
From: SeongJae Park Introduction DAMON is a data access monitoring framework subsystem for the Linux kernel. The core mechanisms of DAMON called 'region based sampling' and adaptive regions adjustment' (refer to :doc:`mechanisms` for the detail) make it accurate, efficient

[PATCH v12 02/16] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index 4ade843ff588..71169b45bba9 100644 --- a/mm/page_ext.c

[PATCH v12 03/16] mm: Introduce Data Access MONitor (DAMON)

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v12 05/16] mm/damon: Adaptively adjust regions

2020-05-18 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v12 04/16] mm/damon: Implement region based sampling

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v12 06/16] mm/damon: Split regions into 3 subregions if necessary

2020-05-18 Thread SeongJae Park
From: SeongJae Park Suppose that there are a very wide and cold region and a hot region, and both regions are identified by DAMON. And then, the middle small region inside the very wide and cold region becomes hot. DAMON will not be able to identify this new region because the adaptive regions

[PATCH v12 07/16] mm/damon: Apply dynamic memory mapping changes

2020-05-18 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v12 08/16] mm/damon: Implement callbacks

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v12 09/16] mm/damon: Implement access pattern recording

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v12 10/16] mm/damon: Add debugfs interface

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v12 11/16] mm/damon: Add tracepoints

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park --- include/trace/events

[PATCH v12 12/16] tools: Add a minimal user-space tool for DAMON

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v12 14/16] mm/damon: Add kunit tests

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v12 15/16] mm/damon: Add user space selftests

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v12 16/16] MAINTAINERS: Update for DAMON

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5a5332b3591d..586513e92b65 100644 --- a/MAINTAINERS +++ b

Upstream fixes not merged in 5.4.y

2020-06-29 Thread SeongJae Park
d35d3660e065 ("binder: fix null deref of proc->context") The script found several more commits but I exclude those here, because those seems not applicable on 5.4.y or fixing trivial problems only. If I'm not following a proper process for this kind of reports, please let me know. Thanks, SeongJae Park

Re: Re: Upstream fixes not merged in 5.4.y

2020-06-30 Thread SeongJae Park
On Mon, 29 Jun 2020 18:15:42 +0200 Greg KH wrote: > On Mon, Jun 29, 2020 at 04:28:05PM +0200, SeongJae Park wrote: > > Hello, > > > > > > With my little script, I found below commits in the mainline tree are more > > than > > 1 week old and fixing co

Re: Re: [RFC v11 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-06-09 Thread SeongJae Park
On Tue, 9 Jun 2020 10:47:45 +0200 David Hildenbrand wrote: > On 09.06.20 08:53, SeongJae Park wrote: > > From: SeongJae Park > > > > In many cases, users might use DAMON for simple data access aware > > memory management optimizations such as applying an operation sc

[PATCH] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit recommends the patches to replace 'blacklist' and 'whitelist' with the 'blocklist' and 'allowlist', because the new suggestions are incontrovertible, doesn't make people hurt, and more self-explanatory. Signed-off-by: SeongJae Park cr https://code.amazon.com

Re: [PATCH] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
On Tue, 9 Jun 2020 14:18:43 +0200 SeongJae Park wrote: > From: SeongJae Park > > This commit recommends the patches to replace 'blacklist' and > 'whitelist' with the 'blocklist' and 'allowlist', because the new > suggestions are incontrovertible, doesn't make people hurt,

[PATCH v2] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit recommends the patches to replace 'blacklist' and 'whitelist' with the 'blocklist' and 'allowlist', because the new suggestions are incontrovertible, doesn't make people hurt, and more self-explanatory. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 2

Re: Re: [PATCH v2] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
On Tue, 9 Jun 2020 14:56:07 +0200 Greg KH wrote: > On Tue, Jun 09, 2020 at 02:25:49PM +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit recommends the patches to replace 'blacklist' and > > 'whitelist' with the 'blocklist' and 'allowlist', beca

[RFC v3 01/10] mm/damon: Use vm-independent address range concept

2020-06-09 Thread SeongJae Park
From: SeongJae Park DAMON's main idea is not limited to virtual address space. To prepare for further expansion of the support for other address spaces including physical memory, this commit modifies one of its core struct, 'struct damon_region' to use virtual memory independent address space

[RFC v3 00/10] DAMON: Support Access Monitoring of Any Address Space Including Physical Memory

2020-06-09 Thread SeongJae Park
From: SeongJae Park Currently, DAMON[1] supports only virtual memory address spaces because it utilizes PTE Accessed bits as its low-level access check primitive and ``struct vma`` as a way to address the monitoring target regions. However, the core idea of DAMON, which makes it able to provide

[RFC v3 02/10] mm/damon: Make monitoring target regions init/update configurable

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit allows DAMON users to configure their own monitoring target regions initializer / updater. Using this, users can confine the monitoring address spaces as they want. For example, users can track only stack, heap, shared memory area, or specific file-backed area

[RFC v3 04/10] tools/damon: Implement init target regions feature

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit updates the damon user space tool to support the initial monitoring target regions feature. Signed-off-by: SeongJae Park --- tools/damon/_damon.py | 39 +++ tools/damon/record.py | 12 +++- tools/damon/schemes.py

[RFC v3 03/10] mm/damon/debugfs: Allow users to set initial monitoring target regions

2020-06-09 Thread SeongJae Park
From: SeongJae Park Some users would want to monitor only a part of the entire virtual memory address space. The '->init_target_regions' callback is therefore provided, but only programming interface can use it. For the reason, this commit introduces a new debugfs file, 'init_region'. Us

[RFC v3 05/10] Docs/damon: Document 'initial_regions' feature

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit documents the 'initial_regions' feature. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 34 1 file changed, 34 insertions(+) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin

[RFC v3 06/10] mm/damon: Make access check primitive configurable

2020-06-09 Thread SeongJae Park
From: SeongJae Park DAMON assumes the target region is in virtual address space and therefore uses PTE Accessed bit checking for access checking. However, as some CPU provides H/W based memory access check features that usually more accurate and light-weight than PTE Accessed bit checking, some

[RFC v3 07/10] mm/damon: Implement callbacks for physical memory monitoring

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit implements the four callbacks (->init_target_regions, ->update_target_regions, ->prepare_access_check, and ->check_accesses) for the basic access monitoring of the physical memory address space. By setting the callback pointers to point those, user

[RFC v3 08/10] mm/damon/debugfs: Support physical memory monitoring

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit makes the debugfs interface to support the physical memory monitoring, in addition to the virtual memory monitoring. Users can do the physical memory monitoring by writing a special keyword, 'paddr\n' to the 'pids' debugfs file. Then, DAMON will check

[RFC v3 10/10] Docs/damon: Document physical memory monitoring support

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit adds description for the physical memory monitoring usage in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/faq.rst| 16 +++-- Documentation/admin-guide/mm/damon/index.rst | 1 - .../admin-guide/mm/damon

[RFC v3 09/10] tools/damon/record: Support physical memory address spce

2020-06-09 Thread SeongJae Park
From: SeongJae Park This commit allows users to record the data accesses on physical memory address space by passing 'paddr' as target to 'damo-record'. If the init regions are given, the regions will be monitored. Else, it will monitor biggest conitguous 'System RAM' region in '/proc/iomem

Re: Re: [PATCH v2] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-09 Thread SeongJae Park
On Tue, 09 Jun 2020 18:35:46 -0700 Joe Perches wrote: > On Tue, 2020-06-09 at 14:25 +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit recommends the patches to replace 'blacklist' and > > 'whitelist' with the 'blocklist' and 'allowlist', beca

[PATCH v3 1/2] checkpatch: support deprecated terms checking

2020-06-10 Thread SeongJae Park
From: SeongJae Park Some terms could be deprecated for various reasons, but it is hard to update the entire old usages. That said, we could at least encourage new patches to use the suggested replacements. This commit adds check of deprecated terms in the 'checkpatch.pl' for that. The script

[PATCH v3 0/2] Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-10 Thread SeongJae Park
From: SeongJae Park This patchset 1) adds support of deprecated terms in the 'checkpatch.pl' and 2) set the 'blacklist' and 'whitelist' as deprecated with replacement suggestion of 'blocklist' and 'allowlist', because the suggestions are incontrovertible, doesn't make people hurt, and more self

[PATCH v3 2/2] scripts/deprecated_terms: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-10 Thread SeongJae Park
From: SeongJae Park This commit recommends the patches to replace 'blacklist' and 'whitelist' with the 'blocklist' and 'allowlist', because the new suggestions are incontrovertible, doesn't make people hurt, and more self-explanatory. Signed-off-by: SeongJae Park --- scripts

Re: Re: [PATCH v3 1/2] checkpatch: support deprecated terms checking

2020-06-10 Thread SeongJae Park
On Wed, 10 Jun 2020 00:13:42 -0700 Joe Perches wrote: > On Wed, 2020-06-10 at 08:52 +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > Some terms could be deprecated for various reasons, but it is hard to > > update the entire old usages. That said,

Re: Re: Re: [PATCH v3 1/2] checkpatch: support deprecated terms checking

2020-06-10 Thread SeongJae Park
On Wed, 10 Jun 2020 01:45:41 -0700 Joe Perches wrote: > On Wed, 2020-06-10 at 10:01 +0200, SeongJae Park wrote: > > On Wed, 10 Jun 2020 00:13:42 -0700 Joe Perches wrote: > [] > > > This is a direct copy of the spelling dictionary > > > loading code, so may

Re: Re: [PATCH v2] scripts/spelling: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-10 Thread SeongJae Park
On Wed, 10 Jun 2020 10:50:24 +0200 Jiri Slaby wrote: > On 09. 06. 20, 14:25, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit recommends the patches to replace 'blacklist' and > > 'whitelist' with the 'blocklist' and 'allowlist', beca

Re: [PATCH v15 04/14] mm/damon: Adaptively adjust regions

2020-06-10 Thread SeongJae Park
On Mon, 8 Jun 2020 13:40:37 +0200 SeongJae Park wrote: > From: SeongJae Park > > At the beginning of the monitoring, DAMON constructs the initial regions > by evenly splitting the memory mapped address space of the process into > the user-specified minimal number of regions.

Re: [PATCH v3 2/2] scripts/deprecated_terms: Recommend blocklist/allowlist instead of blacklist/whitelist

2020-06-10 Thread SeongJae Park
/ == >8 = >From 1376e327de8316ef30c393507b29d70d38bffd05 Mon Sep 17 00:00:00 2001 From: SeongJae Park Date: Wed, 10 Jun 2020 07:23:33 +0200 Subject: [PATCH v3.1] scripts/deprecated_terms: Recommend denylist/allowlist i

[PATCH v18 00/14] Introduce Data Access MONitor (DAMON)

2020-07-13 Thread SeongJae Park
From: SeongJae Park Changes from Previous Version = - Reorganize the doc and remove png blobs (Mike Rapoport) - Wordsmith mechnisms doc and commit messages - tools/wss: Set default working set access frequency threshold - Avoid race in damon deamon start

[PATCH v18 02/14] mm: Introduce Data Access MONitor (DAMON)

2020-07-13 Thread SeongJae Park
From: SeongJae Park DAMON is a data access monitoring framework subsystem for the Linux kernel. The core mechanisms of DAMON make it - accurate (the monitoring output is useful enough for DRAM level memory management; It might not appropriate for CPU Cache levels, though), - light

[PATCH v18 04/14] mm/damon: Adaptively adjust regions

2020-07-13 Thread SeongJae Park
From: SeongJae Park Even somehow the initial monitoring target regions are well constructed to fulfill the assumption (pages in same region have similar access frequencies), the data access pattern can be dynamically changed. This will result in low monitoring quality. To keep the assumption

[PATCH v18 03/14] mm/damon: Implement region based sampling

2020-07-13 Thread SeongJae Park
From: SeongJae Park DAMON separates its monitoring target address space independent high level logics from the target space dependent low level primitives for flexible support of various address spaces. This commit implements DAMON's target address space independent high level logics for basic

[PATCH v18 01/14] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. It will be used by DAMON in following commit for the implementation of the region based sampling. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster Reviewed-by: Varad Gautam --- mm/page_ext.c | 1 + 1 file

[PATCH v18 07/14] mm/damon: Implement access pattern recording

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v18 05/14] mm/damon: Track dynamic monitoring target regions update

2020-07-13 Thread SeongJae Park
From: SeongJae Park The monitoring target address range can be dynamically changed. For example, virtual memory could be dynamically mapped and unmapped. Physical memory could be hot-plugged. As the changes could be quite frequent in some cases, DAMON checks the dynamic memory mapping changes

[PATCH v18 08/14] mm/damon: Add a tracepoint

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON can easily integrated with tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster

[PATCH v18 06/14] mm/damon: Implement callbacks for the virtual memory address spaces

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit introduces a reference implementation of the address space specific low level primitives for the virtual address space, so that users of DAMON can easily monitor the data accesses on virtual address spaces of specific processes by simply configuring

[PATCH v18 10/14] tools: Introduce a minimal user-space tool for DAMON

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit imtroduces a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off

[PATCH v18 09/14] mm/damon: Implement a debugfs interface

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for DAMON. It works for the virtual address spaces monitoring. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write

[PATCH v18 13/14] mm/damon: Add user space selftests

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v18 12/14] mm/damon: Add kunit tests

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 661 mm/damon.c | 6 + 3 files changed, 678 insertions

[PATCH v18 11/14] Documentation: Add documents for DAMON

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit adds documents for DAMON under `Documentation/admin-guide/mm/damon/` and `Documentation/vm/damon/`. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/guide.rst | 157 ++ Documentation/admin-guide/mm/damon/index.rst | 15

[PATCH v18 14/14] MAINTAINERS: Update for DAMON

2020-07-13 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 13 + 1 file changed, 13 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 50659d76976b..23348005f5bd 100644 --- a/MAINTAINERS +++ b

Re: Re: [PATCH v18 01/14] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-07-13 Thread SeongJae Park
On Mon, 13 Jul 2020 15:08:42 +0300 Mike Rapoport wrote: > Hi, > > On Mon, Jul 13, 2020 at 10:41:31AM +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit exports 'lookup_page_ext()' to GPL modules. It will be used > > by DAMON in follow

Re: Re: Re: [PATCH v18 01/14] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-07-13 Thread SeongJae Park
On Mon, 13 Jul 2020 20:19:09 +0300 Mike Rapoport wrote: > On Mon, Jul 13, 2020 at 02:21:43PM +0200, SeongJae Park wrote: > > On Mon, 13 Jul 2020 15:08:42 +0300 Mike Rapoport wrote: > > > > > Hi, > > > > > > On Mon, Jul 13, 2020 at 10:41:31AM +0200, S

Re: [PATCH v17 03/15] mm/damon: Implement region based sampling

2020-07-07 Thread SeongJae Park
On Mon, 6 Jul 2020 13:53:10 +0200 SeongJae Park wrote: > From: SeongJae Park > > This commit implements DAMON's target address space independent high > level logics for basic access check and region based sampling. This > doesn't work alone, but needs the target address spa

Re: Re: [PATCH v17 12/15] Documentation/admin-guide/mm: Add a document for DAMON

2020-07-07 Thread SeongJae Park
On Tue, 7 Jul 2020 10:49:06 +0300 Mike Rapoport wrote: > Hello SeongJae, > > On Mon, Jul 06, 2020 at 01:53:19PM +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This commit adds a document for DAMON under > > `Documentation/admin-guide/mm/damon/`.

[RFC v13 1/8] mm/madvise: Export do_madvise() to external GPL modules

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 1ad7522567d4

[RFC v13 0/8] Implement Data Access Monitoring-based Memory Operation Schemes

2020-07-07 Thread SeongJae Park
From: SeongJae Park Changes from Previous Version = - Wordsmith the document, comment, commit messages - Support a scheme of max access count 0 - Use 'unsigned long' for (min|max)_sz_region Introduction DAMON[1] can be used as a primitive for data

[RFC v13 4/8] mm/damon/schemes: Implement a debugfs interface

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v13 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-07-07 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v13 2/8] mm/damon: Account age of target regions

2020-07-07 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v13 6/8] mm/damon/selftests: Add 'schemes' debugfs tests

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v13 5/8] mm/damon/schemes: Implement statistics feature

2020-07-07 Thread SeongJae Park
From: SeongJae Park To tune the DAMON-based operation schemes, knowing how many and how large regions are affected by each of the schemes will be helful. Those stats could be used for not only the tuning, but also monitoring of the working set size and the number of regions, if the scheme does

[RFC v13 7/8] damon/tools: Support more human friendly 'schemes' control

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 141

[RFC v13 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/guide.rst | 41 +- Documentation/admin-guide/mm/damon/plans.rst | 24 +--- Documentation/admin-guide/mm/damon/start.rst

[RFC v5 00/11] DAMON: Support Physical Memory Address Space Monitoring

2020-07-07 Thread SeongJae Park
From: SeongJae Park DAMON[1] programming interface users can extend DAMON for any address space by configuring the address-space specific low level primitives with appropriate ones including their own implementations. However, because the implementation for the virtual address space is only

[RFC v5 01/11] mm/damon/debugfs: Allow users to set initial monitoring target regions

2020-07-07 Thread SeongJae Park
From: SeongJae Park Some users would want to monitor only a part of the entire virtual memory address space. The '->init_target_regions' callback is therefore provided, but only programming interface can use it. For the reason, this commit introduces a new debugfs file, 'init_region'. Us

[RFC v5 02/11] tools/damon: Support init target regions specification

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit updates the damon user space tool to support the initial monitoring target regions specification. Signed-off-by: SeongJae Park --- tools/damon/_damon.py | 39 +++ tools/damon/record.py | 12 +++- tools/damon

[RFC v5 03/11] mm/damon-test: Add more unit tests for 'init_regions'

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit adds more test cases for the new feature, 'init_regions'. Signed-off-by: SeongJae Park --- mm/damon-test.h | 53 + 1 file changed, 53 insertions(+) diff --git a/mm/damon-test.h b/mm/damon-test.h index

[RFC v5 04/11] selftests/damon/_chk_record: Do not check number of gaps

2020-07-07 Thread SeongJae Park
From: SeongJae Park Now the regions can be explicitly set as users want. Therefore checking the number of gaps doesn't make sense. Remove the condition. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/_chk_record.py | 6 -- 1 file changed, 6 deletions(-) diff --git

[RFC v5 06/11] mm/rmap: Export essential functions for rmap_run

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit exports the three essential functions for ramp walk, 'page_lock_anon_vma_read()', 'rmap_walk()', and 'page_rmapping()', to GPL modules. Those will be used by DAMON for the physical memory address based access monitoring in the following commit. Signed-off

[RFC v5 07/11] mm/damon: Implement callbacks for physical memory monitoring

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit implements the four callbacks (->init_target_regions, ->update_target_regions, ->prepare_access_check, and ->check_accesses) for the basic access monitoring of the physical memory address space. By setting the callback pointers to point those, user

[RFC v5 05/11] Docs/damon: Document 'initial_regions' feature

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit documents the 'initial_regions' feature. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 35 1 file changed, 35 insertions(+) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin

[RFC v5 08/11] mm/damon/debugfs: Support physical memory monitoring

2020-07-07 Thread SeongJae Park
From: SeongJae Park This commit makes the debugfs interface to support the physical memory monitoring, in addition to the virtual memory monitoring. Users can do the physical memory monitoring by writing a special keyword, 'paddr\n' to the 'pids' debugfs file. Then, DAMON will check

<    4   5   6   7   8   9   10   11   12   13   >