[PATCH v13 01/15] scripts/spelling: Add a few more typos

2020-05-25 Thread SeongJae Park
From: SeongJae Park This commit adds typos found from DAMON patchset. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 8 1 file changed, 8 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index ffa838f3a2b5..a8214d8e8e4b 100644 --- a/scripts

Re: [PATCH v12 10/16] mm/damon: Add debugfs interface

2020-05-19 Thread SeongJae Park
On Mon, 18 May 2020 12:00:12 +0200 SeongJae Park wrote: > From: SeongJae Park > > This commit adds a debugfs interface for DAMON. [...] > diff --git a/mm/damon.c b/mm/damon.c > index ddd78843f79a..f31310536c79 100644 > --- a/mm/damon.c > +++ b/mm/damon.c > @@ -10,6 +1

[PATCH v12 16/16] MAINTAINERS: Update for DAMON

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5a5332b3591d..586513e92b65 100644 --- a/MAINTAINERS +++ b

[PATCH v12 15/16] mm/damon: Add user space selftests

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v12 14/16] mm/damon: Add kunit tests

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v12 12/16] tools: Add a minimal user-space tool for DAMON

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v12 11/16] mm/damon: Add tracepoints

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park --- include/trace/events

[PATCH v12 10/16] mm/damon: Add debugfs interface

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v12 09/16] mm/damon: Implement access pattern recording

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v12 08/16] mm/damon: Implement callbacks

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v12 07/16] mm/damon: Apply dynamic memory mapping changes

2020-05-18 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v12 06/16] mm/damon: Split regions into 3 subregions if necessary

2020-05-18 Thread SeongJae Park
From: SeongJae Park Suppose that there are a very wide and cold region and a hot region, and both regions are identified by DAMON. And then, the middle small region inside the very wide and cold region becomes hot. DAMON will not be able to identify this new region because the adaptive regions

[PATCH v12 04/16] mm/damon: Implement region based sampling

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v12 05/16] mm/damon: Adaptively adjust regions

2020-05-18 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v12 03/16] mm: Introduce Data Access MONitor (DAMON)

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v12 02/16] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index 4ade843ff588..71169b45bba9 100644 --- a/mm/page_ext.c

[PATCH v12 00/16] Introduce Data Access MONitor (DAMON)

2020-05-18 Thread SeongJae Park
From: SeongJae Park Introduction DAMON is a data access monitoring framework subsystem for the Linux kernel. The core mechanisms of DAMON called 'region based sampling' and adaptive regions adjustment' (refer to :doc:`mechanisms` for the detail) make it accurate, efficient

[PATCH v12 01/16] scripts/spelling: Add a few more typos

2020-05-18 Thread SeongJae Park
From: SeongJae Park This commit adds typos found from DAMON patchset. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 8 1 file changed, 8 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index ffa838f3a2b5..a8214d8e8e4b 100644 --- a/scripts

[RFC v8 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- .../admin-guide/mm/data_access_monitor.rst| 100 +- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/mm

[RFC v8 7/8] damon/tools: Support more human friendly 'schemes' control

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 127

[RFC v8 5/8] mm/damon-test: Add kunit test case for regions age accounting

2020-05-12 Thread SeongJae Park
From: SeongJae Park After merges of regions, each region should know their last shape in proper way to measure the changes from the last modification and reset the age if the changes are significant. This commit adds kunit test cases checking whether the regions are knowing their last shape

[RFC v8 6/8] mm/damon/selftests: Add 'schemes' debugfs tests

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v8 4/8] mm/damon/schemes: Implement a debugfs interface

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v8 3/8] mm/damon: Implement data access monitoring-based operation schemes

2020-05-12 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v8 2/8] mm/damon: Account age of target regions

2020-05-12 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v8 1/8] mm/madvise: Export do_madvise() to external GPL modules

2020-05-12 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 80f8a1839f70

[RFC v8 0/8] Implement Data Access Monitoring-based Memory Operation Schemes

2020-05-12 Thread SeongJae Park
From: SeongJae Park DAMON[1] can be used as a primitive for data access awared memory management optimizations. That said, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[PATCH v11 16/16] MAINTAINERS: Update for DAMON

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5a5332b3591d..cb4a7fa3cdfe 100644 --- a/MAINTAINERS +++ b

[PATCH v11 14/16] mm/damon: Add kunit tests

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v11 15/16] mm/damon: Add user space selftests

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v11 13/16] Documentation/admin-guide/mm: Add a document for DAMON

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a simple document for DAMON under `Documentation/admin-guide/mm`. Signed-off-by: SeongJae Park --- .../admin-guide/mm/data_access_monitor.rst| 428 ++ Documentation/admin-guide/mm/index.rst| 1 + 2 files changed, 429

[PATCH v11 11/16] mm/damon: Add tracepoints

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park --- include/trace/events

[PATCH v11 12/16] tools: Add a minimal user-space tool for DAMON

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v11 10/16] mm/damon: Add debugfs interface

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v11 09/16] mm/damon: Implement access pattern recording

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v11 07/16] mm/damon: Apply dynamic memory mapping changes

2020-05-11 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v11 08/16] mm/damon: Implement callbacks

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v11 05/16] mm/damon: Adaptively adjust regions

2020-05-11 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v11 06/16] mm/damon: Split regions into 3 subregions if necessary

2020-05-11 Thread SeongJae Park
From: SeongJae Park Suppose that there are a very wide and cold region and a hot region, and both regions are identified by DAMON. And then, the middle small region inside the very wide and cold region becomes hot. DAMON will not be able to identify this new region because the adaptive regions

[PATCH v11 03/16] mm: Introduce Data Access MONitor (DAMON)

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v11 04/16] mm/damon: Implement region based sampling

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v11 01/16] scripts/spelling: Add a few more typos

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit adds typos found from DAMON patchset. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 8 1 file changed, 8 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index ffa838f3a2b5..a8214d8e8e4b 100644 --- a/scripts

[PATCH v11 02/16] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-05-11 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index 4ade843ff588..71169b45bba9 100644 --- a/mm/page_ext.c

[PATCH v11 00/16] Introduce Data Access MONitor (DAMON)

2020-05-11 Thread SeongJae Park
From: SeongJae Park Introduction Memory management decisions can be improved if finer data access information is available. However, because such finer information usually comes with higher overhead, most systems including Linux forgives the potential benefit and rely on only

Re: [PATCH v10 06/16] mm/damon: Split regions into 4 subregions if possible

2020-05-07 Thread SeongJae Park
On Tue, 5 May 2020 13:08:05 +0200 SeongJae Park wrote: > From: SeongJae Park > > Suppose that there are a very wide and cold region and a hot region, and > both regions are identified by DAMON. And then, the middle small region > inside the very wide and cold region beco

Re: Re: Re: Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-06 Thread SeongJae Park
On Wed, 6 May 2020 07:41:51 -0700 "Paul E. McKenney" wrote: > On Wed, May 06, 2020 at 02:59:26PM +0200, SeongJae Park wrote: > > TL; DR: It was not kernel's fault, but the benchmark program. > > > > So, the problem is reproducible using the lebench[1] only. I car

Re: Re: Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-06 Thread SeongJae Park
aster/TEST_DIR/OS_Eval.c#L820 [3] https://github.com/LinuxPerfStudy/LEBench/blob/master/TEST_DIR/OS_Eval.c#L822 Thanks, SeongJae Park

Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-06 Thread SeongJae Park
nitude and scope you must do an > allmodconfig build. Definitely my fault. I will fix this in next spin. Thanks, SeongJae Park > > Thank you.

Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 11:27:20 -0700 "Paul E. McKenney" wrote: > On Tue, May 05, 2020 at 07:49:43PM +0200, SeongJae Park wrote: > > On Tue, 5 May 2020 10:23:58 -0700 "Paul E. McKenney" > > wrote: > > > > > On Tue, Ma

Re: Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 11:17:07 -0700 "Paul E. McKenney" wrote: > On Tue, May 05, 2020 at 07:56:05PM +0200, SeongJae Park wrote: > > On Tue, 5 May 2020 10:30:36 -0700 "Paul E. McKenney" > > wrote: > > > > > On Tue, May 05, 2020 at 07:05:53PM +020

Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 10:28:50 -0700 "Paul E. McKenney" wrote: > On Tue, May 05, 2020 at 09:37:42AM -0700, Eric Dumazet wrote: > > > > > > On 5/5/20 9:31 AM, Eric Dumazet wrote: > > > > > > > > > On 5/5/20 9:25 AM, Eric Dumazet wrote: &

Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 10:30:36 -0700 "Paul E. McKenney" wrote: > On Tue, May 05, 2020 at 07:05:53PM +0200, SeongJae Park wrote: > > On Tue, 5 May 2020 09:37:42 -0700 Eric Dumazet > > wrote: > > > > > > > > > > > On 5/5/20 9:31 AM, Eric

Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 10:23:58 -0700 "Paul E. McKenney" wrote: > On Tue, May 05, 2020 at 09:25:06AM -0700, Eric Dumazet wrote: > > > > > > On 5/5/20 9:13 AM, SeongJae Park wrote: > > > On Tue, 5 May 2020 09:00:44 -0700 Eric Dumazet > > > wr

Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 09:37:42 -0700 Eric Dumazet wrote: > > > On 5/5/20 9:31 AM, Eric Dumazet wrote: > > > > > > On 5/5/20 9:25 AM, Eric Dumazet wrote: > >> > >> > >> On 5/5/20 9:13 AM, SeongJae Park wrote: > >>> On Tue, 5 May 2

Re: Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 09:00:44 -0700 Eric Dumazet wrote: > On Tue, May 5, 2020 at 8:47 AM SeongJae Park wrote: > > > > On Tue, 5 May 2020 08:20:50 -0700 Eric Dumazet > > wrote: > > > > > > > > > > > On 5/5/20 8:07 AM, SeongJae Park wrote:

Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 08:20:50 -0700 Eric Dumazet wrote: > > > On 5/5/20 8:07 AM, SeongJae Park wrote: > > On Tue, 5 May 2020 07:53:39 -0700 Eric Dumazet wrote: > > > > >> Why do we have 10,000,000 objects around ? Could this be because of > >>

Re: Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 07:53:39 -0700 Eric Dumazet wrote: > On Tue, May 5, 2020 at 4:54 AM SeongJae Park wrote: > > > > CC-ing sta...@vger.kernel.org and adding some more explanations. > > > > On Tue, 5 May 2020 10:10:33 +0200 SeongJae Park wrote: &g

Re: RE: [PATCH net 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 13:44:42 +0100 Al Viro wrote: > CAUTION: This email originated from outside of the organization. Do not cli= > ck links or open attachments unless you can confirm the sender and know the= > content is safe. > > > > On Tue, May 05, 2020 at 09:28:39AM

Re: [PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
CC-ing sta...@vger.kernel.org and adding some more explanations. On Tue, 5 May 2020 10:10:33 +0200 SeongJae Park wrote: > From: SeongJae Park > > The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the > deallocation of 'socket_alloc' to be done

[PATCH v10 16/16] MAINTAINERS: Update for DAMON

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5a5332b3591d..cb4a7fa3cdfe 100644 --- a/MAINTAINERS +++ b

[PATCH v10 13/16] Documentation/admin-guide/mm: Add a document for DAMON

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds a simple document for DAMON under `Documentation/admin-guide/mm`. Signed-off-by: SeongJae Park --- .../admin-guide/mm/data_access_monitor.rst| 428 ++ Documentation/admin-guide/mm/index.rst| 1 + 2 files changed, 429

[PATCH v10 12/16] tools: Add a minimal user-space tool for DAMON

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds a shallow wrapper python script, ``/tools/damon/damo`` that provides more convenient interface. Note that it is only aimed to be used for minimal reference of the DAMON's debugfs interfaces and for debugging of the DAMON itself. Signed-off-by: SeongJae Park

[PATCH v10 15/16] mm/damon: Add user space selftests

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile| 7 + .../selftests/damon/_chk_dependency.sh| 28 tools/testing/selftests/damon

[PATCH v10 14/16] mm/damon: Add kunit tests

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds kunit based unit tests for DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/Kconfig | 11 + mm/damon-test.h | 622 mm/damon.c | 6 + 3 files changed, 639 insertions

[PATCH v10 11/16] mm/damon: Add tracepoints

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON will be easily integrated with any tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park --- include/trace/events

[PATCH v10 10/16] mm/damon: Add debugfs interface

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds a debugfs interface for DAMON. DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes -- Users can read and write the ``sampling interval``, ``aggregation interval

[PATCH v10 09/16] mm/damon: Implement access pattern recording

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However

[PATCH v10 07/16] mm/damon: Apply dynamic memory mapping changes

2020-05-05 Thread SeongJae Park
From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON

[PATCH v10 08/16] mm/damon: Implement callbacks

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored

[PATCH v10 04/16] mm/damon: Implement region based sampling

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check -- DAMON

[PATCH v10 06/16] mm/damon: Split regions into 4 subregions if possible

2020-05-05 Thread SeongJae Park
From: SeongJae Park Suppose that there are a very wide and cold region and a hot region, and both regions are identified by DAMON. And then, the middle small region inside the very wide and cold region becomes hot. DAMON will not be able to identify this new region because the adaptive regions

[PATCH v10 03/16] mm: Introduce Data Access MONitor (DAMON)

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON

[PATCH v10 05/16] mm/damon: Adaptively adjust regions

2020-05-05 Thread SeongJae Park
From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have

[PATCH v10 00/16] Introduce Data Access MONitor (DAMON)

2020-05-05 Thread SeongJae Park
From: SeongJae Park Introduction Memory management decisions can be improved if finer data access information is available. However, because such finer information usually comes with higher overhead, most systems including Linux forgives the potential benefit and rely on only

[PATCH v10 02/16] mm/page_ext: Export lookup_page_ext() to GPL modules

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index 4ade843ff588..71169b45bba9 100644 --- a/mm/page_ext.c

[PATCH v10 01/16] scripts/spelling: Add a few more typos

2020-05-05 Thread SeongJae Park
From: SeongJae Park This commit adds typos found from DAMON patchset. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 8 1 file changed, 8 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index ffa838f3a2b5..a8214d8e8e4b 100644 --- a/scripts

[PATCH net v2 2/2] Revert "sockfs: switch to ->free_inode()"

2020-05-05 Thread SeongJae Park
From: SeongJae Park This reverts commit 6d7855c54e1e269275d7c504f8f62a0b7a5b3f18. The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the deallocation of 'socket_alloc' to be done asynchronously using RCU, as same to 'sock.wq'. The change made 'socket_alloc' liv

[PATCH net v2 1/2] Revert "coallocate socket_wq with socket itself"

2020-05-05 Thread SeongJae Park
From: SeongJae Park This reverts commit 333f7909a8573145811c4ab7d8c9092301707721. The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the deallocation of 'socket_alloc' to be done asynchronously using RCU, as same to 'sock.wq'. And the following commit 333f7909a857

[PATCH net v2 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
From: SeongJae Park The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the deallocation of 'socket_alloc' to be done asynchronously using RCU, as same to 'sock.wq'. And the following commit 333f7909a857 ("coallocate socket_sq with socket itself") made

Re: Re: [PATCH net 2/2] Revert "sockfs: switch to ->free_inode()"

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 09:45:35 +0200 Greg KH wrote: > On Tue, May 05, 2020 at 09:28:41AM +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This reverts commit 6d7855c54e1e269275d7c504f8f62a0b7a5b3f18. > > > > The commit 6d7855c54e1e ("s

Re: Re: [PATCH net 1/2] Revert "coallocate socket_wq with socket itself"

2020-05-05 Thread SeongJae Park
On Tue, 5 May 2020 09:45:11 +0200 Greg KH wrote: > On Tue, May 05, 2020 at 09:28:40AM +0200, SeongJae Park wrote: > > From: SeongJae Park > > > > This reverts commit 333f7909a8573145811c4ab7d8c9092301707721. > > > > The commit 6d7855c54e1e ("s

[PATCH net 2/2] Revert "sockfs: switch to ->free_inode()"

2020-05-05 Thread SeongJae Park
From: SeongJae Park This reverts commit 6d7855c54e1e269275d7c504f8f62a0b7a5b3f18. The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the deallocation of 'socket_alloc' to be done asynchronously using RCU, as same to 'sock.wq'. The change made 'socket_alloc' liv

[PATCH net 0/2] Revert the 'socket_alloc' life cycle change

2020-05-05 Thread SeongJae Park
From: SeongJae Park The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the deallocation of 'socket_alloc' to be done asynchronously using RCU, as same to 'sock.wq'. And the following commit 333f7909a857 ("coallocate socket_sq with socket itself") made

[PATCH net 1/2] Revert "coallocate socket_wq with socket itself"

2020-05-05 Thread SeongJae Park
From: SeongJae Park This reverts commit 333f7909a8573145811c4ab7d8c9092301707721. The commit 6d7855c54e1e ("sockfs: switch to ->free_inode()") made the deallocation of 'socket_alloc' to be done asynchronously using RCU, as same to 'sock.wq'. And the following commit 333f7909a857

[RFC v7 5/7] mm/damon-test: Add kunit test case for regions age accounting

2020-04-29 Thread SeongJae Park
From: SeongJae Park After merges of regions, each region should know their last shape in proper way to measure the changes from the last modification and reset the age if the changes are significant. This commit adds kunit test cases checking whether the regions are knowing their last shape

[RFC v7 7/7] damon/tools: Support more human friendly 'schemes' control

2020-04-29 Thread SeongJae Park
From: SeongJae Park This commit implements 'schemes' subcommand of the damon userspace tool. It can be used to describe and apply the data access monitoring-based operation schemes in more human friendly fashion. Signed-off-by: SeongJae Park --- tools/damon/_convert_damos.py | 126

[RFC v7 6/7] mm/damon/selftests: Add 'schemes' debugfs tests

2020-04-29 Thread SeongJae Park
From: SeongJae Park This commit adds simple selftets for 'schemes' debugfs file of DAMON. Signed-off-by: SeongJae Park --- .../testing/selftests/damon/debugfs_attrs.sh | 29 +++ 1 file changed, 29 insertions(+) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b

[RFC v7 3/7] mm/damon: Implement data access monitoring-based operation schemes

2020-04-29 Thread SeongJae Park
From: SeongJae Park In many cases, users might use DAMON for simple data access aware memory management optimizations such as applying an operation scheme to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region l

[RFC v7 2/7] mm/damon: Account age of target regions

2020-04-29 Thread SeongJae Park
From: SeongJae Park DAMON can be used as a primitive for data access pattern aware memory management optimizations. However, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

[RFC v7 4/7] mm/damon/schemes: Implement a debugfs interface

2020-04-29 Thread SeongJae Park
From: SeongJae Park This commit implements a debugfs interface for the data access monitoring oriented memory management schemes. It is supposed to be used by administrators and/or privileged user space programs. Users can read and update the rules using ``/damon/schemes`` file. The format

[RFC v7 1/7] mm/madvise: Export do_madvise() to external GPL modules

2020-04-29 Thread SeongJae Park
From: SeongJae Park This commit exports 'do_madvise()' to external GPL modules, so that other modules including DAMON could use the function. Signed-off-by: SeongJae Park --- mm/madvise.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/madvise.c b/mm/madvise.c index 80f8a1839f70

[RFC v7 0/7] Implement Data Access Monitoring-based Memory Operation Schemes

2020-04-29 Thread SeongJae Park
From: SeongJae Park DAMON[1] can be used as a primitive for data access awared memory management optimizations. That said, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves

Re: Re: [PATCH v9 00/15] Introduce Data Access MONitor (DAMON)

2020-04-29 Thread SeongJae Park
On Wed, 29 Apr 2020 10:18:06 +0100 Jonathan Cameron wrote: > On Wed, 29 Apr 2020 09:49:54 +0200 > SeongJae Park wrote: > > > On Tue, 28 Apr 2020 17:17:13 +0100 Jonathan Cameron > > wrote: > > > > > On Tue, 28 Apr 2020 15:23:42 +0200 > > > Seon

Re: Re: [PATCH v9 00/15] Introduce Data Access MONitor (DAMON)

2020-04-29 Thread SeongJae Park
On Tue, 28 Apr 2020 17:17:13 +0100 Jonathan Cameron wrote: > On Tue, 28 Apr 2020 15:23:42 +0200 > SeongJae Park wrote: > > > On Tue, 28 Apr 2020 13:27:04 +0100 Jonathan Cameron > > wrote: > > > > > On Mon, 27 Apr 2020 14:04:27 +0200 > > >

Re: Re: [PATCH v9 00/15] Introduce Data Access MONitor (DAMON)

2020-04-28 Thread SeongJae Park
On Tue, 28 Apr 2020 13:27:04 +0100 Jonathan Cameron wrote: > On Mon, 27 Apr 2020 14:04:27 +0200 > SeongJae Park wrote: > > > From: SeongJae Park > > > > Introduction > > > > > > Memory management decisions can be improved if finer

[PATCH] Documentation: kunit: Fix verification command

2019-09-07 Thread SeongJae Park
the example. Signed-off-by: SeongJae Park --- Documentation/dev-tools/kunit/start.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 6dc229e..aeeddfa 100644 --- a/Documentation/dev-tools/kunit

Re: [PATCH] kunit: Fix '--build_dir' option

2019-09-07 Thread SeongJae Park
On Sat, Sep 7, 2019 at 4:33 AM shuah wrote: > > On 9/6/19 7:16 PM, Brendan Higgins wrote: > > On Fri, Sep 6, 2019 at 9:12 AM SeongJae Park wrote: > >> > >> Running kunit with '--build_dir' option gives following error message: > >> > >> ``` >

[PATCH] kunit: Fix '--build_dir' option

2019-09-06 Thread SeongJae Park
otFoundError'. This commit fixes the problem by properly passing the '--build_dir' option value to the 'run_kernel()'. Signed-off-by: SeongJae Park --- tools/testing/kunit/kunit.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/tes

Re: [PATCH] kunit: Fix '--build_dir' option

2019-09-06 Thread SeongJae Park
On Fri, Sep 6, 2019 at 5:51 PM shuah wrote: > > On 9/6/19 6:05 AM, SeongJae Park wrote: > > kunit fails to run with '--build_dir' option because the option is not > > properly sent to kernel running procedure. This commit fixes the > > problem. > > Can you

[PATCH] kunit: Fix '--build_dir' option

2019-09-06 Thread SeongJae Park
kunit fails to run with '--build_dir' option because the option is not properly sent to kernel running procedure. This commit fixes the problem. Signed-off-by: SeongJae Park --- tools/testing/kunit/kunit.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/testing

<    3   4   5   6   7   8   9   10   11   12   >