Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On 11/5/18 10:35 PM, Mike Rapoport wrote: > On Mon, Nov 05, 2018 at 01:12:40PM -0800, Matthew Wilcox wrote: >> On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: >>> @@ -21,10 +21,10 @@ Virtual Memory Primer >>> The physical memory in a computer system is a limited resource and >>> even for systems that support memory hotplug there is a hard limit on >>> the amount of memory that can be installed. The physical memory is not >>> -necessary contiguous, it might be accessible as a set of distinct >>> +necessary contiguous; it might be accessible as a set of distinct >> >> necessarily >> >>> address ranges. Besides, different CPU architectures, and even >>> -different implementations of the same architecture have different view >>> -how these address ranges defined. >>> +different implementations of the same architecture have different views >>> +of how these address ranges defined. >> >> "are defined"? >> >>> Each physical memory page can be mapped as one or more virtual >>> pages. These mappings are described by page tables that allow >>> -translation from virtual address used by programs to real address in >>> -the physical memory. The page tables organized hierarchically. >>> +translation from a virtual address used by programs to the real >>> +address in the physical memory. The page tables are organized >>> +hierarchically. >> >> I don't like the term "real address". Can we say "physical address in >> memory" here, or "address of physical memory" or something? > > I didn't really like it as well, but I couldn't think of any better > adjective to emphasize that address in the physical memory is "the real > thing". > > Maybe the best would be to drop "real" and make it > > "translation from a virtual address used by programs to the > address in the physical memory" physical memory address ? -- ~Randy
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On Mon, Nov 05, 2018 at 01:12:40PM -0800, Matthew Wilcox wrote: > On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: > > @@ -21,10 +21,10 @@ Virtual Memory Primer > > The physical memory in a computer system is a limited resource and > > even for systems that support memory hotplug there is a hard limit on > > the amount of memory that can be installed. The physical memory is not > > -necessary contiguous, it might be accessible as a set of distinct > > +necessary contiguous; it might be accessible as a set of distinct > > necessarily > > > address ranges. Besides, different CPU architectures, and even > > -different implementations of the same architecture have different view > > -how these address ranges defined. > > +different implementations of the same architecture have different views > > +of how these address ranges defined. > > "are defined"? > > > Each physical memory page can be mapped as one or more virtual > > pages. These mappings are described by page tables that allow > > -translation from virtual address used by programs to real address in > > -the physical memory. The page tables organized hierarchically. > > +translation from a virtual address used by programs to the real > > +address in the physical memory. The page tables are organized > > +hierarchically. > > I don't like the term "real address". Can we say "physical address in > memory" here, or "address of physical memory" or something? I didn't really like it as well, but I couldn't think of any better adjective to emphasize that address in the physical memory is "the real thing". Maybe the best would be to drop "real" and make it "translation from a virtual address used by programs to the address in the physical memory" > > -page filled with zeroes. When the program performs a write, regular > > +page filled with zeroes. When the program performs a write, a regular > > physical page will be allocated to hold the written data. The page > > will be marked dirty and if the kernel will decide to repurpose it, > > the dirty page will be swapped out. > > "if the kernel will decide to repurpose it" is awkward. How about > > if the kernel decides to repurpose it"? > > > OOM killer > > == > > > > -It may happen, that on a loaded machine memory will be exhausted. When > > +It may happen that on a loaded machine memory will be exhausted. When > > the kernel detects that the system runs out of memory (OOM) it invokes > > `OOM killer`. > > Its mission is simple: all it has to do is to select a > > task to sacrifice for the sake of the overall system health. The > > It is possible for the kernel to be unable to reclaim enough memory to > continue to operate. In order to save the rest of the system, it invokes > the `OOM killer` to sacrifice one task. How about: It is possible that on a loaded machine memory will be exhausted and the kernel will be unable to reclaim enough memory to continue to operate. In order to save the rest of the system, it invokes the `OOM killer`. The `OOM killer` selects a task to sacrifice for the sake of the overall system health. The selected task is killed in a hope that after it exits enough memory will be freed to continue normal operation. -- Sincerely yours, Mike.
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: > @@ -21,10 +21,10 @@ Virtual Memory Primer > The physical memory in a computer system is a limited resource and > even for systems that support memory hotplug there is a hard limit on > the amount of memory that can be installed. The physical memory is not > -necessary contiguous, it might be accessible as a set of distinct > +necessary contiguous; it might be accessible as a set of distinct necessarily > address ranges. Besides, different CPU architectures, and even > -different implementations of the same architecture have different view > -how these address ranges defined. > +different implementations of the same architecture have different views > +of how these address ranges defined. "are defined"? > Each physical memory page can be mapped as one or more virtual > pages. These mappings are described by page tables that allow > -translation from virtual address used by programs to real address in > -the physical memory. The page tables organized hierarchically. > +translation from a virtual address used by programs to the real > +address in the physical memory. The page tables are organized > +hierarchically. I don't like the term "real address". Can we say "physical address in memory" here, or "address of physical memory" or something? > -page filled with zeroes. When the program performs a write, regular > +page filled with zeroes. When the program performs a write, a regular > physical page will be allocated to hold the written data. The page > will be marked dirty and if the kernel will decide to repurpose it, > the dirty page will be swapped out. "if the kernel will decide to repurpose it" is awkward. How about if the kernel decides to repurpose it"? > OOM killer > == > > -It may happen, that on a loaded machine memory will be exhausted. When > +It may happen that on a loaded machine memory will be exhausted. When > the kernel detects that the system runs out of memory (OOM) it invokes > `OOM killer`. > Its mission is simple: all it has to do is to select a > task to sacrifice for the sake of the overall system health. The It is possible for the kernel to be unable to reclaim enough memory to continue to operate. In order to save the rest of the system, it invokes the `OOM killer` to sacrifice one task.
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On 11/5/18 11:58 AM, Mike Rapoport wrote: > From: Mike Rapoport > > Signed-off-by: Mike Rapoport > Cc: Randy Dunlap > --- > > There was a couple of grammar fixes Randy suggested a while ago, but it > seems I've never sent them out. > > Documentation/admin-guide/mm/concepts.rst | 39 > --- > 1 file changed, 20 insertions(+), 19 deletions(-) Hi Mike, one nit: > @@ -121,8 +122,8 @@ Nodes > Many multi-processor machines are NUMA - Non-Uniform Memory Access - > systems. In such systems the memory is arranged into banks that have > different access latency depending on the "distance" from the > -processor. Each bank is referred as `node` and for each node Linux > -constructs an independent memory management subsystem. A node has it's > +processor. Each bank is referred as a `node` and for each node Linux is referred to as a > +constructs an independent memory management subsystem. A node has its > own set of zones, lists of free and used pages and various statistics > counters. You can find more details about NUMA in > :ref:`Documentation/vm/numa.rst ` and in Reviewed-by: Randy Dunlap thanks. -- ~Randy
[PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
From: Mike Rapoport Signed-off-by: Mike Rapoport Cc: Randy Dunlap --- There was a couple of grammar fixes Randy suggested a while ago, but it seems I've never sent them out. Documentation/admin-guide/mm/concepts.rst | 39 --- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/Documentation/admin-guide/mm/concepts.rst b/Documentation/admin-guide/mm/concepts.rst index 291699c..ab7a0f9 100644 --- a/Documentation/admin-guide/mm/concepts.rst +++ b/Documentation/admin-guide/mm/concepts.rst @@ -4,13 +4,13 @@ Concepts overview = -The memory management in Linux is complex system that evolved over the -years and included more and more functionality to support variety of +The memory management in Linux is a complex system that evolved over the +years and included more and more functionality to support a variety of systems from MMU-less microcontrollers to supercomputers. The memory -management for systems without MMU is called ``nommu`` and it +management for systems without an MMU is called ``nommu`` and it definitely deserves a dedicated document, which hopefully will be eventually written. Yet, although some of the concepts are the same, -here we assume that MMU is available and CPU can translate a virtual +here we assume that an MMU is available and a CPU can translate a virtual address to a physical address. .. contents:: :local: @@ -21,10 +21,10 @@ Virtual Memory Primer The physical memory in a computer system is a limited resource and even for systems that support memory hotplug there is a hard limit on the amount of memory that can be installed. The physical memory is not -necessary contiguous, it might be accessible as a set of distinct +necessary contiguous; it might be accessible as a set of distinct address ranges. Besides, different CPU architectures, and even -different implementations of the same architecture have different view -how these address ranges defined. +different implementations of the same architecture have different views +of how these address ranges defined. All this makes dealing directly with physical memory quite complex and to avoid this complexity a concept of virtual memory was developed. @@ -48,8 +48,9 @@ appropriate kernel configuration option. Each physical memory page can be mapped as one or more virtual pages. These mappings are described by page tables that allow -translation from virtual address used by programs to real address in -the physical memory. The page tables organized hierarchically. +translation from a virtual address used by programs to the real +address in the physical memory. The page tables are organized +hierarchically. The tables at the lowest level of the hierarchy contain physical addresses of actual pages used by the software. The tables at higher @@ -121,8 +122,8 @@ Nodes Many multi-processor machines are NUMA - Non-Uniform Memory Access - systems. In such systems the memory is arranged into banks that have different access latency depending on the "distance" from the -processor. Each bank is referred as `node` and for each node Linux -constructs an independent memory management subsystem. A node has it's +processor. Each bank is referred as a `node` and for each node Linux +constructs an independent memory management subsystem. A node has its own set of zones, lists of free and used pages and various statistics counters. You can find more details about NUMA in :ref:`Documentation/vm/numa.rst ` and in @@ -149,7 +150,7 @@ for program's stack and heap or by explicit calls to mmap(2) system call. Usually, the anonymous mappings only define virtual memory areas that the program is allowed to access. The read accesses will result in creation of a page table entry that references a special physical -page filled with zeroes. When the program performs a write, regular +page filled with zeroes. When the program performs a write, a regular physical page will be allocated to hold the written data. The page will be marked dirty and if the kernel will decide to repurpose it, the dirty page will be swapped out. @@ -181,8 +182,8 @@ pressure. The process of freeing the reclaimable physical memory pages and repurposing them is called (surprise!) `reclaim`. Linux can reclaim pages either asynchronously or synchronously, depending on the state -of the system. When system is not loaded, most of the memory is free -and allocation request will be satisfied immediately from the free +of the system. When the system is not loaded, most of the memory is free +and allocation requests will be satisfied immediately from the free pages supply. As the load increases, the amount of the free pages goes down and when it reaches a certain threshold (high watermark), an allocation request will awaken the ``kswapd`` daemon. It will @@ -190,7 +191,7 @@ asynchronously scan memory pages and either just free them if the data they contain is available elsewhere, or evict to the
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Mon, Nov 5, 2018 at 5:15 AM Martin Schwidefsky wrote: > > This patch would work for me: Thanks, applied, Linus
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Mon, 5 Nov 2018 14:15:35 +0100
Martin Schwidefsky wrote:
>
> Follow-up question: the __no_sanitize_address_or_inline define has the
> 'notrace'
> option that is missing for __no_kasan_or_inline. We need that option for
> __load_psw_mask, if we do the replacement all users of __no_kasan_or_inline
> would get the option as well. This affects __read_once_size_nocheck and
> read_word_at_a_time. Do these function have to be traceable ?
Some history on why I added notrace to inline. It was to make things
more consistent. Since gcc wont add a fentry/mcount call to inlined
functions, having those functions show up in the trace depending on
whether or not gcc honored the "inline" tag made things confusing. And
it even once caused a crash when local_irq_save() stopped being
inlined.
I added notrace to inline so that if you mark something as inline, it
would not show up in the trace regardless of gcc deciding to inline the
function or not.
>
> This patch would work for me:
> --
> >From 4aaa09fe4b54e930edabac86606dee979b12647c Mon Sep 17 00:00:00 2001
> From: Martin Schwidefsky
> Date: Mon, 5 Nov 2018 07:36:28 +0100
> Subject: [PATCH] compiler: remove __no_sanitize_address_or_inline again
>
> The __no_sanitize_address_or_inline and __no_kasan_or_inline defines
> are almost identical. The only difference is that __no_kasan_or_inline
> does not have the 'notrace' attribute.
>
> To be able to replace __no_sanitize_address_or_inline with the older
> definition, add 'notrace' to __no_kasan_or_inline and change to two
> users of __no_sanitize_address_or_inline in the s390 code.
>
> The 'notrace' option is necessary for e.g. the __load_psw_mask function
> in arch/s390/include/asm/processor.h. Without the option it is possible
> to trace __load_psw_mask which leads to kernel stack overflow.
Acked-by: Steven Rostedt (VMware)
-- Steve
>
> Signed-off-by: Martin Schwidefsky
> ---
> arch/s390/include/asm/processor.h | 4 ++--
> include/linux/compiler-gcc.h | 12
> include/linux/compiler.h | 2 +-
> 3 files changed, 3 insertions(+), 15 deletions(-)
>
> diff --git a/arch/s390/include/asm/processor.h
> b/arch/s390/include/asm/processor.h
> index 302795c47c06..81038ab357ce 100644
> --- a/arch/s390/include/asm/processor.h
> +++ b/arch/s390/include/asm/processor.h
> @@ -236,7 +236,7 @@ static inline unsigned long current_stack_pointer(void)
> return sp;
> }
>
> -static __no_sanitize_address_or_inline unsigned short stap(void)
> +static __no_kasan_or_inline unsigned short stap(void)
> {
> unsigned short cpu_address;
>
> @@ -330,7 +330,7 @@ static inline void __load_psw(psw_t psw)
> * Set PSW mask to specified value, while leaving the
> * PSW addr pointing to the next instruction.
> */
> -static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long
> mask)
> +static __no_kasan_or_inline void __load_psw_mask(unsigned long mask)
> {
> unsigned long addr;
> psw_t psw;
> diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
> index c0f5db3a9621..2010493e1040 100644
> --- a/include/linux/compiler-gcc.h
> +++ b/include/linux/compiler-gcc.h
> @@ -143,18 +143,6 @@
> #define KASAN_ABI_VERSION 3
> #endif
>
> -/*
> - * Because __no_sanitize_address conflicts with inlining:
> - * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
> - * we do one or the other.
> - */
> -#ifdef CONFIG_KASAN
> -#define __no_sanitize_address_or_inline
> \
> - __no_sanitize_address __maybe_unused notrace
> -#else
> -#define __no_sanitize_address_or_inline inline
> -#endif
> -
> #if GCC_VERSION >= 50100
> #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
> #endif
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index 18c80cfa4fc4..06396c1cf127 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -189,7 +189,7 @@ void __read_once_size(const volatile void *p, void *res,
> int size)
> * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
> * '__maybe_unused' allows us to avoid defined-but-not-used warnings.
> */
> -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
> +# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused
> #else
> # define __no_kasan_or_inline __always_inline
> #endif
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Mon, 5 Nov 2018 07:02:56 +0100
Martin Schwidefsky wrote:
> On Fri, 2 Nov 2018 09:09:32 -0700
> Linus Torvalds wrote:
>
> > On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin
> > wrote:
> > >
> > > You're right, version checks shouldn't matter here. But
> > > __no_sanitize_address_or_inline
> > > shouldn't have been added in the first place, because we already have
> > > almost the same
> > >__no_kasan_or_inline:
> >
> > Ahh, very good.
> >
> > Vasily, Martin - since __no_sanitize_address_or_inline was added just
> > for s390, would you mind replacing it with __no_kasan_or_inline
> > instead, and testing that in whatever failed before?
> >
> > Then we can just remove that unnecessary symbol #define entirely..
>
> Ok, will do.
Follow-up question: the __no_sanitize_address_or_inline define has the 'notrace'
option that is missing for __no_kasan_or_inline. We need that option for
__load_psw_mask, if we do the replacement all users of __no_kasan_or_inline
would get the option as well. This affects __read_once_size_nocheck and
read_word_at_a_time. Do these function have to be traceable ?
This patch would work for me:
--
>From 4aaa09fe4b54e930edabac86606dee979b12647c Mon Sep 17 00:00:00 2001
From: Martin Schwidefsky
Date: Mon, 5 Nov 2018 07:36:28 +0100
Subject: [PATCH] compiler: remove __no_sanitize_address_or_inline again
The __no_sanitize_address_or_inline and __no_kasan_or_inline defines
are almost identical. The only difference is that __no_kasan_or_inline
does not have the 'notrace' attribute.
To be able to replace __no_sanitize_address_or_inline with the older
definition, add 'notrace' to __no_kasan_or_inline and change to two
users of __no_sanitize_address_or_inline in the s390 code.
The 'notrace' option is necessary for e.g. the __load_psw_mask function
in arch/s390/include/asm/processor.h. Without the option it is possible
to trace __load_psw_mask which leads to kernel stack overflow.
Signed-off-by: Martin Schwidefsky
---
arch/s390/include/asm/processor.h | 4 ++--
include/linux/compiler-gcc.h | 12
include/linux/compiler.h | 2 +-
3 files changed, 3 insertions(+), 15 deletions(-)
diff --git a/arch/s390/include/asm/processor.h
b/arch/s390/include/asm/processor.h
index 302795c47c06..81038ab357ce 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -236,7 +236,7 @@ static inline unsigned long current_stack_pointer(void)
return sp;
}
-static __no_sanitize_address_or_inline unsigned short stap(void)
+static __no_kasan_or_inline unsigned short stap(void)
{
unsigned short cpu_address;
@@ -330,7 +330,7 @@ static inline void __load_psw(psw_t psw)
* Set PSW mask to specified value, while leaving the
* PSW addr pointing to the next instruction.
*/
-static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long mask)
+static __no_kasan_or_inline void __load_psw_mask(unsigned long mask)
{
unsigned long addr;
psw_t psw;
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index c0f5db3a9621..2010493e1040 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -143,18 +143,6 @@
#define KASAN_ABI_VERSION 3
#endif
-/*
- * Because __no_sanitize_address conflicts with inlining:
- * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
- * we do one or the other.
- */
-#ifdef CONFIG_KASAN
-#define __no_sanitize_address_or_inline
\
- __no_sanitize_address __maybe_unused notrace
-#else
-#define __no_sanitize_address_or_inline inline
-#endif
-
#if GCC_VERSION >= 50100
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 18c80cfa4fc4..06396c1cf127 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -189,7 +189,7 @@ void __read_once_size(const volatile void *p, void *res,
int size)
* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
* '__maybe_unused' allows us to avoid defined-but-not-used warnings.
*/
-# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
+# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused
#else
# define __no_kasan_or_inline __always_inline
#endif
--
2.16.4
--
blue skies,
Martin.
"Reality continues to ruin my life." - Calvin.
