[tip:x86/cleanups] lib: Remove redundant ftrace flag removal

[tip-bot for Mark Rutland]

Commit-ID:  41b57d1bb8a4084e651c1f9a754fca64952666a0
Gitweb: https://git.kernel.org/tip/41b57d1bb8a4084e651c1f9a754fca64952666a0
Author: Mark Rutland 
AuthorDate: Tue, 6 Aug 2019 17:25:39 +0100
Committer:  Borislav Petkov 
CommitDate: Wed, 14 Aug 2019 09:48:58 +0200

lib: Remove redundant ftrace flag removal

Since architectures can implement ftrace using a variety of mechanisms,
generic code should always use CC_FLAGS_FTRACE rather than assuming that
ftrace is built using -pg.

Since commit:

  2464a609ded09420 ("ftrace: do not trace library functions")

... lib/Makefile has removed CC_FLAGS_FTRACE from KBUILD_CFLAGS, so ftrace is
disabled for all files under lib/.

Given that, we shouldn't explicitly remove -pg when building
lib/string.o, as this is redundant and bad form.

Clean things up accordingly.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Borislav Petkov 
Cc: Andrew Morton 
Cc: Andy Shevchenko 
Cc: Ard Biesheuvel 
Cc: Arnd Bergmann 
Cc: Coly Li 
Cc: Gary R Hook 
Cc: Ingo Molnar 
Cc: Kees Cook 
Cc: Kent Overstreet 
Cc: Masahiro Yamada 
Cc: Matthew Wilcox 
Link: https://lkml.kernel.org/r/20190806162539.51918-1-mark.rutl...@arm.com
---
 lib/Makefile | 4 
 1 file changed, 4 deletions(-)

diff --git a/lib/Makefile b/lib/Makefile
index 095601ce371d..34f8a83b2cbd 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -21,10 +21,6 @@ KCOV_INSTRUMENT_dynamic_debug.o := n
 ifdef CONFIG_AMD_MEM_ENCRYPT
 KASAN_SANITIZE_string.o := n
 
-ifdef CONFIG_FUNCTION_TRACER
-CFLAGS_REMOVE_string.o = -pg
-endif
-
 CFLAGS_string.o := $(call cc-option, -fno-stack-protector)
 endif
 

[tip:locking/core] locking/atomic, s390/pci: Remove redundant casts

[tip-bot for Mark Rutland]

Commit-ID:  6a6a9d5fb9f26d2c2127497f3a42adbeb5ccc2a4
Gitweb: https://git.kernel.org/tip/6a6a9d5fb9f26d2c2127497f3a42adbeb5ccc2a4
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:50 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:57 +0200

locking/atomic, s390/pci: Remove redundant casts

Now that atomic64_read() returns s64 consistently, we don't need to
explicitly cast its return value. Drop the redundant casts.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Heiko Carstens 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-19-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/s390/pci/pci_debug.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/s390/pci/pci_debug.c b/arch/s390/pci/pci_debug.c
index 45eccf79e990..3408c0df3ebf 100644
--- a/arch/s390/pci/pci_debug.c
+++ b/arch/s390/pci/pci_debug.c
@@ -75,7 +75,7 @@ static void pci_sw_counter_show(struct seq_file *m)
 
for (i = 0; i < ARRAY_SIZE(pci_sw_names); i++, counter++)
seq_printf(m, "%26s:\t%llu\n", pci_sw_names[i],
-  (s64)atomic64_read(counter));
+  atomic64_read(counter));
 }
 
 static int pci_perf_show(struct seq_file *m, void *v)

[tip:locking/core] locking/atomic, crypto/nx: Remove redundant casts

[tip-bot for Mark Rutland]

Commit-ID:  2af7a0f91c3a645ec9f1cd68e41472021746db35
Gitweb: https://git.kernel.org/tip/2af7a0f91c3a645ec9f1cd68e41472021746db35
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:49 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:57 +0200

locking/atomic, crypto/nx: Remove redundant casts

Now that atomic64_read() returns s64 consistently, we don't need to
explicitly cast its return value. Drop the redundant casts.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Herbert Xu 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-18-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 drivers/crypto/nx/nx-842-pseries.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/nx/nx-842-pseries.c 
b/drivers/crypto/nx/nx-842-pseries.c
index 938332ce3b60..2de5e3672e42 100644
--- a/drivers/crypto/nx/nx-842-pseries.c
+++ b/drivers/crypto/nx/nx-842-pseries.c
@@ -857,7 +857,7 @@ static ssize_t nx842_##_name##_show(struct device *dev, 
\
local_devdata = rcu_dereference(devdata);   \
if (local_devdata)  \
p = snprintf(buf, PAGE_SIZE, "%lld\n",  \
-  (s64)atomic64_read(_devdata->counters->_name));
\
+  atomic64_read(_devdata->counters->_name)); \
rcu_read_unlock();  \
return p;   \
 }
@@ -911,7 +911,7 @@ static ssize_t nx842_timehist_show(struct device *dev,
for (i = 0; i < (NX842_HIST_SLOTS - 2); i++) {
bytes = snprintf(p, bytes_remain, "%u-%uus:\t%lld\n",
   i ? (2<<(i-1)) : 0, (2<

[tip:locking/core] locking/atomic: Use s64 for atomic64_t on 64-bit

[tip-bot for Mark Rutland]

Commit-ID:  3724921396dd1a07c93e3516b8d7c9ff570d17a9
Gitweb: https://git.kernel.org/tip/3724921396dd1a07c93e3516b8d7c9ff570d17a9
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:48 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:57 +0200

locking/atomic: Use s64 for atomic64_t on 64-bit

Now that all architectures use 64 consistently as the base type for the
atomic64 API, let's have the CONFIG_64BIT definition of atomic64_t use
s64 as the underlying type for atomic64_t, rather than long, matching
the generated headers.

On architectures where atomic64_read(v) is READ_ONCE(v->counter), this
patch will cause the return type of atomic64_read() to be s64.

As of this patch, the atomic64 API can be relied upon to consistently
return s64 where a value rather than boolean condition is returned. This
should make code more robust, and simpler, allowing for the removal of
casts previously required to ensure consistent types.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-17-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/types.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/types.h b/include/linux/types.h
index 231114ae38f4..05030f608be3 100644
--- a/include/linux/types.h
+++ b/include/linux/types.h
@@ -174,7 +174,7 @@ typedef struct {
 
 #ifdef CONFIG_64BIT
 typedef struct {
-   long counter;
+   s64 counter;
 } atomic64_t;
 #endif
 

[tip:locking/core] locking/atomic, x86: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  79c53a83d7a31a5b5c7bafce4f0723bebf26836a
Gitweb: https://git.kernel.org/tip/79c53a83d7a31a5b5c7bafce4f0723bebf26836a
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:47 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:57 +0200

locking/atomic, x86: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the x86 atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long or long long, matching the
generated headers.

Note that the x86 arch_atomic64 implementation is already wrapped by the
generic instrumented atomic64 implementation, which uses s64
consistently.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Borislav Petkov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Russell King 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-16-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/x86/include/asm/atomic64_32.h | 66 ++
 arch/x86/include/asm/atomic64_64.h | 38 +++---
 2 files changed, 51 insertions(+), 53 deletions(-)

diff --git a/arch/x86/include/asm/atomic64_32.h 
b/arch/x86/include/asm/atomic64_32.h
index 6a5b0ec460da..52cfaecb13f9 100644
--- a/arch/x86/include/asm/atomic64_32.h
+++ b/arch/x86/include/asm/atomic64_32.h
@@ -9,7 +9,7 @@
 /* An 64bit atomic type */
 
 typedef struct {
-   u64 __aligned(8) counter;
+   s64 __aligned(8) counter;
 } atomic64_t;
 
 #define ATOMIC64_INIT(val) { (val) }
@@ -71,8 +71,7 @@ ATOMIC64_DECL(add_unless);
  * the old value.
  */
 
-static inline long long arch_atomic64_cmpxchg(atomic64_t *v, long long o,
- long long n)
+static inline s64 arch_atomic64_cmpxchg(atomic64_t *v, s64 o, s64 n)
 {
return arch_cmpxchg64(>counter, o, n);
 }
@@ -85,9 +84,9 @@ static inline long long arch_atomic64_cmpxchg(atomic64_t *v, 
long long o,
  * Atomically xchgs the value of @v to @n and returns
  * the old value.
  */
-static inline long long arch_atomic64_xchg(atomic64_t *v, long long n)
+static inline s64 arch_atomic64_xchg(atomic64_t *v, s64 n)
 {
-   long long o;
+   s64 o;
unsigned high = (unsigned)(n >> 32);
unsigned low = (unsigned)n;
alternative_atomic64(xchg, "=" (o),
@@ -103,7 +102,7 @@ static inline long long arch_atomic64_xchg(atomic64_t *v, 
long long n)
  *
  * Atomically sets the value of @v to @n.
  */
-static inline void arch_atomic64_set(atomic64_t *v, long long i)
+static inline void arch_atomic64_set(atomic64_t *v, s64 i)
 {
unsigned high = (unsigned)(i >> 32);
unsigned low = (unsigned)i;
@@ -118,9 +117,9 @@ static inline void arch_atomic64_set(atomic64_t *v, long 
long i)
  *
  * Atomically reads the value of @v and returns it.
  */
-static inline long long arch_atomic64_read(const atomic64_t *v)
+static inline s64 arch_atomic64_read(const atomic64_t *v)
 {
-   long long r;
+   s64 r;
alternative_atomic64(read, "=" (r), "c" (v) : "memory");
return r;
 }
@@ -132,7 +131,7 @@ static inline long long arch_atomic64_read(const atomic64_t 
*v)
  *
  * Atomically adds @i to @v and returns @i + *@v
  */
-static inline long long arch_atomic64_add_return(long long i, atomic64_t *v)
+static inline s64 arch_atomic64_add_return(s64 i, atomic64_t *v)
 {
alternative_atomic64(add_return,
 ASM_OUTPUT2("+A" (i), "+c" (v)),
@@ -143,7 +142,7 @@ static inline long long arch_atomic64_add_return(long long 
i, atomic64_t *v)
 /*
  * Other variants with different arithmetic operators:
  */
-static inline long long arch_atomic64_sub_return(long long i, atomic64_t *v)
+static inline s64 arch_atomic64_sub_return(s64 i, atomic64_t *v)
 {
alternative_atomic64(sub_return,
 ASM_OUTPUT2("+A" (i), "+c" (v)),
@@ -151,18 +150,18 @@ static inline long long arch_atomic64_sub_return(long 
long i, atomic64_t *v)
return i;
 }
 
-static inline long long arch_atomic64_inc_return(atomic64_t *v)
+static inline s64 arch_atomic64_inc_return(atomic64_t *v)
 {
-   long long a;
+   s64 a;
alternative_atomic64(inc_return, "=" (a),
 "S" (v) : "memory", "ecx");
return a;
 }
 #define arch_atomic64_inc_return arch_atomic64_inc_return
 
-static inline long long arch_atomic64_dec_return(atomic64_t *v)

[tip:locking/core] locking/atomic, sparc: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  04e8851af767153c0878cc79ce30c0d8806eec43
Gitweb: https://git.kernel.org/tip/04e8851af767153c0878cc79ce30c0d8806eec43
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:46 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, sparc: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the sparc atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long, matching the generated headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long. This will be converted in a subsequent patch.

Otherwise, there should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: David S. Miller 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-15-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/sparc/include/asm/atomic_64.h | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/include/asm/atomic_64.h 
b/arch/sparc/include/asm/atomic_64.h
index 6963482c81d8..b60448397d4f 100644
--- a/arch/sparc/include/asm/atomic_64.h
+++ b/arch/sparc/include/asm/atomic_64.h
@@ -23,15 +23,15 @@
 
 #define ATOMIC_OP(op)  \
 void atomic_##op(int, atomic_t *); \
-void atomic64_##op(long, atomic64_t *);
+void atomic64_##op(s64, atomic64_t *);
 
 #define ATOMIC_OP_RETURN(op)   \
 int atomic_##op##_return(int, atomic_t *); \
-long atomic64_##op##_return(long, atomic64_t *);
+s64 atomic64_##op##_return(s64, atomic64_t *);
 
 #define ATOMIC_FETCH_OP(op)\
 int atomic_fetch_##op(int, atomic_t *);
\
-long atomic64_fetch_##op(long, atomic64_t *);
+s64 atomic64_fetch_##op(s64, atomic64_t *);
 
 #define ATOMIC_OPS(op) ATOMIC_OP(op) ATOMIC_OP_RETURN(op) ATOMIC_FETCH_OP(op)
 
@@ -61,7 +61,7 @@ static inline int atomic_xchg(atomic_t *v, int new)
((__typeof__((v)->counter))cmpxchg(&((v)->counter), (o), (n)))
 #define atomic64_xchg(v, new) (xchg(&((v)->counter), new))
 
-long atomic64_dec_if_positive(atomic64_t *v);
+s64 atomic64_dec_if_positive(atomic64_t *v);
 #define atomic64_dec_if_positive atomic64_dec_if_positive
 
 #endif /* !(__ARCH_SPARC64_ATOMIC__) */

[tip:locking/core] locking/atomic, s390: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  0ca94800762e8a2f7037c9b02ba74aff8016dd82
Gitweb: https://git.kernel.org/tip/0ca94800762e8a2f7037c9b02ba74aff8016dd82
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:45 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, s390: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the s390 atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long, matching the generated headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long. This will be converted in a subsequent patch.

The s390-internal __atomic64_*() ops are also used by the s390 bitops,
and expect pointers to long. Since atomic64_t::counter will be converted
to s64 in a subsequent patch, pointes to this are explicitly cast to
pointers to long when passed to __atomic64_*() ops.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Heiko Carstens 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-14-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/s390/include/asm/atomic.h | 38 +++---
 1 file changed, 19 insertions(+), 19 deletions(-)

diff --git a/arch/s390/include/asm/atomic.h b/arch/s390/include/asm/atomic.h
index fd20ab5d4cf7..491ad53a0d4e 100644
--- a/arch/s390/include/asm/atomic.h
+++ b/arch/s390/include/asm/atomic.h
@@ -84,9 +84,9 @@ static inline int atomic_cmpxchg(atomic_t *v, int old, int 
new)
 
 #define ATOMIC64_INIT(i)  { (i) }
 
-static inline long atomic64_read(const atomic64_t *v)
+static inline s64 atomic64_read(const atomic64_t *v)
 {
-   long c;
+   s64 c;
 
asm volatile(
"   lg  %0,%1\n"
@@ -94,49 +94,49 @@ static inline long atomic64_read(const atomic64_t *v)
return c;
 }
 
-static inline void atomic64_set(atomic64_t *v, long i)
+static inline void atomic64_set(atomic64_t *v, s64 i)
 {
asm volatile(
"   stg %1,%0\n"
: "=Q" (v->counter) : "d" (i));
 }
 
-static inline long atomic64_add_return(long i, atomic64_t *v)
+static inline s64 atomic64_add_return(s64 i, atomic64_t *v)
 {
-   return __atomic64_add_barrier(i, >counter) + i;
+   return __atomic64_add_barrier(i, (long *)>counter) + i;
 }
 
-static inline long atomic64_fetch_add(long i, atomic64_t *v)
+static inline s64 atomic64_fetch_add(s64 i, atomic64_t *v)
 {
-   return __atomic64_add_barrier(i, >counter);
+   return __atomic64_add_barrier(i, (long *)>counter);
 }
 
-static inline void atomic64_add(long i, atomic64_t *v)
+static inline void atomic64_add(s64 i, atomic64_t *v)
 {
 #ifdef CONFIG_HAVE_MARCH_Z196_FEATURES
if (__builtin_constant_p(i) && (i > -129) && (i < 128)) {
-   __atomic64_add_const(i, >counter);
+   __atomic64_add_const(i, (long *)>counter);
return;
}
 #endif
-   __atomic64_add(i, >counter);
+   __atomic64_add(i, (long *)>counter);
 }
 
 #define atomic64_xchg(v, new) (xchg(&((v)->counter), new))
 
-static inline long atomic64_cmpxchg(atomic64_t *v, long old, long new)
+static inline s64 atomic64_cmpxchg(atomic64_t *v, s64 old, s64 new)
 {
-   return __atomic64_cmpxchg(>counter, old, new);
+   return __atomic64_cmpxchg((long *)>counter, old, new);
 }
 
 #define ATOMIC64_OPS(op)   \
-static inline void atomic64_##op(long i, atomic64_t *v)
\
+static inline void atomic64_##op(s64 i, atomic64_t *v) \
 {  \
-   __atomic64_##op(i, >counter);\
+   __atomic64_##op(i, (long *)>counter);\
 }  \
-static inline long atomic64_fetch_##op(long i, atomic64_t *v)  \
+static inline long atomic64_fetch_##op(s64 i, atomic64_t *v)   \
 {  \
-   return __atomic64_##op##_barrier(i, >counter);   \
+   return __atomic64_##op##_barrier(i, (long *)>counter);   \
 }
 
 ATOMIC64_OPS(and)
@@ -145,8 +145,8 @@ ATOMIC64_OPS(xor)
 
 #undef ATOMIC64_OPS
 
-#define 

[tip:locking/core] locking/atomic, riscv: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  0754211847d7a228f1c34a49fd122979dfd19a1a
Gitweb: https://git.kernel.org/tip/0754211847d7a228f1c34a49fd122979dfd19a1a
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:44 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, riscv: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the RISC-V atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long, matching the generated headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long on 64-bit. This will be converted in a subsequent
patch.

Otherwise, there should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Palmer Dabbelt 
Cc: Albert Ou 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-13-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/riscv/include/asm/atomic.h | 44 +
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index 9c263bd9d5ad..96f95c9ebd97 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -38,11 +38,11 @@ static __always_inline void atomic_set(atomic_t *v, int i)
 
 #ifndef CONFIG_GENERIC_ATOMIC64
 #define ATOMIC64_INIT(i) { (i) }
-static __always_inline long atomic64_read(const atomic64_t *v)
+static __always_inline s64 atomic64_read(const atomic64_t *v)
 {
return READ_ONCE(v->counter);
 }
-static __always_inline void atomic64_set(atomic64_t *v, long i)
+static __always_inline void atomic64_set(atomic64_t *v, s64 i)
 {
WRITE_ONCE(v->counter, i);
 }
@@ -66,11 +66,11 @@ void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) 
\
 
 #ifdef CONFIG_GENERIC_ATOMIC64
 #define ATOMIC_OPS(op, asm_op, I)  \
-ATOMIC_OP (op, asm_op, I, w,  int,   )
+ATOMIC_OP (op, asm_op, I, w, int,   )
 #else
 #define ATOMIC_OPS(op, asm_op, I)  \
-ATOMIC_OP (op, asm_op, I, w,  int,   ) \
-ATOMIC_OP (op, asm_op, I, d, long, 64)
+ATOMIC_OP (op, asm_op, I, w, int,   )  \
+ATOMIC_OP (op, asm_op, I, d, s64, 64)
 #endif
 
 ATOMIC_OPS(add, add,  i)
@@ -127,14 +127,14 @@ c_type atomic##prefix##_##op##_return(c_type i, 
atomic##prefix##_t *v)\
 
 #ifdef CONFIG_GENERIC_ATOMIC64
 #define ATOMIC_OPS(op, asm_op, c_op, I)
\
-ATOMIC_FETCH_OP( op, asm_op,   I, w,  int,   ) \
-ATOMIC_OP_RETURN(op, asm_op, c_op, I, w,  int,   )
+ATOMIC_FETCH_OP( op, asm_op,   I, w, int,   )  \
+ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int,   )
 #else
 #define ATOMIC_OPS(op, asm_op, c_op, I)
\
-ATOMIC_FETCH_OP( op, asm_op,   I, w,  int,   ) \
-ATOMIC_OP_RETURN(op, asm_op, c_op, I, w,  int,   ) \
-ATOMIC_FETCH_OP( op, asm_op,   I, d, long, 64) \
-ATOMIC_OP_RETURN(op, asm_op, c_op, I, d, long, 64)
+ATOMIC_FETCH_OP( op, asm_op,   I, w, int,   )  \
+ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int,   )  \
+ATOMIC_FETCH_OP( op, asm_op,   I, d, s64, 64)  \
+ATOMIC_OP_RETURN(op, asm_op, c_op, I, d, s64, 64)
 #endif
 
 ATOMIC_OPS(add, add, +,  i)
@@ -166,11 +166,11 @@ ATOMIC_OPS(sub, add, +, -i)
 
 #ifdef CONFIG_GENERIC_ATOMIC64
 #define ATOMIC_OPS(op, asm_op, I)  \
-ATOMIC_FETCH_OP(op, asm_op, I, w,  int,   )
+ATOMIC_FETCH_OP(op, asm_op, I, w, int,   )
 #else
 #define ATOMIC_OPS(op, asm_op, I)  \
-ATOMIC_FETCH_OP(op, asm_op, I, w,  int,   )\
-ATOMIC_FETCH_OP(op, asm_op, I, d, long, 64)
+ATOMIC_FETCH_OP(op, asm_op, I, w, int,   ) \
+ATOMIC_FETCH_OP(op, asm_op, I, d, s64, 64)
 #endif
 
 ATOMIC_OPS(and, and, i)
@@ -219,9 +219,10 @@ static __always_inline int 
atomic_fetch_add_unless(atomic_t *v, int a, int u)
 #define atomic_fetch_add_unless atomic_fetch_add_unless
 
 #ifndef CONFIG_GENERIC_ATOMIC64
-static __always_inline long 

[tip:locking/core] locking/atomic, riscv: Fix atomic64_sub_if_positive() offset argument

[tip-bot for Mark Rutland]

Commit-ID:  33e42ef571979fe6601ac838d338eb599d842a6d
Gitweb: https://git.kernel.org/tip/33e42ef571979fe6601ac838d338eb599d842a6d
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:43 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, riscv: Fix atomic64_sub_if_positive() offset argument

Presently the riscv implementation of atomic64_sub_if_positive() takes
a 32-bit offset value rather than a 64-bit offset value as it should do.
Thus, if called with a 64-bit offset, the value will be unexpectedly
truncated to 32 bits.

Fix this by taking the offset as a long rather than an int.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Palmer Dabbelt 
Cc: Albert Ou 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-12-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/riscv/include/asm/atomic.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index 9038aeb900a6..9c263bd9d5ad 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -332,7 +332,7 @@ static __always_inline int atomic_sub_if_positive(atomic_t 
*v, int offset)
 #define atomic_dec_if_positive(v)  atomic_sub_if_positive(v, 1)
 
 #ifndef CONFIG_GENERIC_ATOMIC64
-static __always_inline long atomic64_sub_if_positive(atomic64_t *v, int offset)
+static __always_inline long atomic64_sub_if_positive(atomic64_t *v, long 
offset)
 {
long prev, rc;
 

[tip:locking/core] locking/atomic, powerpc: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  8cd8de59748ba71b476d1b7101f9ecaccd5cb8c2
Gitweb: https://git.kernel.org/tip/8cd8de59748ba71b476d1b7101f9ecaccd5cb8c2
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:42 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, powerpc: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the powerpc atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long, matching the generated headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long on 64-bit. This will be converted in a subsequent
patch.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Michael Ellerman 
Cc: Linus Torvalds 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-11-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/powerpc/include/asm/atomic.h | 44 +++
 1 file changed, 22 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/include/asm/atomic.h 
b/arch/powerpc/include/asm/atomic.h
index 52eafaf74054..31c231ea56b7 100644
--- a/arch/powerpc/include/asm/atomic.h
+++ b/arch/powerpc/include/asm/atomic.h
@@ -297,24 +297,24 @@ static __inline__ int atomic_dec_if_positive(atomic_t *v)
 
 #define ATOMIC64_INIT(i)   { (i) }
 
-static __inline__ long atomic64_read(const atomic64_t *v)
+static __inline__ s64 atomic64_read(const atomic64_t *v)
 {
-   long t;
+   s64 t;
 
__asm__ __volatile__("ld%U1%X1 %0,%1" : "=r"(t) : "m"(v->counter));
 
return t;
 }
 
-static __inline__ void atomic64_set(atomic64_t *v, long i)
+static __inline__ void atomic64_set(atomic64_t *v, s64 i)
 {
__asm__ __volatile__("std%U0%X0 %1,%0" : "=m"(v->counter) : "r"(i));
 }
 
 #define ATOMIC64_OP(op, asm_op)
\
-static __inline__ void atomic64_##op(long a, atomic64_t *v)\
+static __inline__ void atomic64_##op(s64 a, atomic64_t *v) \
 {  \
-   long t; \
+   s64 t;  \
\
__asm__ __volatile__(   \
 "1:ldarx   %0,0,%3 # atomic64_" #op "\n"   \
@@ -327,10 +327,10 @@ static __inline__ void atomic64_##op(long a, atomic64_t 
*v)   \
 }
 
 #define ATOMIC64_OP_RETURN_RELAXED(op, asm_op) \
-static inline long \
-atomic64_##op##_return_relaxed(long a, atomic64_t *v)  \
+static inline s64  \
+atomic64_##op##_return_relaxed(s64 a, atomic64_t *v)   \
 {  \
-   long t; \
+   s64 t;  \
\
__asm__ __volatile__(   \
 "1:ldarx   %0,0,%3 # atomic64_" #op "_return_relaxed\n"\
@@ -345,10 +345,10 @@ atomic64_##op##_return_relaxed(long a, atomic64_t *v) 
\
 }
 
 #define ATOMIC64_FETCH_OP_RELAXED(op, asm_op)  \
-static inline long \
-atomic64_fetch_##op##_relaxed(long a, atomic64_t *v)   \
+static inline s64  \
+atomic64_fetch_##op##_relaxed(s64 a, atomic64_t *v)\
 {  \
-   long res, t;\
+   s64 res, t; \
\
__asm__ __volatile__(   \
 "1:ldarx   %0,0,%4 # atomic64_fetch_" #op "_relaxed\n" \
@@ -396,7 +396,7 @@ ATOMIC64_OPS(xor, xor)
 
 static 

[tip:locking/core] locking/atomic, mips: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  d184cf1a449ca4cb0d86f3dd82c3337c645ea6c0
Gitweb: https://git.kernel.org/tip/d184cf1a449ca4cb0d86f3dd82c3337c645ea6c0
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:41 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, mips: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the mips atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long or __s64, matching the generated
headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long on 64-bit. This will be converted in a subsequent
patch.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: James Hogan 
Cc: Linus Torvalds 
Cc: Paul Burton 
Cc: Peter Zijlstra 
Cc: Ralf Baechle 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: pau...@samba.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-10-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/mips/include/asm/atomic.h | 22 +++---
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h
index 94096299fc56..9a82dd11c0e9 100644
--- a/arch/mips/include/asm/atomic.h
+++ b/arch/mips/include/asm/atomic.h
@@ -254,10 +254,10 @@ static __inline__ int atomic_sub_if_positive(int i, 
atomic_t * v)
 #define atomic64_set(v, i) WRITE_ONCE((v)->counter, (i))
 
 #define ATOMIC64_OP(op, c_op, asm_op)\
-static __inline__ void atomic64_##op(long i, atomic64_t * v) \
+static __inline__ void atomic64_##op(s64 i, atomic64_t * v)  \
 {\
if (kernel_uses_llsc) {   \
-   long temp;\
+   s64 temp; \
  \
loongson_llsc_mb();   \
__asm__ __volatile__( \
@@ -280,12 +280,12 @@ static __inline__ void atomic64_##op(long i, atomic64_t * 
v)\
 }
 
 #define ATOMIC64_OP_RETURN(op, c_op, asm_op) \
-static __inline__ long atomic64_##op##_return_relaxed(long i, atomic64_t * v) \
+static __inline__ s64 atomic64_##op##_return_relaxed(s64 i, atomic64_t * v)   \
 {\
-   long result;  \
+   s64 result;   \
  \
if (kernel_uses_llsc) {   \
-   long temp;\
+   s64 temp; \
  \
loongson_llsc_mb();   \
__asm__ __volatile__( \
@@ -314,12 +314,12 @@ static __inline__ long 
atomic64_##op##_return_relaxed(long i, atomic64_t * v) \
 }
 
 #define ATOMIC64_FETCH_OP(op, c_op, asm_op)  \
-static __inline__ long atomic64_fetch_##op##_relaxed(long i, atomic64_t * v)  \
+static __inline__ s64 atomic64_fetch_##op##_relaxed(s64 i, atomic64_t * v)\
 {\
-   long result;  \
+   s64 result;   \
  \
if (kernel_uses_llsc) {   \
-   long temp;\
+   s64 temp; \
  \
loongson_llsc_mb();   \
__asm__ 

[tip:locking/core] locking/atomic, ia64: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  d84e28d250150adc6526dcce4ca089e2b57430f3
Gitweb: https://git.kernel.org/tip/d84e28d250150adc6526dcce4ca089e2b57430f3
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:40 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, ia64: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the ia64 atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long or __s64, matching the generated
headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long. This will be converted in a subsequent patch.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Fenghua Yu 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-9-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/ia64/include/asm/atomic.h | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/ia64/include/asm/atomic.h b/arch/ia64/include/asm/atomic.h
index 206530d0751b..50440f3ddc43 100644
--- a/arch/ia64/include/asm/atomic.h
+++ b/arch/ia64/include/asm/atomic.h
@@ -124,10 +124,10 @@ ATOMIC_FETCH_OP(xor, ^)
 #undef ATOMIC_OP
 
 #define ATOMIC64_OP(op, c_op)  \
-static __inline__ long \
-ia64_atomic64_##op (__s64 i, atomic64_t *v)\
+static __inline__ s64  \
+ia64_atomic64_##op (s64 i, atomic64_t *v)  \
 {  \
-   __s64 old, new; \
+   s64 old, new;   \
CMPXCHG_BUGCHECK_DECL   \
\
do {\
@@ -139,10 +139,10 @@ ia64_atomic64_##op (__s64 i, atomic64_t *v)   
\
 }
 
 #define ATOMIC64_FETCH_OP(op, c_op)\
-static __inline__ long \
-ia64_atomic64_fetch_##op (__s64 i, atomic64_t *v)  \
+static __inline__ s64  \
+ia64_atomic64_fetch_##op (s64 i, atomic64_t *v)
\
 {  \
-   __s64 old, new; \
+   s64 old, new;   \
CMPXCHG_BUGCHECK_DECL   \
\
do {\
@@ -162,7 +162,7 @@ ATOMIC64_OPS(sub, -)
 
 #define atomic64_add_return(i,v)   \
 ({ \
-   long __ia64_aar_i = (i);\
+   s64 __ia64_aar_i = (i); \
__ia64_atomic_const(i)  \
? ia64_fetch_and_add(__ia64_aar_i, &(v)->counter)   \
: ia64_atomic64_add(__ia64_aar_i, v);   \
@@ -170,7 +170,7 @@ ATOMIC64_OPS(sub, -)
 
 #define atomic64_sub_return(i,v)   \
 ({ \
-   long __ia64_asr_i = (i);\
+   s64 __ia64_asr_i = (i); \
__ia64_atomic_const(i)  \
? ia64_fetch_and_add(-__ia64_asr_i, &(v)->counter)  \
: ia64_atomic64_sub(__ia64_asr_i, v);   \
@@ -178,7 +178,7 @@ ATOMIC64_OPS(sub, -)
 
 #define atomic64_fetch_add(i,v)
\
 ({ \
-   long __ia64_aar_i = (i);\
+   s64 

[tip:locking/core] locking/atomic, arm64: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  16f18688af7ea6c65f6daa3efb4661415e2e6041
Gitweb: https://git.kernel.org/tip/16f18688af7ea6c65f6daa3efb4661415e2e6041
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:39 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, arm64: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the arm64 atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long, matching the generated headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long. This will be converted in a subsequent patch.

Note that in arch_atomic64_dec_if_positive(), the x0 variable is left as
long, as this variable is also used to hold the pointer to the
atomic64_t.

Otherwise, there should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Catalin Marinas 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-8-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/arm64/include/asm/atomic_ll_sc.h | 20 ++--
 arch/arm64/include/asm/atomic_lse.h   | 34 +-
 2 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/arch/arm64/include/asm/atomic_ll_sc.h 
b/arch/arm64/include/asm/atomic_ll_sc.h
index e321293e0c89..f3b12d7f431f 100644
--- a/arch/arm64/include/asm/atomic_ll_sc.h
+++ b/arch/arm64/include/asm/atomic_ll_sc.h
@@ -133,9 +133,9 @@ ATOMIC_OPS(xor, eor)
 
 #define ATOMIC64_OP(op, asm_op)
\
 __LL_SC_INLINE void\
-__LL_SC_PREFIX(arch_atomic64_##op(long i, atomic64_t *v))  \
+__LL_SC_PREFIX(arch_atomic64_##op(s64 i, atomic64_t *v))   \
 {  \
-   long result;\
+   s64 result; \
unsigned long tmp;  \
\
asm volatile("// atomic64_" #op "\n"\
@@ -150,10 +150,10 @@ __LL_SC_PREFIX(arch_atomic64_##op(long i, atomic64_t *v)) 
\
 __LL_SC_EXPORT(arch_atomic64_##op);
 
 #define ATOMIC64_OP_RETURN(name, mb, acq, rel, cl, op, asm_op) \
-__LL_SC_INLINE long\
-__LL_SC_PREFIX(arch_atomic64_##op##_return##name(long i, atomic64_t *v))\
+__LL_SC_INLINE s64 \
+__LL_SC_PREFIX(arch_atomic64_##op##_return##name(s64 i, atomic64_t *v))\
 {  \
-   long result;\
+   s64 result; \
unsigned long tmp;  \
\
asm volatile("// atomic64_" #op "_return" #name "\n"\
@@ -172,10 +172,10 @@ __LL_SC_PREFIX(arch_atomic64_##op##_return##name(long i, 
atomic64_t *v))\
 __LL_SC_EXPORT(arch_atomic64_##op##_return##name);
 
 #define ATOMIC64_FETCH_OP(name, mb, acq, rel, cl, op, asm_op)  \
-__LL_SC_INLINE long\
-__LL_SC_PREFIX(arch_atomic64_fetch_##op##name(long i, atomic64_t *v))  \
+__LL_SC_INLINE s64 \
+__LL_SC_PREFIX(arch_atomic64_fetch_##op##name(s64 i, atomic64_t *v))   \
 {  \
-   long result, val;   \
+   s64 result, val;\
unsigned long tmp;  \
\
asm volatile("// atomic64_fetch_" #op #name "\n"\
@@ -225,10 +225,10 @@ ATOMIC64_OPS(xor, eor)
 #undef ATOMIC64_OP_RETURN
 #undef ATOMIC64_OP
 
-__LL_SC_INLINE long
+__LL_SC_INLINE s64
 __LL_SC_PREFIX(arch_atomic64_dec_if_positive(atomic64_t *v))
 {
-   long result;
+   s64 result;
  

[tip:locking/core] locking/atomic, arm: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  ef4cdc09260e2b0576423ca708e245e7549aa8e0
Gitweb: https://git.kernel.org/tip/ef4cdc09260e2b0576423ca708e245e7549aa8e0
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:38 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, arm: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the arm atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long long, matching the generated
headers.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Russell King 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-7-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/arm/include/asm/atomic.h | 50 +--
 1 file changed, 24 insertions(+), 26 deletions(-)

diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h
index f74756641410..d45c41f6f69c 100644
--- a/arch/arm/include/asm/atomic.h
+++ b/arch/arm/include/asm/atomic.h
@@ -249,15 +249,15 @@ ATOMIC_OPS(xor, ^=, eor)
 
 #ifndef CONFIG_GENERIC_ATOMIC64
 typedef struct {
-   long long counter;
+   s64 counter;
 } atomic64_t;
 
 #define ATOMIC64_INIT(i) { (i) }
 
 #ifdef CONFIG_ARM_LPAE
-static inline long long atomic64_read(const atomic64_t *v)
+static inline s64 atomic64_read(const atomic64_t *v)
 {
-   long long result;
+   s64 result;
 
__asm__ __volatile__("@ atomic64_read\n"
 "  ldrd%0, %H0, [%1]"
@@ -268,7 +268,7 @@ static inline long long atomic64_read(const atomic64_t *v)
return result;
 }
 
-static inline void atomic64_set(atomic64_t *v, long long i)
+static inline void atomic64_set(atomic64_t *v, s64 i)
 {
__asm__ __volatile__("@ atomic64_set\n"
 "  strd%2, %H2, [%1]"
@@ -277,9 +277,9 @@ static inline void atomic64_set(atomic64_t *v, long long i)
);
 }
 #else
-static inline long long atomic64_read(const atomic64_t *v)
+static inline s64 atomic64_read(const atomic64_t *v)
 {
-   long long result;
+   s64 result;
 
__asm__ __volatile__("@ atomic64_read\n"
 "  ldrexd  %0, %H0, [%1]"
@@ -290,9 +290,9 @@ static inline long long atomic64_read(const atomic64_t *v)
return result;
 }
 
-static inline void atomic64_set(atomic64_t *v, long long i)
+static inline void atomic64_set(atomic64_t *v, s64 i)
 {
-   long long tmp;
+   s64 tmp;
 
prefetchw(>counter);
__asm__ __volatile__("@ atomic64_set\n"
@@ -307,9 +307,9 @@ static inline void atomic64_set(atomic64_t *v, long long i)
 #endif
 
 #define ATOMIC64_OP(op, op1, op2)  \
-static inline void atomic64_##op(long long i, atomic64_t *v)   \
+static inline void atomic64_##op(s64 i, atomic64_t *v) \
 {  \
-   long long result;   \
+   s64 result; \
unsigned long tmp;  \
\
prefetchw(>counter); \
@@ -326,10 +326,10 @@ static inline void atomic64_##op(long long i, atomic64_t 
*v)  \
 }  \
 
 #define ATOMIC64_OP_RETURN(op, op1, op2)   \
-static inline long long
\
-atomic64_##op##_return_relaxed(long long i, atomic64_t *v) \
+static inline s64  \
+atomic64_##op##_return_relaxed(s64 i, atomic64_t *v)   \
 {  \
-   long long result;   \
+   s64 result; \
unsigned long tmp;  \
\
prefetchw(>counter); \
@@ -349,10 +349,10 @@ atomic64_##op##_return_relaxed(long long i, atomic64_t 
*v)\
 }
 
 #define 

[tip:locking/core] locking/atomic, arc: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  16fbad086976574b99ea7019c0504d0194e95dc3
Gitweb: https://git.kernel.org/tip/16fbad086976574b99ea7019c0504d0194e95dc3
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:37 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, arc: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the arc atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than u64, matching the generated headers.

Otherwise, there should be no functional change as a result of this
patch.

Acked-By: Vineet Gupta 
Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Link: https://lkml.kernel.org/r/20190522132250.26499-6-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/arc/include/asm/atomic.h | 41 -
 1 file changed, 20 insertions(+), 21 deletions(-)

diff --git a/arch/arc/include/asm/atomic.h b/arch/arc/include/asm/atomic.h
index 158af079838d..2c75df55d0d2 100644
--- a/arch/arc/include/asm/atomic.h
+++ b/arch/arc/include/asm/atomic.h
@@ -324,14 +324,14 @@ ATOMIC_OPS(xor, ^=, CTOP_INST_AXOR_DI_R2_R2_R3)
  */
 
 typedef struct {
-   aligned_u64 counter;
+   s64 __aligned(8) counter;
 } atomic64_t;
 
 #define ATOMIC64_INIT(a) { (a) }
 
-static inline long long atomic64_read(const atomic64_t *v)
+static inline s64 atomic64_read(const atomic64_t *v)
 {
-   unsigned long long val;
+   s64 val;
 
__asm__ __volatile__(
"   ldd   %0, [%1]  \n"
@@ -341,7 +341,7 @@ static inline long long atomic64_read(const atomic64_t *v)
return val;
 }
 
-static inline void atomic64_set(atomic64_t *v, long long a)
+static inline void atomic64_set(atomic64_t *v, s64 a)
 {
/*
 * This could have been a simple assignment in "C" but would need
@@ -362,9 +362,9 @@ static inline void atomic64_set(atomic64_t *v, long long a)
 }
 
 #define ATOMIC64_OP(op, op1, op2)  \
-static inline void atomic64_##op(long long a, atomic64_t *v)   \
+static inline void atomic64_##op(s64 a, atomic64_t *v) \
 {  \
-   unsigned long long val; \
+   s64 val;\
\
__asm__ __volatile__(   \
"1: \n" \
@@ -375,13 +375,13 @@ static inline void atomic64_##op(long long a, atomic64_t 
*v)  \
"   bnz 1b  \n" \
: "="(val)\
: "r"(>counter), "ir"(a) \
-   : "cc");\
+   : "cc");\
 }  \
 
 #define ATOMIC64_OP_RETURN(op, op1, op2)   \
-static inline long long atomic64_##op##_return(long long a, atomic64_t *v) 
\
+static inline s64 atomic64_##op##_return(s64 a, atomic64_t *v) \
 {  \
-   unsigned long long val; \
+   s64 val;\
\
smp_mb();   \
\
@@ -402,9 +402,9 @@ static inline long long atomic64_##op##_return(long long a, 
atomic64_t *v)  \
 }
 
 #define ATOMIC64_FETCH_OP(op, op1, op2)
\
-static inline long long atomic64_fetch_##op(long long a, atomic64_t *v)
\
+static inline s64 atomic64_fetch_##op(s64 a, atomic64_t *v)\
 {  \
-   unsigned long long val, orig;   \
+   s64 val, orig;  \
  

[tip:locking/core] locking/atomic, alpha: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  0203fdc160a8c8d8651a3b79aa453ec36cfbd867
Gitweb: https://git.kernel.org/tip/0203fdc160a8c8d8651a3b79aa453ec36cfbd867
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:36 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, alpha: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the alpha atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long, matching the generated headers.

As atomic64_read() depends on the generic defintion of atomic64_t, this
still returns long. This will be converted in a subsequent patch.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Ivan Kokshaysky 
Cc: Linus Torvalds 
Cc: Matt Turner 
Cc: Peter Zijlstra 
Cc: Richard Henderson 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/alpha/include/asm/atomic.h | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/alpha/include/asm/atomic.h b/arch/alpha/include/asm/atomic.h
index 150a1c5d6a2c..2144530d1428 100644
--- a/arch/alpha/include/asm/atomic.h
+++ b/arch/alpha/include/asm/atomic.h
@@ -93,9 +93,9 @@ static inline int atomic_fetch_##op##_relaxed(int i, atomic_t 
*v) \
 }
 
 #define ATOMIC64_OP(op, asm_op)
\
-static __inline__ void atomic64_##op(long i, atomic64_t * v)   \
+static __inline__ void atomic64_##op(s64 i, atomic64_t * v)\
 {  \
-   unsigned long temp; \
+   s64 temp;   \
__asm__ __volatile__(   \
"1: ldq_l %0,%1\n"  \
"   " #asm_op " %0,%2,%0\n" \
@@ -109,9 +109,9 @@ static __inline__ void atomic64_##op(long i, atomic64_t * 
v)\
 }  \
 
 #define ATOMIC64_OP_RETURN(op, asm_op) \
-static __inline__ long atomic64_##op##_return_relaxed(long i, atomic64_t * v)  
\
+static __inline__ s64 atomic64_##op##_return_relaxed(s64 i, atomic64_t * v)
\
 {  \
-   long temp, result;  \
+   s64 temp, result;   \
__asm__ __volatile__(   \
"1: ldq_l %0,%1\n"  \
"   " #asm_op " %0,%3,%2\n" \
@@ -128,9 +128,9 @@ static __inline__ long atomic64_##op##_return_relaxed(long 
i, atomic64_t * v)   \
 }
 
 #define ATOMIC64_FETCH_OP(op, asm_op)  \
-static __inline__ long atomic64_fetch_##op##_relaxed(long i, atomic64_t * v)   
\
+static __inline__ s64 atomic64_fetch_##op##_relaxed(s64 i, atomic64_t * v) 
\
 {  \
-   long temp, result;  \
+   s64 temp, result;   \
__asm__ __volatile__(   \
"1: ldq_l %2,%1\n"  \
"   " #asm_op " %2,%3,%0\n" \
@@ -246,9 +246,9 @@ static __inline__ int atomic_fetch_add_unless(atomic_t *v, 
int a, int u)
  * Atomically adds @a to @v, so long as it was not @u.
  * Returns the old value of @v.
  */
-static __inline__ long atomic64_fetch_add_unless(atomic64_t *v, long a, long u)
+static __inline__ s64 atomic64_fetch_add_unless(atomic64_t *v, s64 a, s64 u)
 {
-   long c, new, old;
+   s64 c, new, old;
smp_mb();
__asm__ __volatile__(
"1: ldq_l   %[old],%[mem]\n"
@@ -276,9 +276,9 @@ static __inline__ long atomic64_fetch_add_unless(atomic64_t 
*v, long a, long u)
  * The function returns the old value of *v minus 1, even if
  * the atomic variable, v, was not decremented.
  */
-static inline long atomic64_dec_if_positive(atomic64_t *v)
+static inline s64 

[tip:locking/core] locking/atomic, s390/pci: Prepare for atomic64_read() conversion

[tip-bot for Mark Rutland]

Commit-ID:  982164d62a4b2097c0db28ae7c31fc905af26bb8
Gitweb: https://git.kernel.org/tip/982164d62a4b2097c0db28ae7c31fc905af26bb8
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:34 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, s390/pci: Prepare for atomic64_read() conversion

The return type of atomic64_read() varies by architecture. It may return
long (e.g. powerpc), long long (e.g. arm), or s64 (e.g. x86_64). This is
somewhat painful, and mandates the use of explicit casts in some cases
(e.g. when printing the return value).

To ameliorate matters, subsequent patches will make the atomic64 API
consistently use s64.

As a preparatory step, this patch updates the s390 pci debug code to
treat the return value of atomic64_read() as s64, using an explicit
cast. This cast will be removed once the s64 conversion is complete.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Heiko Carstens 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/s390/pci/pci_debug.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/s390/pci/pci_debug.c b/arch/s390/pci/pci_debug.c
index 6b48ca7760a7..45eccf79e990 100644
--- a/arch/s390/pci/pci_debug.c
+++ b/arch/s390/pci/pci_debug.c
@@ -74,8 +74,8 @@ static void pci_sw_counter_show(struct seq_file *m)
int i;
 
for (i = 0; i < ARRAY_SIZE(pci_sw_names); i++, counter++)
-   seq_printf(m, "%26s:\t%lu\n", pci_sw_names[i],
-  atomic64_read(counter));
+   seq_printf(m, "%26s:\t%llu\n", pci_sw_names[i],
+  (s64)atomic64_read(counter));
 }
 
 static int pci_perf_show(struct seq_file *m, void *v)

[tip:locking/core] locking/atomic: Use s64 for atomic64

[tip-bot for Mark Rutland]

Commit-ID:  9255813d5841e158f033e0d83d455bffdae009a4
Gitweb: https://git.kernel.org/tip/9255813d5841e158f033e0d83d455bffdae009a4
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:35 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic: Use s64 for atomic64

As a step towards making the atomic64 API use consistent types treewide,
let's have the generic atomic64 implementation use s64 as the underlying
type for atomic64_t, rather than long long, matching the generated
headers.

Otherwise, there should be no functional change as a result of this
patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Arnd Bergmann 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: herb...@gondor.apana.org.au
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic64.h | 20 ++--
 lib/atomic64.c | 32 
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/include/asm-generic/atomic64.h b/include/asm-generic/atomic64.h
index d7a15096fb3b..370f01d4450f 100644
--- a/include/asm-generic/atomic64.h
+++ b/include/asm-generic/atomic64.h
@@ -10,24 +10,24 @@
 #include 
 
 typedef struct {
-   long long counter;
+   s64 counter;
 } atomic64_t;
 
 #define ATOMIC64_INIT(i)   { (i) }
 
-extern long long atomic64_read(const atomic64_t *v);
-extern void atomic64_set(atomic64_t *v, long long i);
+extern s64 atomic64_read(const atomic64_t *v);
+extern void atomic64_set(atomic64_t *v, s64 i);
 
 #define atomic64_set_release(v, i) atomic64_set((v), (i))
 
 #define ATOMIC64_OP(op)
\
-extern void atomic64_##op(long long a, atomic64_t *v);
+extern void atomic64_##op(s64 a, atomic64_t *v);
 
 #define ATOMIC64_OP_RETURN(op) \
-extern long long atomic64_##op##_return(long long a, atomic64_t *v);
+extern s64 atomic64_##op##_return(s64 a, atomic64_t *v);
 
 #define ATOMIC64_FETCH_OP(op)  \
-extern long long atomic64_fetch_##op(long long a, atomic64_t *v);
+extern s64 atomic64_fetch_##op(s64 a, atomic64_t *v);
 
 #define ATOMIC64_OPS(op)   ATOMIC64_OP(op) ATOMIC64_OP_RETURN(op) 
ATOMIC64_FETCH_OP(op)
 
@@ -46,11 +46,11 @@ ATOMIC64_OPS(xor)
 #undef ATOMIC64_OP_RETURN
 #undef ATOMIC64_OP
 
-extern long long atomic64_dec_if_positive(atomic64_t *v);
+extern s64 atomic64_dec_if_positive(atomic64_t *v);
 #define atomic64_dec_if_positive atomic64_dec_if_positive
-extern long long atomic64_cmpxchg(atomic64_t *v, long long o, long long n);
-extern long long atomic64_xchg(atomic64_t *v, long long new);
-extern long long atomic64_fetch_add_unless(atomic64_t *v, long long a, long 
long u);
+extern s64 atomic64_cmpxchg(atomic64_t *v, s64 o, s64 n);
+extern s64 atomic64_xchg(atomic64_t *v, s64 new);
+extern s64 atomic64_fetch_add_unless(atomic64_t *v, s64 a, s64 u);
 #define atomic64_fetch_add_unless atomic64_fetch_add_unless
 
 #endif  /*  _ASM_GENERIC_ATOMIC64_H  */
diff --git a/lib/atomic64.c b/lib/atomic64.c
index 7e6905751522..e98c85a99787 100644
--- a/lib/atomic64.c
+++ b/lib/atomic64.c
@@ -42,11 +42,11 @@ static inline raw_spinlock_t *lock_addr(const atomic64_t *v)
return _lock[addr & (NR_LOCKS - 1)].lock;
 }
 
-long long atomic64_read(const atomic64_t *v)
+s64 atomic64_read(const atomic64_t *v)
 {
unsigned long flags;
raw_spinlock_t *lock = lock_addr(v);
-   long long val;
+   s64 val;
 
raw_spin_lock_irqsave(lock, flags);
val = v->counter;
@@ -55,7 +55,7 @@ long long atomic64_read(const atomic64_t *v)
 }
 EXPORT_SYMBOL(atomic64_read);
 
-void atomic64_set(atomic64_t *v, long long i)
+void atomic64_set(atomic64_t *v, s64 i)
 {
unsigned long flags;
raw_spinlock_t *lock = lock_addr(v);
@@ -67,7 +67,7 @@ void atomic64_set(atomic64_t *v, long long i)
 EXPORT_SYMBOL(atomic64_set);
 
 #define ATOMIC64_OP(op, c_op)  \
-void atomic64_##op(long long a, atomic64_t *v) \
+void atomic64_##op(s64 a, atomic64_t *v)   \
 {  \
unsigned long flags;\
raw_spinlock_t *lock = lock_addr(v);\
@@ -79,11 +79,11 @@ void 

[tip:locking/core] locking/atomic, crypto/nx: Prepare for atomic64_read() conversion

[tip-bot for Mark Rutland]

Commit-ID:  90fde663aed0a1c27e50dd1bf3f121141b2fe9f2
Gitweb: https://git.kernel.org/tip/90fde663aed0a1c27e50dd1bf3f121141b2fe9f2
Author: Mark Rutland 
AuthorDate: Wed, 22 May 2019 14:22:33 +0100
Committer:  Ingo Molnar 
CommitDate: Mon, 3 Jun 2019 12:32:56 +0200

locking/atomic, crypto/nx: Prepare for atomic64_read() conversion

The return type of atomic64_read() varies by architecture. It may return
long (e.g. powerpc), long long (e.g. arm), or s64 (e.g. x86_64). This is
somewhat painful, and mandates the use of explicit casts in some cases
(e.g. when printing the return value).

To ameliorate matters, subsequent patches will make the atomic64 API
consistently use s64.

As a preparatory step, this patch updates the nx-842 code to treat the
return value of atomic64_read() as s64, using explicit casts. These
casts will be removed once the s64 conversion is complete.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Herbert Xu 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: a...@eecs.berkeley.edu
Cc: a...@arndb.de
Cc: b...@alien8.de
Cc: catalin.mari...@arm.com
Cc: da...@davemloft.net
Cc: fenghua...@intel.com
Cc: heiko.carst...@de.ibm.com
Cc: i...@jurassic.park.msu.ru
Cc: jho...@kernel.org
Cc: li...@armlinux.org.uk
Cc: matts...@gmail.com
Cc: m...@ellerman.id.au
Cc: pal...@sifive.com
Cc: paul.bur...@mips.com
Cc: pau...@samba.org
Cc: r...@linux-mips.org
Cc: r...@twiddle.net
Cc: tony.l...@intel.com
Cc: vgu...@synopsys.com
Link: https://lkml.kernel.org/r/20190522132250.26499-2-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 drivers/crypto/nx/nx-842-pseries.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/crypto/nx/nx-842-pseries.c 
b/drivers/crypto/nx/nx-842-pseries.c
index 5c4aa606208c..938332ce3b60 100644
--- a/drivers/crypto/nx/nx-842-pseries.c
+++ b/drivers/crypto/nx/nx-842-pseries.c
@@ -856,8 +856,8 @@ static ssize_t nx842_##_name##_show(struct device *dev, 
\
rcu_read_lock();\
local_devdata = rcu_dereference(devdata);   \
if (local_devdata)  \
-   p = snprintf(buf, PAGE_SIZE, "%ld\n",   \
-  atomic64_read(_devdata->counters->_name)); \
+   p = snprintf(buf, PAGE_SIZE, "%lld\n",  \
+  (s64)atomic64_read(_devdata->counters->_name));
\
rcu_read_unlock();  \
return p;   \
 }
@@ -909,17 +909,17 @@ static ssize_t nx842_timehist_show(struct device *dev,
}
 
for (i = 0; i < (NX842_HIST_SLOTS - 2); i++) {
-   bytes = snprintf(p, bytes_remain, "%u-%uus:\t%ld\n",
+   bytes = snprintf(p, bytes_remain, "%u-%uus:\t%lld\n",
   i ? (2<<(i-1)) : 0, (2<

[tip:perf/core] perf/core: Don't WARN() for impossible ring-buffer sizes

[tip-bot for Mark Rutland]

Commit-ID:  9dff0aa95a324e262ffb03f425d00e4751f3294e
Gitweb: https://git.kernel.org/tip/9dff0aa95a324e262ffb03f425d00e4751f3294e
Author: Mark Rutland 
AuthorDate: Thu, 10 Jan 2019 14:27:45 +
Committer:  Ingo Molnar 
CommitDate: Mon, 4 Feb 2019 08:45:25 +0100

perf/core: Don't WARN() for impossible ring-buffer sizes

The perf tool uses /proc/sys/kernel/perf_event_mlock_kb to determine how
large its ringbuffer mmap should be. This can be configured to arbitrary
values, which can be larger than the maximum possible allocation from
kmalloc.

When this is configured to a suitably large value (e.g. thanks to the
perf fuzzer), attempting to use perf record triggers a WARN_ON_ONCE() in
__alloc_pages_nodemask():

   WARNING: CPU: 2 PID: 5666 at mm/page_alloc.c:4511 
__alloc_pages_nodemask+0x3f8/0xbc8

Let's avoid this by checking that the requested allocation is possible
before calling kzalloc.

Reported-by: Julien Thierry 
Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Julien Thierry 
Cc: Alexander Shishkin 
Cc: Arnaldo Carvalho de Melo 
Cc: Jiri Olsa 
Cc: Linus Torvalds 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: 
Link: https://lkml.kernel.org/r/20190110142745.25495-1-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 kernel/events/ring_buffer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 4a9937076331..309ef5a64af5 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -734,6 +734,9 @@ struct ring_buffer *rb_alloc(int nr_pages, long watermark, 
int cpu, int flags)
size = sizeof(struct ring_buffer);
size += nr_pages * sizeof(void *);
 
+   if (order_base_2(size) >= MAX_ORDER)
+   goto fail;
+
rb = kzalloc(size, GFP_KERNEL);
if (!rb)
goto fail;

[tip:locking/core] arm64, locking/atomics: Use instrumented atomics

[tip-bot for Mark Rutland]

Commit-ID:  c0df10812835040e261b915f04887b0cf0411851
Gitweb: https://git.kernel.org/tip/c0df10812835040e261b915f04887b0cf0411851
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:30 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:01:40 +0100

arm64, locking/atomics: Use instrumented atomics

Now that the generic atomic headers provide instrumented wrappers of all
the atomics implemented by arm64, let's migrate arm64 over to these.

The additional instrumentation will help to find bugs (e.g. when fuzzing
with Syzkaller).

Mostly this change involves adding an arch_ prefix to a number of
function names and macro definitions. When LSE atomics are used, the
out-of-line LL/SC atomics will be named __ll_sc_arch_atomic_${OP}.

Adding the arch_ prefix requires some whitespace fixups to keep things
aligned. Some other unusual whitespace is fixed up at the same time
(e.g. in the cmpxchg wrappers).

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: Catalin Marinas 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: boqun.f...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-7-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/arm64/include/asm/atomic.h   | 237 +-
 arch/arm64/include/asm/atomic_ll_sc.h |  28 ++--
 arch/arm64/include/asm/atomic_lse.h   |  38 +++---
 arch/arm64/include/asm/cmpxchg.h  |  60 -
 arch/arm64/include/asm/sync_bitops.h  |  16 +--
 5 files changed, 193 insertions(+), 186 deletions(-)

diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
index 9bca54dda75c..1f4e9ee641c9 100644
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -42,124 +42,131 @@
 
 #define ATOMIC_INIT(i) { (i) }
 
-#define atomic_read(v) READ_ONCE((v)->counter)
-#define atomic_set(v, i)   WRITE_ONCE(((v)->counter), (i))
-
-#define atomic_add_return_relaxed  atomic_add_return_relaxed
-#define atomic_add_return_acquire  atomic_add_return_acquire
-#define atomic_add_return_release  atomic_add_return_release
-#define atomic_add_return  atomic_add_return
-
-#define atomic_sub_return_relaxed  atomic_sub_return_relaxed
-#define atomic_sub_return_acquire  atomic_sub_return_acquire
-#define atomic_sub_return_release  atomic_sub_return_release
-#define atomic_sub_return  atomic_sub_return
-
-#define atomic_fetch_add_relaxed   atomic_fetch_add_relaxed
-#define atomic_fetch_add_acquire   atomic_fetch_add_acquire
-#define atomic_fetch_add_release   atomic_fetch_add_release
-#define atomic_fetch_add   atomic_fetch_add
-
-#define atomic_fetch_sub_relaxed   atomic_fetch_sub_relaxed
-#define atomic_fetch_sub_acquire   atomic_fetch_sub_acquire
-#define atomic_fetch_sub_release   atomic_fetch_sub_release
-#define atomic_fetch_sub   atomic_fetch_sub
-
-#define atomic_fetch_and_relaxed   atomic_fetch_and_relaxed
-#define atomic_fetch_and_acquire   atomic_fetch_and_acquire
-#define atomic_fetch_and_release   atomic_fetch_and_release
-#define atomic_fetch_and   atomic_fetch_and
-
-#define atomic_fetch_andnot_relaxedatomic_fetch_andnot_relaxed
-#define atomic_fetch_andnot_acquireatomic_fetch_andnot_acquire
-#define atomic_fetch_andnot_releaseatomic_fetch_andnot_release
-#define atomic_fetch_andnotatomic_fetch_andnot
-
-#define atomic_fetch_or_relaxedatomic_fetch_or_relaxed
-#define atomic_fetch_or_acquireatomic_fetch_or_acquire
-#define atomic_fetch_or_releaseatomic_fetch_or_release
-#define atomic_fetch_oratomic_fetch_or
-
-#define atomic_fetch_xor_relaxed   atomic_fetch_xor_relaxed
-#define atomic_fetch_xor_acquire   atomic_fetch_xor_acquire
-#define atomic_fetch_xor_release   atomic_fetch_xor_release
-#define atomic_fetch_xor   atomic_fetch_xor
-
-#define atomic_xchg_relaxed(v, new)xchg_relaxed(&((v)->counter), (new))
-#define atomic_xchg_acquire(v, new)xchg_acquire(&((v)->counter), (new))
-#define atomic_xchg_release(v, new)xchg_release(&((v)->counter), (new))
-#define atomic_xchg(v, new)xchg(&((v)->counter), (new))
-
-#define atomic_cmpxchg_relaxed(v, old, new)\
-   cmpxchg_relaxed(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_acquire(v, old, new)\
-   cmpxchg_acquire(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_release(v, old, new)\
-   cmpxchg_release(&((v)->counter), (old), (new))
-#define atomic_cmpxchg(v, old, new)cmpxchg(&((v)->counter), (old), (new))
-
-#define atomic_andnot  atomic_andnot
+#define 

[tip:locking/core] arm64, locking/atomics: Use instrumented atomics

[tip-bot for Mark Rutland]

Commit-ID:  c0df10812835040e261b915f04887b0cf0411851
Gitweb: https://git.kernel.org/tip/c0df10812835040e261b915f04887b0cf0411851
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:30 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:01:40 +0100

arm64, locking/atomics: Use instrumented atomics

Now that the generic atomic headers provide instrumented wrappers of all
the atomics implemented by arm64, let's migrate arm64 over to these.

The additional instrumentation will help to find bugs (e.g. when fuzzing
with Syzkaller).

Mostly this change involves adding an arch_ prefix to a number of
function names and macro definitions. When LSE atomics are used, the
out-of-line LL/SC atomics will be named __ll_sc_arch_atomic_${OP}.

Adding the arch_ prefix requires some whitespace fixups to keep things
aligned. Some other unusual whitespace is fixed up at the same time
(e.g. in the cmpxchg wrappers).

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: Catalin Marinas 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: boqun.f...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-7-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/arm64/include/asm/atomic.h   | 237 +-
 arch/arm64/include/asm/atomic_ll_sc.h |  28 ++--
 arch/arm64/include/asm/atomic_lse.h   |  38 +++---
 arch/arm64/include/asm/cmpxchg.h  |  60 -
 arch/arm64/include/asm/sync_bitops.h  |  16 +--
 5 files changed, 193 insertions(+), 186 deletions(-)

diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
index 9bca54dda75c..1f4e9ee641c9 100644
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -42,124 +42,131 @@
 
 #define ATOMIC_INIT(i) { (i) }
 
-#define atomic_read(v) READ_ONCE((v)->counter)
-#define atomic_set(v, i)   WRITE_ONCE(((v)->counter), (i))
-
-#define atomic_add_return_relaxed  atomic_add_return_relaxed
-#define atomic_add_return_acquire  atomic_add_return_acquire
-#define atomic_add_return_release  atomic_add_return_release
-#define atomic_add_return  atomic_add_return
-
-#define atomic_sub_return_relaxed  atomic_sub_return_relaxed
-#define atomic_sub_return_acquire  atomic_sub_return_acquire
-#define atomic_sub_return_release  atomic_sub_return_release
-#define atomic_sub_return  atomic_sub_return
-
-#define atomic_fetch_add_relaxed   atomic_fetch_add_relaxed
-#define atomic_fetch_add_acquire   atomic_fetch_add_acquire
-#define atomic_fetch_add_release   atomic_fetch_add_release
-#define atomic_fetch_add   atomic_fetch_add
-
-#define atomic_fetch_sub_relaxed   atomic_fetch_sub_relaxed
-#define atomic_fetch_sub_acquire   atomic_fetch_sub_acquire
-#define atomic_fetch_sub_release   atomic_fetch_sub_release
-#define atomic_fetch_sub   atomic_fetch_sub
-
-#define atomic_fetch_and_relaxed   atomic_fetch_and_relaxed
-#define atomic_fetch_and_acquire   atomic_fetch_and_acquire
-#define atomic_fetch_and_release   atomic_fetch_and_release
-#define atomic_fetch_and   atomic_fetch_and
-
-#define atomic_fetch_andnot_relaxedatomic_fetch_andnot_relaxed
-#define atomic_fetch_andnot_acquireatomic_fetch_andnot_acquire
-#define atomic_fetch_andnot_releaseatomic_fetch_andnot_release
-#define atomic_fetch_andnotatomic_fetch_andnot
-
-#define atomic_fetch_or_relaxedatomic_fetch_or_relaxed
-#define atomic_fetch_or_acquireatomic_fetch_or_acquire
-#define atomic_fetch_or_releaseatomic_fetch_or_release
-#define atomic_fetch_oratomic_fetch_or
-
-#define atomic_fetch_xor_relaxed   atomic_fetch_xor_relaxed
-#define atomic_fetch_xor_acquire   atomic_fetch_xor_acquire
-#define atomic_fetch_xor_release   atomic_fetch_xor_release
-#define atomic_fetch_xor   atomic_fetch_xor
-
-#define atomic_xchg_relaxed(v, new)xchg_relaxed(&((v)->counter), (new))
-#define atomic_xchg_acquire(v, new)xchg_acquire(&((v)->counter), (new))
-#define atomic_xchg_release(v, new)xchg_release(&((v)->counter), (new))
-#define atomic_xchg(v, new)xchg(&((v)->counter), (new))
-
-#define atomic_cmpxchg_relaxed(v, old, new)\
-   cmpxchg_relaxed(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_acquire(v, old, new)\
-   cmpxchg_acquire(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_release(v, old, new)\
-   cmpxchg_release(&((v)->counter), (old), (new))
-#define atomic_cmpxchg(v, old, new)cmpxchg(&((v)->counter), (old), (new))
-
-#define atomic_andnot  atomic_andnot
+#define 

[tip:locking/core] locking/atomics: Check generated headers are up-to-date

[tip-bot for Mark Rutland]

Commit-ID:  8d32588077bdc390420cfa6946f407033a20d7a8
Gitweb: https://git.kernel.org/tip/8d32588077bdc390420cfa6946f407033a20d7a8
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:29 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:01:10 +0100

locking/atomics: Check generated headers are up-to-date

Now that all the generated atomic headers are in place, it would be good
to ensure that:

a) the headers are up-to-date when scripting changes.

b) developers don't directly modify the generated headers.

To ensure both of these properties, let's add a Kbuild step to check
that the generated headers are up-to-date.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-6-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 Kbuild  | 18 --
 scripts/atomic/check-atomics.sh | 19 +++
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/Kbuild b/Kbuild
index 005304205482..47c9fe175bd9 100644
--- a/Kbuild
+++ b/Kbuild
@@ -6,7 +6,8 @@
 # 2) Generate timeconst.h
 # 3) Generate asm-offsets.h (may need bounds.h and timeconst.h)
 # 4) Check for missing system calls
-# 5) Generate constants.py (may need bounds.h)
+# 5) check atomics headers are up-to-date
+# 6) Generate constants.py (may need bounds.h)
 
 #
 # 1) Generate bounds.h
@@ -72,7 +73,20 @@ missing-syscalls: scripts/checksyscalls.sh $(offsets-file) 
FORCE
$(call cmd,syscalls)
 
 #
-# 5) Generate constants for Python GDB integration
+# 5) Check atomic headers are up-to-date
+#
+
+always += old-atomics
+targets += old-atomics
+
+quiet_cmd_atomics = CALL$<
+  cmd_atomics = $(CONFIG_SHELL) scripts/atomic/check-atomics.sh
+
+old-atomics: scripts/atomic/check-atomics.sh FORCE
+   $(call cmd,atomics)
+
+#
+# 6) Generate constants for Python GDB integration
 #
 
 extra-$(CONFIG_GDB_SCRIPTS) += build_constants_py
diff --git a/scripts/atomic/check-atomics.sh b/scripts/atomic/check-atomics.sh
new file mode 100644
index ..c30101cddf2d
--- /dev/null
+++ b/scripts/atomic/check-atomics.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check if atomic headers are up-to-date
+
+ATOMICDIR=$(dirname $0)
+ATOMICTBL=${ATOMICDIR}/atomics.tbl
+LINUXDIR=${ATOMICDIR}/../..
+
+cat < /dev/null); then
+   printf "warning: include/${header} is out-of-date.\n"
+   fi
+done

[tip:locking/core] locking/atomics: Check generated headers are up-to-date

[tip-bot for Mark Rutland]

Commit-ID:  8d32588077bdc390420cfa6946f407033a20d7a8
Gitweb: https://git.kernel.org/tip/8d32588077bdc390420cfa6946f407033a20d7a8
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:29 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:01:10 +0100

locking/atomics: Check generated headers are up-to-date

Now that all the generated atomic headers are in place, it would be good
to ensure that:

a) the headers are up-to-date when scripting changes.

b) developers don't directly modify the generated headers.

To ensure both of these properties, let's add a Kbuild step to check
that the generated headers are up-to-date.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-6-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 Kbuild  | 18 --
 scripts/atomic/check-atomics.sh | 19 +++
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/Kbuild b/Kbuild
index 005304205482..47c9fe175bd9 100644
--- a/Kbuild
+++ b/Kbuild
@@ -6,7 +6,8 @@
 # 2) Generate timeconst.h
 # 3) Generate asm-offsets.h (may need bounds.h and timeconst.h)
 # 4) Check for missing system calls
-# 5) Generate constants.py (may need bounds.h)
+# 5) check atomics headers are up-to-date
+# 6) Generate constants.py (may need bounds.h)
 
 #
 # 1) Generate bounds.h
@@ -72,7 +73,20 @@ missing-syscalls: scripts/checksyscalls.sh $(offsets-file) 
FORCE
$(call cmd,syscalls)
 
 #
-# 5) Generate constants for Python GDB integration
+# 5) Check atomic headers are up-to-date
+#
+
+always += old-atomics
+targets += old-atomics
+
+quiet_cmd_atomics = CALL$<
+  cmd_atomics = $(CONFIG_SHELL) scripts/atomic/check-atomics.sh
+
+old-atomics: scripts/atomic/check-atomics.sh FORCE
+   $(call cmd,atomics)
+
+#
+# 6) Generate constants for Python GDB integration
 #
 
 extra-$(CONFIG_GDB_SCRIPTS) += build_constants_py
diff --git a/scripts/atomic/check-atomics.sh b/scripts/atomic/check-atomics.sh
new file mode 100644
index ..c30101cddf2d
--- /dev/null
+++ b/scripts/atomic/check-atomics.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check if atomic headers are up-to-date
+
+ATOMICDIR=$(dirname $0)
+ATOMICTBL=${ATOMICDIR}/atomics.tbl
+LINUXDIR=${ATOMICDIR}/../..
+
+cat < /dev/null); then
+   printf "warning: include/${header} is out-of-date.\n"
+   fi
+done

[tip:locking/core] locking/atomics: Switch to generated instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  aa525d063851a98e020b827fdd1d7776ae652301
Gitweb: https://git.kernel.org/tip/aa525d063851a98e020b827fdd1d7776ae652301
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:28 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:01:03 +0100

locking/atomics: Switch to generated instrumentation

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-instrumented.h, using the same table
used to generate the fallbacks and atomic-long wrappers.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Generating the atomic headers means that the instrumented wrappers will
remain in sync with the rest of the atomic APIs, and we gain all the
ordering variants of each atomic without having to manually expanded
them all.

The KASAN checks are automatically generated based on the function
parameters defined in atomics.tbl. Note that try_cmpxchg() now correctly
treats 'old' as a parameter that may be written to, and not only read as
the hand-written instrumentation assumed.

Other than the change to try_cmpxchg(), existing code should not be
affected by this patch. The patch introduces instrumentation for all
optional atomics (and ordering variants), along with the ifdeffery this
requires, enabling other architectures to make use of the instrumented
atomics.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: linuxdriv...@attotech.com
Cc: Dmitry Vyukov 
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: Andrey Ryabinin 
Cc: Alexander Potapenko 
Link: http://lkml.kernel.org/r/20180904104830.2975-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-instrumented.h | 1688 +
 1 file changed, 1504 insertions(+), 184 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 0d4b1d3dbc1e..b8f5b35216e1 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -1,3 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-instrumented.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 /*
  * This file provides wrappers with KASAN instrumentation for atomic 
operations.
  * To use this functionality an arch's atomic.h file needs to define all
@@ -9,459 +14,1774 @@
  * arch_ variants (i.e. arch_atomic_read()/arch_atomic_cmpxchg()) to avoid
  * double instrumentation.
  */
+#ifndef _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+#define _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+
+#include 
+#include 
+
+static inline int
+atomic_read(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read(v);
+}
+#define atomic_read atomic_read
+
+#if defined(arch_atomic_read_acquire)
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read_acquire(v);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+static inline void
+atomic_set(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set(v, i);
+}
+#define atomic_set atomic_set
+
+#if defined(arch_atomic_set_release)
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set_release(v, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+static inline void
+atomic_add(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_add(i, v);
+}
+#define atomic_add atomic_add
+
+#if !defined(arch_atomic_add_return_relaxed) || defined(arch_atomic_add_return)
+static inline int
+atomic_add_return(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return(i, v);
+}
+#define atomic_add_return atomic_add_return
+#endif
+
+#if defined(arch_atomic_add_return_acquire)
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_acquire(i, v);
+}
+#define atomic_add_return_acquire atomic_add_return_acquire
+#endif
+
+#if defined(arch_atomic_add_return_release)
+static inline int
+atomic_add_return_release(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_release(i, v);
+}
+#define atomic_add_return_release atomic_add_return_release
+#endif
+

[tip:locking/core] locking/atomics: Switch to generated instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  aa525d063851a98e020b827fdd1d7776ae652301
Gitweb: https://git.kernel.org/tip/aa525d063851a98e020b827fdd1d7776ae652301
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:28 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:01:03 +0100

locking/atomics: Switch to generated instrumentation

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-instrumented.h, using the same table
used to generate the fallbacks and atomic-long wrappers.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Generating the atomic headers means that the instrumented wrappers will
remain in sync with the rest of the atomic APIs, and we gain all the
ordering variants of each atomic without having to manually expanded
them all.

The KASAN checks are automatically generated based on the function
parameters defined in atomics.tbl. Note that try_cmpxchg() now correctly
treats 'old' as a parameter that may be written to, and not only read as
the hand-written instrumentation assumed.

Other than the change to try_cmpxchg(), existing code should not be
affected by this patch. The patch introduces instrumentation for all
optional atomics (and ordering variants), along with the ifdeffery this
requires, enabling other architectures to make use of the instrumented
atomics.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: linuxdriv...@attotech.com
Cc: Dmitry Vyukov 
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: Andrey Ryabinin 
Cc: Alexander Potapenko 
Link: http://lkml.kernel.org/r/20180904104830.2975-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-instrumented.h | 1688 +
 1 file changed, 1504 insertions(+), 184 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 0d4b1d3dbc1e..b8f5b35216e1 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -1,3 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-instrumented.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 /*
  * This file provides wrappers with KASAN instrumentation for atomic 
operations.
  * To use this functionality an arch's atomic.h file needs to define all
@@ -9,459 +14,1774 @@
  * arch_ variants (i.e. arch_atomic_read()/arch_atomic_cmpxchg()) to avoid
  * double instrumentation.
  */
+#ifndef _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+#define _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+
+#include 
+#include 
+
+static inline int
+atomic_read(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read(v);
+}
+#define atomic_read atomic_read
+
+#if defined(arch_atomic_read_acquire)
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read_acquire(v);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+static inline void
+atomic_set(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set(v, i);
+}
+#define atomic_set atomic_set
+
+#if defined(arch_atomic_set_release)
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set_release(v, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+static inline void
+atomic_add(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_add(i, v);
+}
+#define atomic_add atomic_add
+
+#if !defined(arch_atomic_add_return_relaxed) || defined(arch_atomic_add_return)
+static inline int
+atomic_add_return(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return(i, v);
+}
+#define atomic_add_return atomic_add_return
+#endif
+
+#if defined(arch_atomic_add_return_acquire)
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_acquire(i, v);
+}
+#define atomic_add_return_acquire atomic_add_return_acquire
+#endif
+
+#if defined(arch_atomic_add_return_release)
+static inline int
+atomic_add_return_release(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_release(i, v);
+}
+#define atomic_add_return_release atomic_add_return_release
+#endif
+

[tip:locking/core] locking/atomics: Switch to generated atomic-long

[tip-bot for Mark Rutland]

Commit-ID:  b5d47ef9ea5c5fe31d7eabeb79f697629bd9e2cb
Gitweb: https://git.kernel.org/tip/b5d47ef9ea5c5fe31d7eabeb79f697629bd9e2cb
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:27 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:00:55 +0100

locking/atomics: Switch to generated atomic-long

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-long.h, using the same table that
gen-atomic-fallbacks.h uses to fill in gaps in the atomic_* and
atomic64_* APIs.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Other than *_INIT() and *_cond_read_acquire(), all API functions are
implemented as static inline C functions, ensuring consistent type
promotion and/or truncation without requiring explicit casts to be
applied to parameters or return values.

Since we typedef atomic_long_t to either atomic_t or atomic64_t, we know
these types are equivalent, and don't require explicit casts between
them. However, as the try_cmpxchg*() functions take a pointer for the
'old' parameter, which may be an int or s64, an explicit cast is
generated for this.

There should be no functional change as a result of this patch (i.e.
existing code should not be affected). However, this introduces a number
of functions into the atomic_long_* API, bringing it into line with the
atomic_* and atomic64_* APIs.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-long.h | 1173 ++---
 1 file changed, 958 insertions(+), 215 deletions(-)

diff --git a/include/asm-generic/atomic-long.h 
b/include/asm-generic/atomic-long.h
index 87d14476edc2..a833d385a70b 100644
--- a/include/asm-generic/atomic-long.h
+++ b/include/asm-generic/atomic-long.h
@@ -1,269 +1,1012 @@
-/* SPDX-License-Identifier: GPL-2.0 */
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-long.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 #ifndef _ASM_GENERIC_ATOMIC_LONG_H
 #define _ASM_GENERIC_ATOMIC_LONG_H
-/*
- * Copyright (C) 2005 Silicon Graphics, Inc.
- * Christoph Lameter
- *
- * Allows to provide arch independent atomic definitions without the need to
- * edit all arch specific atomic.h files.
- */
 
 #include 
 
-/*
- * Suppport for atomic_long_t
- *
- * Casts for parameters are avoided for existing atomic functions in order to
- * avoid issues with cast-as-lval under gcc 4.x and other limitations that the
- * macros of a platform may have.
- */
+#ifdef CONFIG_64BIT
+typedef atomic64_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
+#define atomic_long_cond_read_acquire  atomic64_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic64_cond_read_relaxed
+#else
+typedef atomic_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
+#define atomic_long_cond_read_acquire  atomic_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic_cond_read_relaxed
+#endif
 
-#if BITS_PER_LONG == 64
+#ifdef CONFIG_64BIT
 
-typedef atomic64_t atomic_long_t;
+static inline long
+atomic_long_read(const atomic_long_t *v)
+{
+   return atomic64_read(v);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic64 ## x
-#define ATOMIC_LONG_TYPE   s64
+static inline long
+atomic_long_read_acquire(const atomic_long_t *v)
+{
+   return atomic64_read_acquire(v);
+}
 
-#else
+static inline void
+atomic_long_set(atomic_long_t *v, long i)
+{
+   atomic64_set(v, i);
+}
 
-typedef atomic_t atomic_long_t;
+static inline void
+atomic_long_set_release(atomic_long_t *v, long i)
+{
+   atomic64_set_release(v, i);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic ## x
-#define ATOMIC_LONG_TYPE   int
+static inline void
+atomic_long_add(long i, atomic_long_t *v)
+{
+   atomic64_add(i, v);
+}
 
-#endif
+static inline long
+atomic_long_add_return(long i, atomic_long_t *v)
+{
+   return atomic64_add_return(i, v);
+}
+
+static inline long
+atomic_long_add_return_acquire(long i, atomic_long_t *v)
+{
+   return atomic64_add_return_acquire(i, v);
+}
+

[tip:locking/core] locking/atomics: Switch to generated atomic-long

[tip-bot for Mark Rutland]

Commit-ID:  b5d47ef9ea5c5fe31d7eabeb79f697629bd9e2cb
Gitweb: https://git.kernel.org/tip/b5d47ef9ea5c5fe31d7eabeb79f697629bd9e2cb
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:27 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:00:55 +0100

locking/atomics: Switch to generated atomic-long

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-long.h, using the same table that
gen-atomic-fallbacks.h uses to fill in gaps in the atomic_* and
atomic64_* APIs.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Other than *_INIT() and *_cond_read_acquire(), all API functions are
implemented as static inline C functions, ensuring consistent type
promotion and/or truncation without requiring explicit casts to be
applied to parameters or return values.

Since we typedef atomic_long_t to either atomic_t or atomic64_t, we know
these types are equivalent, and don't require explicit casts between
them. However, as the try_cmpxchg*() functions take a pointer for the
'old' parameter, which may be an int or s64, an explicit cast is
generated for this.

There should be no functional change as a result of this patch (i.e.
existing code should not be affected). However, this introduces a number
of functions into the atomic_long_* API, bringing it into line with the
atomic_* and atomic64_* APIs.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-long.h | 1173 ++---
 1 file changed, 958 insertions(+), 215 deletions(-)

diff --git a/include/asm-generic/atomic-long.h 
b/include/asm-generic/atomic-long.h
index 87d14476edc2..a833d385a70b 100644
--- a/include/asm-generic/atomic-long.h
+++ b/include/asm-generic/atomic-long.h
@@ -1,269 +1,1012 @@
-/* SPDX-License-Identifier: GPL-2.0 */
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-long.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 #ifndef _ASM_GENERIC_ATOMIC_LONG_H
 #define _ASM_GENERIC_ATOMIC_LONG_H
-/*
- * Copyright (C) 2005 Silicon Graphics, Inc.
- * Christoph Lameter
- *
- * Allows to provide arch independent atomic definitions without the need to
- * edit all arch specific atomic.h files.
- */
 
 #include 
 
-/*
- * Suppport for atomic_long_t
- *
- * Casts for parameters are avoided for existing atomic functions in order to
- * avoid issues with cast-as-lval under gcc 4.x and other limitations that the
- * macros of a platform may have.
- */
+#ifdef CONFIG_64BIT
+typedef atomic64_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
+#define atomic_long_cond_read_acquire  atomic64_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic64_cond_read_relaxed
+#else
+typedef atomic_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
+#define atomic_long_cond_read_acquire  atomic_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic_cond_read_relaxed
+#endif
 
-#if BITS_PER_LONG == 64
+#ifdef CONFIG_64BIT
 
-typedef atomic64_t atomic_long_t;
+static inline long
+atomic_long_read(const atomic_long_t *v)
+{
+   return atomic64_read(v);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic64 ## x
-#define ATOMIC_LONG_TYPE   s64
+static inline long
+atomic_long_read_acquire(const atomic_long_t *v)
+{
+   return atomic64_read_acquire(v);
+}
 
-#else
+static inline void
+atomic_long_set(atomic_long_t *v, long i)
+{
+   atomic64_set(v, i);
+}
 
-typedef atomic_t atomic_long_t;
+static inline void
+atomic_long_set_release(atomic_long_t *v, long i)
+{
+   atomic64_set_release(v, i);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic ## x
-#define ATOMIC_LONG_TYPE   int
+static inline void
+atomic_long_add(long i, atomic_long_t *v)
+{
+   atomic64_add(i, v);
+}
 
-#endif
+static inline long
+atomic_long_add_return(long i, atomic_long_t *v)
+{
+   return atomic64_add_return(i, v);
+}
+
+static inline long
+atomic_long_add_return_acquire(long i, atomic_long_t *v)
+{
+   return atomic64_add_return_acquire(i, v);
+}
+

[tip:locking/core] locking/atomics: Switch to generated fallbacks

[tip-bot for Mark Rutland]

Commit-ID:  9fa45070a2e59a871e1cd3370173369f3a4f61e2
Gitweb: https://git.kernel.org/tip/9fa45070a2e59a871e1cd3370173369f3a4f61e2
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:26 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:00:46 +0100

locking/atomics: Switch to generated fallbacks

As a step to ensuring the atomic* APIs are consistent, switch to fallbacks
generated by gen-atomic-fallback.sh.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

The new fallback header should be equivalent to the old fallbacks in
, but:

* It is formatted a little differently due to scripting ensuring things
  are more regular than they used to be.

* Fallbacks are now expanded in-place as static inline functions rather
  than macros.

* The prototypes for fallbacks are arragned consistently with the return
  type on a separate line to try to keep to a sensible line length.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/atomic-fallback.h | 2294 +++
 include/linux/atomic.h  | 1241 +
 2 files changed, 2295 insertions(+), 1240 deletions(-)

diff --git a/include/linux/atomic-fallback.h b/include/linux/atomic-fallback.h
new file mode 100644
index ..1c02c0112fbb
--- /dev/null
+++ b/include/linux/atomic-fallback.h
@@ -0,0 +1,2294 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-fallback.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
+#ifndef _LINUX_ATOMIC_FALLBACK_H
+#define _LINUX_ATOMIC_FALLBACK_H
+
+#ifndef xchg_relaxed
+#define xchg_relaxed   xchg
+#define xchg_acquire   xchg
+#define xchg_release   xchg
+#else /* xchg_relaxed */
+
+#ifndef xchg_acquire
+#define xchg_acquire(...) \
+   __atomic_op_acquire(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg_release
+#define xchg_release(...) \
+   __atomic_op_release(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg
+#define xchg(...) \
+   __atomic_op_fence(xchg, __VA_ARGS__)
+#endif
+
+#endif /* xchg_relaxed */
+
+#ifndef cmpxchg_relaxed
+#define cmpxchg_relaxedcmpxchg
+#define cmpxchg_acquirecmpxchg
+#define cmpxchg_releasecmpxchg
+#else /* cmpxchg_relaxed */
+
+#ifndef cmpxchg_acquire
+#define cmpxchg_acquire(...) \
+   __atomic_op_acquire(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg_release
+#define cmpxchg_release(...) \
+   __atomic_op_release(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg
+#define cmpxchg(...) \
+   __atomic_op_fence(cmpxchg, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg_relaxed */
+
+#ifndef cmpxchg64_relaxed
+#define cmpxchg64_relaxed  cmpxchg64
+#define cmpxchg64_acquire  cmpxchg64
+#define cmpxchg64_release  cmpxchg64
+#else /* cmpxchg64_relaxed */
+
+#ifndef cmpxchg64_acquire
+#define cmpxchg64_acquire(...) \
+   __atomic_op_acquire(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64_release
+#define cmpxchg64_release(...) \
+   __atomic_op_release(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64
+#define cmpxchg64(...) \
+   __atomic_op_fence(cmpxchg64, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg64_relaxed */
+
+#ifndef atomic_read_acquire
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   return smp_load_acquire(&(v)->counter);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+#ifndef atomic_set_release
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   smp_store_release(&(v)->counter, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+#ifndef atomic_add_return_relaxed
+#define atomic_add_return_acquire atomic_add_return
+#define atomic_add_return_release atomic_add_return
+#define atomic_add_return_relaxed atomic_add_return
+#else /* atomic_add_return_relaxed */
+
+#ifndef atomic_add_return_acquire
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   int ret = atomic_add_return_relaxed(i, v);
+   __atomic_acquire_fence();
+   return ret;
+}
+#define atomic_add_return_acquire 

[tip:locking/core] locking/atomics: Switch to generated fallbacks

[tip-bot for Mark Rutland]

Commit-ID:  9fa45070a2e59a871e1cd3370173369f3a4f61e2
Gitweb: https://git.kernel.org/tip/9fa45070a2e59a871e1cd3370173369f3a4f61e2
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:26 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:00:46 +0100

locking/atomics: Switch to generated fallbacks

As a step to ensuring the atomic* APIs are consistent, switch to fallbacks
generated by gen-atomic-fallback.sh.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

The new fallback header should be equivalent to the old fallbacks in
, but:

* It is formatted a little differently due to scripting ensuring things
  are more regular than they used to be.

* Fallbacks are now expanded in-place as static inline functions rather
  than macros.

* The prototypes for fallbacks are arragned consistently with the return
  type on a separate line to try to keep to a sensible line length.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/atomic-fallback.h | 2294 +++
 include/linux/atomic.h  | 1241 +
 2 files changed, 2295 insertions(+), 1240 deletions(-)

diff --git a/include/linux/atomic-fallback.h b/include/linux/atomic-fallback.h
new file mode 100644
index ..1c02c0112fbb
--- /dev/null
+++ b/include/linux/atomic-fallback.h
@@ -0,0 +1,2294 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-fallback.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
+#ifndef _LINUX_ATOMIC_FALLBACK_H
+#define _LINUX_ATOMIC_FALLBACK_H
+
+#ifndef xchg_relaxed
+#define xchg_relaxed   xchg
+#define xchg_acquire   xchg
+#define xchg_release   xchg
+#else /* xchg_relaxed */
+
+#ifndef xchg_acquire
+#define xchg_acquire(...) \
+   __atomic_op_acquire(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg_release
+#define xchg_release(...) \
+   __atomic_op_release(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg
+#define xchg(...) \
+   __atomic_op_fence(xchg, __VA_ARGS__)
+#endif
+
+#endif /* xchg_relaxed */
+
+#ifndef cmpxchg_relaxed
+#define cmpxchg_relaxedcmpxchg
+#define cmpxchg_acquirecmpxchg
+#define cmpxchg_releasecmpxchg
+#else /* cmpxchg_relaxed */
+
+#ifndef cmpxchg_acquire
+#define cmpxchg_acquire(...) \
+   __atomic_op_acquire(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg_release
+#define cmpxchg_release(...) \
+   __atomic_op_release(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg
+#define cmpxchg(...) \
+   __atomic_op_fence(cmpxchg, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg_relaxed */
+
+#ifndef cmpxchg64_relaxed
+#define cmpxchg64_relaxed  cmpxchg64
+#define cmpxchg64_acquire  cmpxchg64
+#define cmpxchg64_release  cmpxchg64
+#else /* cmpxchg64_relaxed */
+
+#ifndef cmpxchg64_acquire
+#define cmpxchg64_acquire(...) \
+   __atomic_op_acquire(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64_release
+#define cmpxchg64_release(...) \
+   __atomic_op_release(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64
+#define cmpxchg64(...) \
+   __atomic_op_fence(cmpxchg64, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg64_relaxed */
+
+#ifndef atomic_read_acquire
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   return smp_load_acquire(&(v)->counter);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+#ifndef atomic_set_release
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   smp_store_release(&(v)->counter, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+#ifndef atomic_add_return_relaxed
+#define atomic_add_return_acquire atomic_add_return
+#define atomic_add_return_release atomic_add_return
+#define atomic_add_return_relaxed atomic_add_return
+#else /* atomic_add_return_relaxed */
+
+#ifndef atomic_add_return_acquire
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   int ret = atomic_add_return_relaxed(i, v);
+   __atomic_acquire_fence();
+   return ret;
+}
+#define atomic_add_return_acquire 

[tip:locking/core] locking/atomics: Add common header generation files

[tip-bot for Mark Rutland]

Commit-ID:  ace9bad4df2684f31cbfe8c4ce7a0f5d92b27925
Gitweb: https://git.kernel.org/tip/ace9bad4df2684f31cbfe8c4ce7a0f5d92b27925
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:25 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:00:36 +0100

locking/atomics: Add common header generation files

To minimize repetition, to allow for future rework, and to ensure
regularity of the various atomic APIs, we'd like to automatically
generate (the bulk of) a number of headers related to atomics.

This patch adds the infrastructure to do so, leaving actual conversion
of headers to subsequent patches. This infrastructure consists of:

* atomics.tbl - a table describing the functions in the atomics API,
  with names, prototypes, and metadata describing the variants that
  exist (e.g fetch/return, acquire/release/relaxed). Note that the
  return type is dependent on the particular variant.

* atomic-tbl.sh - a library of routines useful for dealing with
  atomics.tbl (e.g. querying which variants exist, or generating
  argument/parameter lists for a given function variant).

* gen-atomic-fallback.sh - a script which generates a header of
  fallbacks, covering cases where architecture omit certain functions
  (e.g. omitting relaxed variants).

* gen-atomic-long.sh - a script which generates wrappers providing the
  atomic_long API atomic of the relevant atomic or atomic64 API,
  ensuring the APIs are consistent.

* gen-atomic-instrumented.sh - a script which generates atomic* wrappers
  atop of arch_atomic* functions, with automatically generated KASAN
  instrumentation.

* fallbacks/* - a set of fallback implementations for atomics, which
  should be used when no implementation of a given atomic is provided.
  These are used by gen-atomic-fallback.sh to generate fallbacks, and
  these are also used by other scripts to determine the set of optional
  atomics (as required to generate preprocessor guards correctly).

  Fallbacks may use the following variables:

  ${atomic} atomic prefix: atomic/atomic64/atomic_long, which can be
used to derive the atomic type, and to prefix functions

  ${int}integer type: int/s64/long

  ${pfx}variant prefix, e.g. fetch_

  ${name}   base function name, e.g. add

  ${sfx}variant suffix, e.g. _return

  ${order}  order suffix, e.g. _relaxed

  ${atomicname} full name, e.g. atomic64_fetch_add_relaxed

  ${ret}return type of the function, e.g. void

  ${retstmt}a return statement (with a trailing space), unless the
variant returns void

  ${params} parameter list for the function declaration, e.g.
"int i, atomic_t *v"

  ${args}   argument list for invoking the function, e.g. "i, v"

  ... for clarity, ${ret}, ${retstmt}, ${params}, and ${args} are
  open-coded for fallbacks where these do not vary, or are critical to
  understanding the logic of the fallback.

The MAINTAINERS entry for the atomic infrastructure is updated to cover
the new scripts.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-2-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 MAINTAINERS  |   1 +
 scripts/atomic/atomic-tbl.sh | 186 +++
 scripts/atomic/atomics.tbl   |  41 ++
 scripts/atomic/fallbacks/acquire |   9 ++
 scripts/atomic/fallbacks/add_negative|  16 +++
 scripts/atomic/fallbacks/add_unless  |  16 +++
 scripts/atomic/fallbacks/andnot  |   7 +
 scripts/atomic/fallbacks/dec |   7 +
 scripts/atomic/fallbacks/dec_and_test|  15 +++
 scripts/atomic/fallbacks/dec_if_positive |  15 +++
 scripts/atomic/fallbacks/dec_unless_positive |  14 ++
 scripts/atomic/fallbacks/fence   |  11 ++
 scripts/atomic/fallbacks/fetch_add_unless|  23 
 scripts/atomic/fallbacks/inc |   7 +
 scripts/atomic/fallbacks/inc_and_test|  15 +++
 scripts/atomic/fallbacks/inc_not_zero|  14 ++
 scripts/atomic/fallbacks/inc_unless_negative |  14 ++
 scripts/atomic/fallbacks/read_acquire|   7 +
 scripts/atomic/fallbacks/release |   8 ++
 scripts/atomic/fallbacks/set_release |   7 +
 scripts/atomic/fallbacks/sub_and_test|  16 +++
 scripts/atomic/fallbacks/try_cmpxchg |  11 ++
 scripts/atomic/gen-atomic-fallback.sh| 181 ++
 scripts/atomic/gen-atomic-instrumented.sh| 182 ++
 scripts/atomic/gen-atomic-long.sh| 101 +++
 25 files changed, 

[tip:locking/core] locking/atomics: Add common header generation files

[tip-bot for Mark Rutland]

Commit-ID:  ace9bad4df2684f31cbfe8c4ce7a0f5d92b27925
Gitweb: https://git.kernel.org/tip/ace9bad4df2684f31cbfe8c4ce7a0f5d92b27925
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:25 +0100
Committer:  Ingo Molnar 
CommitDate: Thu, 1 Nov 2018 11:00:36 +0100

locking/atomics: Add common header generation files

To minimize repetition, to allow for future rework, and to ensure
regularity of the various atomic APIs, we'd like to automatically
generate (the bulk of) a number of headers related to atomics.

This patch adds the infrastructure to do so, leaving actual conversion
of headers to subsequent patches. This infrastructure consists of:

* atomics.tbl - a table describing the functions in the atomics API,
  with names, prototypes, and metadata describing the variants that
  exist (e.g fetch/return, acquire/release/relaxed). Note that the
  return type is dependent on the particular variant.

* atomic-tbl.sh - a library of routines useful for dealing with
  atomics.tbl (e.g. querying which variants exist, or generating
  argument/parameter lists for a given function variant).

* gen-atomic-fallback.sh - a script which generates a header of
  fallbacks, covering cases where architecture omit certain functions
  (e.g. omitting relaxed variants).

* gen-atomic-long.sh - a script which generates wrappers providing the
  atomic_long API atomic of the relevant atomic or atomic64 API,
  ensuring the APIs are consistent.

* gen-atomic-instrumented.sh - a script which generates atomic* wrappers
  atop of arch_atomic* functions, with automatically generated KASAN
  instrumentation.

* fallbacks/* - a set of fallback implementations for atomics, which
  should be used when no implementation of a given atomic is provided.
  These are used by gen-atomic-fallback.sh to generate fallbacks, and
  these are also used by other scripts to determine the set of optional
  atomics (as required to generate preprocessor guards correctly).

  Fallbacks may use the following variables:

  ${atomic} atomic prefix: atomic/atomic64/atomic_long, which can be
used to derive the atomic type, and to prefix functions

  ${int}integer type: int/s64/long

  ${pfx}variant prefix, e.g. fetch_

  ${name}   base function name, e.g. add

  ${sfx}variant suffix, e.g. _return

  ${order}  order suffix, e.g. _relaxed

  ${atomicname} full name, e.g. atomic64_fetch_add_relaxed

  ${ret}return type of the function, e.g. void

  ${retstmt}a return statement (with a trailing space), unless the
variant returns void

  ${params} parameter list for the function declaration, e.g.
"int i, atomic_t *v"

  ${args}   argument list for invoking the function, e.g. "i, v"

  ... for clarity, ${ret}, ${retstmt}, ${params}, and ${args} are
  open-coded for fallbacks where these do not vary, or are critical to
  understanding the logic of the fallback.

The MAINTAINERS entry for the atomic infrastructure is updated to cover
the new scripts.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Link: http://lkml.kernel.org/r/20180904104830.2975-2-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 MAINTAINERS  |   1 +
 scripts/atomic/atomic-tbl.sh | 186 +++
 scripts/atomic/atomics.tbl   |  41 ++
 scripts/atomic/fallbacks/acquire |   9 ++
 scripts/atomic/fallbacks/add_negative|  16 +++
 scripts/atomic/fallbacks/add_unless  |  16 +++
 scripts/atomic/fallbacks/andnot  |   7 +
 scripts/atomic/fallbacks/dec |   7 +
 scripts/atomic/fallbacks/dec_and_test|  15 +++
 scripts/atomic/fallbacks/dec_if_positive |  15 +++
 scripts/atomic/fallbacks/dec_unless_positive |  14 ++
 scripts/atomic/fallbacks/fence   |  11 ++
 scripts/atomic/fallbacks/fetch_add_unless|  23 
 scripts/atomic/fallbacks/inc |   7 +
 scripts/atomic/fallbacks/inc_and_test|  15 +++
 scripts/atomic/fallbacks/inc_not_zero|  14 ++
 scripts/atomic/fallbacks/inc_unless_negative |  14 ++
 scripts/atomic/fallbacks/read_acquire|   7 +
 scripts/atomic/fallbacks/release |   8 ++
 scripts/atomic/fallbacks/set_release |   7 +
 scripts/atomic/fallbacks/sub_and_test|  16 +++
 scripts/atomic/fallbacks/try_cmpxchg |  11 ++
 scripts/atomic/gen-atomic-fallback.sh| 181 ++
 scripts/atomic/gen-atomic-instrumented.sh| 182 ++
 scripts/atomic/gen-atomic-long.sh| 101 +++
 25 files changed, 

[tip:locking/core] arm64: Use instrumented atomics

[tip-bot for Mark Rutland]

Commit-ID:  30bc7baa9de81efc0584b9290ce8040a1130f156
Gitweb: https://git.kernel.org/tip/30bc7baa9de81efc0584b9290ce8040a1130f156
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:30 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

arm64: Use instrumented atomics

Now that the generic atomic headers provide instrumented wrappers of all
the atomics implemented by arm64, let's migrate arm64 over to these.

The additional instrumentation will help to find bugs (e.g. when fuzzing
with Syzkaller).

Mostly this change involves adding an arch_ prefix to a number of
function names and macro definitions. When LSE atomics are used, the
out-of-line LL/SC atomics will be named __ll_sc_arch_atomic_${OP}.

Adding the arch_ prefix requires some whitespace fixups to keep things
aligned. Some other unusual whitespace is fixed up at the same time
(e.g. in the cmpxchg wrappers).

Cc: linux-arm-ker...@lists.infradead.org
Cc: Catalin Marinas 
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: boqun.f...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-7-mark.rutl...@arm.com
---
 arch/arm64/include/asm/atomic.h   | 237 +-
 arch/arm64/include/asm/atomic_ll_sc.h |  28 ++--
 arch/arm64/include/asm/atomic_lse.h   |  38 +++---
 arch/arm64/include/asm/cmpxchg.h  |  60 -
 arch/arm64/include/asm/sync_bitops.h  |  16 +--
 5 files changed, 193 insertions(+), 186 deletions(-)

diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
index 9bca54dda75c..1f4e9ee641c9 100644
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -42,124 +42,131 @@
 
 #define ATOMIC_INIT(i) { (i) }
 
-#define atomic_read(v) READ_ONCE((v)->counter)
-#define atomic_set(v, i)   WRITE_ONCE(((v)->counter), (i))
-
-#define atomic_add_return_relaxed  atomic_add_return_relaxed
-#define atomic_add_return_acquire  atomic_add_return_acquire
-#define atomic_add_return_release  atomic_add_return_release
-#define atomic_add_return  atomic_add_return
-
-#define atomic_sub_return_relaxed  atomic_sub_return_relaxed
-#define atomic_sub_return_acquire  atomic_sub_return_acquire
-#define atomic_sub_return_release  atomic_sub_return_release
-#define atomic_sub_return  atomic_sub_return
-
-#define atomic_fetch_add_relaxed   atomic_fetch_add_relaxed
-#define atomic_fetch_add_acquire   atomic_fetch_add_acquire
-#define atomic_fetch_add_release   atomic_fetch_add_release
-#define atomic_fetch_add   atomic_fetch_add
-
-#define atomic_fetch_sub_relaxed   atomic_fetch_sub_relaxed
-#define atomic_fetch_sub_acquire   atomic_fetch_sub_acquire
-#define atomic_fetch_sub_release   atomic_fetch_sub_release
-#define atomic_fetch_sub   atomic_fetch_sub
-
-#define atomic_fetch_and_relaxed   atomic_fetch_and_relaxed
-#define atomic_fetch_and_acquire   atomic_fetch_and_acquire
-#define atomic_fetch_and_release   atomic_fetch_and_release
-#define atomic_fetch_and   atomic_fetch_and
-
-#define atomic_fetch_andnot_relaxedatomic_fetch_andnot_relaxed
-#define atomic_fetch_andnot_acquireatomic_fetch_andnot_acquire
-#define atomic_fetch_andnot_releaseatomic_fetch_andnot_release
-#define atomic_fetch_andnotatomic_fetch_andnot
-
-#define atomic_fetch_or_relaxedatomic_fetch_or_relaxed
-#define atomic_fetch_or_acquireatomic_fetch_or_acquire
-#define atomic_fetch_or_releaseatomic_fetch_or_release
-#define atomic_fetch_oratomic_fetch_or
-
-#define atomic_fetch_xor_relaxed   atomic_fetch_xor_relaxed
-#define atomic_fetch_xor_acquire   atomic_fetch_xor_acquire
-#define atomic_fetch_xor_release   atomic_fetch_xor_release
-#define atomic_fetch_xor   atomic_fetch_xor
-
-#define atomic_xchg_relaxed(v, new)xchg_relaxed(&((v)->counter), (new))
-#define atomic_xchg_acquire(v, new)xchg_acquire(&((v)->counter), (new))
-#define atomic_xchg_release(v, new)xchg_release(&((v)->counter), (new))
-#define atomic_xchg(v, new)xchg(&((v)->counter), (new))
-
-#define atomic_cmpxchg_relaxed(v, old, new)\
-   cmpxchg_relaxed(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_acquire(v, old, new)\
-   cmpxchg_acquire(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_release(v, old, new)\
-   cmpxchg_release(&((v)->counter), (old), (new))
-#define atomic_cmpxchg(v, old, new)cmpxchg(&((v)->counter), (old), (new))
-
-#define atomic_andnot  atomic_andnot
+#define 

[tip:locking/core] arm64: Use instrumented atomics

[tip-bot for Mark Rutland]

Commit-ID:  30bc7baa9de81efc0584b9290ce8040a1130f156
Gitweb: https://git.kernel.org/tip/30bc7baa9de81efc0584b9290ce8040a1130f156
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:30 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

arm64: Use instrumented atomics

Now that the generic atomic headers provide instrumented wrappers of all
the atomics implemented by arm64, let's migrate arm64 over to these.

The additional instrumentation will help to find bugs (e.g. when fuzzing
with Syzkaller).

Mostly this change involves adding an arch_ prefix to a number of
function names and macro definitions. When LSE atomics are used, the
out-of-line LL/SC atomics will be named __ll_sc_arch_atomic_${OP}.

Adding the arch_ prefix requires some whitespace fixups to keep things
aligned. Some other unusual whitespace is fixed up at the same time
(e.g. in the cmpxchg wrappers).

Cc: linux-arm-ker...@lists.infradead.org
Cc: Catalin Marinas 
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: boqun.f...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-7-mark.rutl...@arm.com
---
 arch/arm64/include/asm/atomic.h   | 237 +-
 arch/arm64/include/asm/atomic_ll_sc.h |  28 ++--
 arch/arm64/include/asm/atomic_lse.h   |  38 +++---
 arch/arm64/include/asm/cmpxchg.h  |  60 -
 arch/arm64/include/asm/sync_bitops.h  |  16 +--
 5 files changed, 193 insertions(+), 186 deletions(-)

diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
index 9bca54dda75c..1f4e9ee641c9 100644
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -42,124 +42,131 @@
 
 #define ATOMIC_INIT(i) { (i) }
 
-#define atomic_read(v) READ_ONCE((v)->counter)
-#define atomic_set(v, i)   WRITE_ONCE(((v)->counter), (i))
-
-#define atomic_add_return_relaxed  atomic_add_return_relaxed
-#define atomic_add_return_acquire  atomic_add_return_acquire
-#define atomic_add_return_release  atomic_add_return_release
-#define atomic_add_return  atomic_add_return
-
-#define atomic_sub_return_relaxed  atomic_sub_return_relaxed
-#define atomic_sub_return_acquire  atomic_sub_return_acquire
-#define atomic_sub_return_release  atomic_sub_return_release
-#define atomic_sub_return  atomic_sub_return
-
-#define atomic_fetch_add_relaxed   atomic_fetch_add_relaxed
-#define atomic_fetch_add_acquire   atomic_fetch_add_acquire
-#define atomic_fetch_add_release   atomic_fetch_add_release
-#define atomic_fetch_add   atomic_fetch_add
-
-#define atomic_fetch_sub_relaxed   atomic_fetch_sub_relaxed
-#define atomic_fetch_sub_acquire   atomic_fetch_sub_acquire
-#define atomic_fetch_sub_release   atomic_fetch_sub_release
-#define atomic_fetch_sub   atomic_fetch_sub
-
-#define atomic_fetch_and_relaxed   atomic_fetch_and_relaxed
-#define atomic_fetch_and_acquire   atomic_fetch_and_acquire
-#define atomic_fetch_and_release   atomic_fetch_and_release
-#define atomic_fetch_and   atomic_fetch_and
-
-#define atomic_fetch_andnot_relaxedatomic_fetch_andnot_relaxed
-#define atomic_fetch_andnot_acquireatomic_fetch_andnot_acquire
-#define atomic_fetch_andnot_releaseatomic_fetch_andnot_release
-#define atomic_fetch_andnotatomic_fetch_andnot
-
-#define atomic_fetch_or_relaxedatomic_fetch_or_relaxed
-#define atomic_fetch_or_acquireatomic_fetch_or_acquire
-#define atomic_fetch_or_releaseatomic_fetch_or_release
-#define atomic_fetch_oratomic_fetch_or
-
-#define atomic_fetch_xor_relaxed   atomic_fetch_xor_relaxed
-#define atomic_fetch_xor_acquire   atomic_fetch_xor_acquire
-#define atomic_fetch_xor_release   atomic_fetch_xor_release
-#define atomic_fetch_xor   atomic_fetch_xor
-
-#define atomic_xchg_relaxed(v, new)xchg_relaxed(&((v)->counter), (new))
-#define atomic_xchg_acquire(v, new)xchg_acquire(&((v)->counter), (new))
-#define atomic_xchg_release(v, new)xchg_release(&((v)->counter), (new))
-#define atomic_xchg(v, new)xchg(&((v)->counter), (new))
-
-#define atomic_cmpxchg_relaxed(v, old, new)\
-   cmpxchg_relaxed(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_acquire(v, old, new)\
-   cmpxchg_acquire(&((v)->counter), (old), (new))
-#define atomic_cmpxchg_release(v, old, new)\
-   cmpxchg_release(&((v)->counter), (old), (new))
-#define atomic_cmpxchg(v, old, new)cmpxchg(&((v)->counter), (old), (new))
-
-#define atomic_andnot  atomic_andnot
+#define 

[tip:locking/core] locking/atomics: Check generated headers are up-to-date

[tip-bot for Mark Rutland]

Commit-ID:  85f8507192fbfb4ad2ac01de879cb50045f4247f
Gitweb: https://git.kernel.org/tip/85f8507192fbfb4ad2ac01de879cb50045f4247f
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:29 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Check generated headers are up-to-date

Now that all the generated atomic headers are in place, it would be good
to ensure that:

a) the headers are up-to-date when scripting changes.

b) developers don't directly modify the generated headers.

To ensure both of these properties, let's add a Kbuild step to check
that the generated headers are up-to-date.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-6-mark.rutl...@arm.com
---
 Kbuild  | 18 --
 scripts/atomic/check-atomics.sh | 19 +++
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/Kbuild b/Kbuild
index 005304205482..47c9fe175bd9 100644
--- a/Kbuild
+++ b/Kbuild
@@ -6,7 +6,8 @@
 # 2) Generate timeconst.h
 # 3) Generate asm-offsets.h (may need bounds.h and timeconst.h)
 # 4) Check for missing system calls
-# 5) Generate constants.py (may need bounds.h)
+# 5) check atomics headers are up-to-date
+# 6) Generate constants.py (may need bounds.h)
 
 #
 # 1) Generate bounds.h
@@ -72,7 +73,20 @@ missing-syscalls: scripts/checksyscalls.sh $(offsets-file) 
FORCE
$(call cmd,syscalls)
 
 #
-# 5) Generate constants for Python GDB integration
+# 5) Check atomic headers are up-to-date
+#
+
+always += old-atomics
+targets += old-atomics
+
+quiet_cmd_atomics = CALL$<
+  cmd_atomics = $(CONFIG_SHELL) scripts/atomic/check-atomics.sh
+
+old-atomics: scripts/atomic/check-atomics.sh FORCE
+   $(call cmd,atomics)
+
+#
+# 6) Generate constants for Python GDB integration
 #
 
 extra-$(CONFIG_GDB_SCRIPTS) += build_constants_py
diff --git a/scripts/atomic/check-atomics.sh b/scripts/atomic/check-atomics.sh
new file mode 100644
index ..c30101cddf2d
--- /dev/null
+++ b/scripts/atomic/check-atomics.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check if atomic headers are up-to-date
+
+ATOMICDIR=$(dirname $0)
+ATOMICTBL=${ATOMICDIR}/atomics.tbl
+LINUXDIR=${ATOMICDIR}/../..
+
+cat < /dev/null); then
+   printf "warning: include/${header} is out-of-date.\n"
+   fi
+done

[tip:locking/core] locking/atomics: Switch to generated instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  7c51ca8b6495e52b98ae505c60f4cb0ebfe3f58c
Gitweb: https://git.kernel.org/tip/7c51ca8b6495e52b98ae505c60f4cb0ebfe3f58c
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:28 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Switch to generated instrumentation

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-instrumented.h, using the same table
used to generate the fallbacks and atomic-long wrappers.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Generating the atomic headers means that the instrumented wrappers will
remain in sync with the rest of the atomic APIs, and we gain all the
ordering variants of each atomic without having to manually expanded
them all.

The KASAN checks are automatically generated based on the function
parameters defined in atomics.tbl. Note that try_cmpxchg() now correctly
treats 'old' as a parameter that may be written to, and not only read as
the hand-written instrumentation assumed.

Other than the change to try_cmpxchg(), existing code should not be
affected by this patch. The patch introduces instrumentation for all
optional atomics (and ordering variants), along with the ifdeffery this
requires, enabling other architectures to make use of the instrumented
atomics.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: Dmitry Vyukov 
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: Andrey Ryabinin 
Cc: Alexander Potapenko 
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-5-mark.rutl...@arm.com
---
 include/asm-generic/atomic-instrumented.h | 1688 +
 1 file changed, 1504 insertions(+), 184 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 0d4b1d3dbc1e..b8f5b35216e1 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -1,3 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-instrumented.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 /*
  * This file provides wrappers with KASAN instrumentation for atomic 
operations.
  * To use this functionality an arch's atomic.h file needs to define all
@@ -9,459 +14,1774 @@
  * arch_ variants (i.e. arch_atomic_read()/arch_atomic_cmpxchg()) to avoid
  * double instrumentation.
  */
+#ifndef _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+#define _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+
+#include 
+#include 
+
+static inline int
+atomic_read(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read(v);
+}
+#define atomic_read atomic_read
+
+#if defined(arch_atomic_read_acquire)
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read_acquire(v);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+static inline void
+atomic_set(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set(v, i);
+}
+#define atomic_set atomic_set
+
+#if defined(arch_atomic_set_release)
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set_release(v, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+static inline void
+atomic_add(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_add(i, v);
+}
+#define atomic_add atomic_add
+
+#if !defined(arch_atomic_add_return_relaxed) || defined(arch_atomic_add_return)
+static inline int
+atomic_add_return(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return(i, v);
+}
+#define atomic_add_return atomic_add_return
+#endif
+
+#if defined(arch_atomic_add_return_acquire)
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_acquire(i, v);
+}
+#define atomic_add_return_acquire atomic_add_return_acquire
+#endif
+
+#if defined(arch_atomic_add_return_release)
+static inline int
+atomic_add_return_release(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_release(i, v);
+}
+#define atomic_add_return_release 

[tip:locking/core] locking/atomics: Check generated headers are up-to-date

[tip-bot for Mark Rutland]

Commit-ID:  85f8507192fbfb4ad2ac01de879cb50045f4247f
Gitweb: https://git.kernel.org/tip/85f8507192fbfb4ad2ac01de879cb50045f4247f
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:29 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Check generated headers are up-to-date

Now that all the generated atomic headers are in place, it would be good
to ensure that:

a) the headers are up-to-date when scripting changes.

b) developers don't directly modify the generated headers.

To ensure both of these properties, let's add a Kbuild step to check
that the generated headers are up-to-date.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-6-mark.rutl...@arm.com
---
 Kbuild  | 18 --
 scripts/atomic/check-atomics.sh | 19 +++
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/Kbuild b/Kbuild
index 005304205482..47c9fe175bd9 100644
--- a/Kbuild
+++ b/Kbuild
@@ -6,7 +6,8 @@
 # 2) Generate timeconst.h
 # 3) Generate asm-offsets.h (may need bounds.h and timeconst.h)
 # 4) Check for missing system calls
-# 5) Generate constants.py (may need bounds.h)
+# 5) check atomics headers are up-to-date
+# 6) Generate constants.py (may need bounds.h)
 
 #
 # 1) Generate bounds.h
@@ -72,7 +73,20 @@ missing-syscalls: scripts/checksyscalls.sh $(offsets-file) 
FORCE
$(call cmd,syscalls)
 
 #
-# 5) Generate constants for Python GDB integration
+# 5) Check atomic headers are up-to-date
+#
+
+always += old-atomics
+targets += old-atomics
+
+quiet_cmd_atomics = CALL$<
+  cmd_atomics = $(CONFIG_SHELL) scripts/atomic/check-atomics.sh
+
+old-atomics: scripts/atomic/check-atomics.sh FORCE
+   $(call cmd,atomics)
+
+#
+# 6) Generate constants for Python GDB integration
 #
 
 extra-$(CONFIG_GDB_SCRIPTS) += build_constants_py
diff --git a/scripts/atomic/check-atomics.sh b/scripts/atomic/check-atomics.sh
new file mode 100644
index ..c30101cddf2d
--- /dev/null
+++ b/scripts/atomic/check-atomics.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check if atomic headers are up-to-date
+
+ATOMICDIR=$(dirname $0)
+ATOMICTBL=${ATOMICDIR}/atomics.tbl
+LINUXDIR=${ATOMICDIR}/../..
+
+cat < /dev/null); then
+   printf "warning: include/${header} is out-of-date.\n"
+   fi
+done

[tip:locking/core] locking/atomics: Switch to generated instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  7c51ca8b6495e52b98ae505c60f4cb0ebfe3f58c
Gitweb: https://git.kernel.org/tip/7c51ca8b6495e52b98ae505c60f4cb0ebfe3f58c
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:28 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Switch to generated instrumentation

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-instrumented.h, using the same table
used to generate the fallbacks and atomic-long wrappers.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Generating the atomic headers means that the instrumented wrappers will
remain in sync with the rest of the atomic APIs, and we gain all the
ordering variants of each atomic without having to manually expanded
them all.

The KASAN checks are automatically generated based on the function
parameters defined in atomics.tbl. Note that try_cmpxchg() now correctly
treats 'old' as a parameter that may be written to, and not only read as
the hand-written instrumentation assumed.

Other than the change to try_cmpxchg(), existing code should not be
affected by this patch. The patch introduces instrumentation for all
optional atomics (and ordering variants), along with the ifdeffery this
requires, enabling other architectures to make use of the instrumented
atomics.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: Dmitry Vyukov 
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: Andrey Ryabinin 
Cc: Alexander Potapenko 
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-5-mark.rutl...@arm.com
---
 include/asm-generic/atomic-instrumented.h | 1688 +
 1 file changed, 1504 insertions(+), 184 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 0d4b1d3dbc1e..b8f5b35216e1 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -1,3 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-instrumented.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 /*
  * This file provides wrappers with KASAN instrumentation for atomic 
operations.
  * To use this functionality an arch's atomic.h file needs to define all
@@ -9,459 +14,1774 @@
  * arch_ variants (i.e. arch_atomic_read()/arch_atomic_cmpxchg()) to avoid
  * double instrumentation.
  */
+#ifndef _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+#define _ASM_GENERIC_ATOMIC_INSTRUMENTED_H
+
+#include 
+#include 
+
+static inline int
+atomic_read(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read(v);
+}
+#define atomic_read atomic_read
+
+#if defined(arch_atomic_read_acquire)
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   kasan_check_read(v, sizeof(*v));
+   return arch_atomic_read_acquire(v);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+static inline void
+atomic_set(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set(v, i);
+}
+#define atomic_set atomic_set
+
+#if defined(arch_atomic_set_release)
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_set_release(v, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+static inline void
+atomic_add(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   arch_atomic_add(i, v);
+}
+#define atomic_add atomic_add
+
+#if !defined(arch_atomic_add_return_relaxed) || defined(arch_atomic_add_return)
+static inline int
+atomic_add_return(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return(i, v);
+}
+#define atomic_add_return atomic_add_return
+#endif
+
+#if defined(arch_atomic_add_return_acquire)
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_acquire(i, v);
+}
+#define atomic_add_return_acquire atomic_add_return_acquire
+#endif
+
+#if defined(arch_atomic_add_return_release)
+static inline int
+atomic_add_return_release(int i, atomic_t *v)
+{
+   kasan_check_write(v, sizeof(*v));
+   return arch_atomic_add_return_release(i, v);
+}
+#define atomic_add_return_release 

[tip:locking/core] locking/atomics: Switch to generated atomic-long

[tip-bot for Mark Rutland]

Commit-ID:  acf81221804111bec24b399c9f44c28a5a3c9367
Gitweb: https://git.kernel.org/tip/acf81221804111bec24b399c9f44c28a5a3c9367
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:27 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Switch to generated atomic-long

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-long.h, using the same table that
gen-atomic-fallbacks.h uses to fill in gaps in the atomic_* and
atomic64_* APIs.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Other than *_INIT() and *_cond_read_acquire(), all API functions are
implemented as static inline C functions, ensuring consistent type
promotion and/or truncation without requiring explicit casts to be
applied to parameters or return values.

Since we typedef atomic_long_t to either atomic_t or atomic64_t, we know
these types are equivalent, and don't require explicit casts between
them. However, as the try_cmpxchg*() functions take a pointer for the
'old' parameter, which may be an int or s64, an explicit cast is
generated for this.

There should be no functional change as a result of this patch (i.e.
existing code should not be affected). However, this introduces a number
of functions into the atomic_long_* API, bringing it into line with the
atomic_* and atomic64_* APIs.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-4-mark.rutl...@arm.com
---
 include/asm-generic/atomic-long.h | 1173 ++---
 1 file changed, 958 insertions(+), 215 deletions(-)

diff --git a/include/asm-generic/atomic-long.h 
b/include/asm-generic/atomic-long.h
index 87d14476edc2..a833d385a70b 100644
--- a/include/asm-generic/atomic-long.h
+++ b/include/asm-generic/atomic-long.h
@@ -1,269 +1,1012 @@
-/* SPDX-License-Identifier: GPL-2.0 */
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-long.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 #ifndef _ASM_GENERIC_ATOMIC_LONG_H
 #define _ASM_GENERIC_ATOMIC_LONG_H
-/*
- * Copyright (C) 2005 Silicon Graphics, Inc.
- * Christoph Lameter
- *
- * Allows to provide arch independent atomic definitions without the need to
- * edit all arch specific atomic.h files.
- */
 
 #include 
 
-/*
- * Suppport for atomic_long_t
- *
- * Casts for parameters are avoided for existing atomic functions in order to
- * avoid issues with cast-as-lval under gcc 4.x and other limitations that the
- * macros of a platform may have.
- */
+#ifdef CONFIG_64BIT
+typedef atomic64_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
+#define atomic_long_cond_read_acquire  atomic64_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic64_cond_read_relaxed
+#else
+typedef atomic_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
+#define atomic_long_cond_read_acquire  atomic_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic_cond_read_relaxed
+#endif
 
-#if BITS_PER_LONG == 64
+#ifdef CONFIG_64BIT
 
-typedef atomic64_t atomic_long_t;
+static inline long
+atomic_long_read(const atomic_long_t *v)
+{
+   return atomic64_read(v);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic64 ## x
-#define ATOMIC_LONG_TYPE   s64
+static inline long
+atomic_long_read_acquire(const atomic_long_t *v)
+{
+   return atomic64_read_acquire(v);
+}
 
-#else
+static inline void
+atomic_long_set(atomic_long_t *v, long i)
+{
+   atomic64_set(v, i);
+}
 
-typedef atomic_t atomic_long_t;
+static inline void
+atomic_long_set_release(atomic_long_t *v, long i)
+{
+   atomic64_set_release(v, i);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic ## x
-#define ATOMIC_LONG_TYPE   int
+static inline void
+atomic_long_add(long i, atomic_long_t *v)
+{
+   atomic64_add(i, v);
+}
 
-#endif
+static inline long
+atomic_long_add_return(long i, atomic_long_t *v)
+{
+   return atomic64_add_return(i, v);
+}
+
+static inline long
+atomic_long_add_return_acquire(long i, atomic_long_t *v)
+{
+   return 

[tip:locking/core] locking/atomics: Switch to generated fallbacks

[tip-bot for Mark Rutland]

Commit-ID:  1ff498091a527f636e8ce10b64d789371f8bebd9
Gitweb: https://git.kernel.org/tip/1ff498091a527f636e8ce10b64d789371f8bebd9
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:26 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Switch to generated fallbacks

As a step to ensuring the atomic* APIs are consistent, switch to fallbacks
generated by gen-atomic-fallback.sh.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

The new fallback header should be equivalent to the old fallbacks in
, but:

* It is formatted a little differently due to scripting ensuring things
  are more regular than they used to be.

* Fallbacks are now expanded in-place as static inline functions rather
  than macros.

* The prototypes for fallbacks are arragned consistently with the return
  type on a separate line to try to keep to a sensible line length.

There should be no functional change as a result of this patch.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-3-mark.rutl...@arm.com
---
 include/linux/atomic-fallback.h | 2294 +++
 include/linux/atomic.h  | 1241 +
 2 files changed, 2295 insertions(+), 1240 deletions(-)

diff --git a/include/linux/atomic-fallback.h b/include/linux/atomic-fallback.h
new file mode 100644
index ..1c02c0112fbb
--- /dev/null
+++ b/include/linux/atomic-fallback.h
@@ -0,0 +1,2294 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-fallback.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
+#ifndef _LINUX_ATOMIC_FALLBACK_H
+#define _LINUX_ATOMIC_FALLBACK_H
+
+#ifndef xchg_relaxed
+#define xchg_relaxed   xchg
+#define xchg_acquire   xchg
+#define xchg_release   xchg
+#else /* xchg_relaxed */
+
+#ifndef xchg_acquire
+#define xchg_acquire(...) \
+   __atomic_op_acquire(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg_release
+#define xchg_release(...) \
+   __atomic_op_release(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg
+#define xchg(...) \
+   __atomic_op_fence(xchg, __VA_ARGS__)
+#endif
+
+#endif /* xchg_relaxed */
+
+#ifndef cmpxchg_relaxed
+#define cmpxchg_relaxedcmpxchg
+#define cmpxchg_acquirecmpxchg
+#define cmpxchg_releasecmpxchg
+#else /* cmpxchg_relaxed */
+
+#ifndef cmpxchg_acquire
+#define cmpxchg_acquire(...) \
+   __atomic_op_acquire(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg_release
+#define cmpxchg_release(...) \
+   __atomic_op_release(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg
+#define cmpxchg(...) \
+   __atomic_op_fence(cmpxchg, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg_relaxed */
+
+#ifndef cmpxchg64_relaxed
+#define cmpxchg64_relaxed  cmpxchg64
+#define cmpxchg64_acquire  cmpxchg64
+#define cmpxchg64_release  cmpxchg64
+#else /* cmpxchg64_relaxed */
+
+#ifndef cmpxchg64_acquire
+#define cmpxchg64_acquire(...) \
+   __atomic_op_acquire(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64_release
+#define cmpxchg64_release(...) \
+   __atomic_op_release(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64
+#define cmpxchg64(...) \
+   __atomic_op_fence(cmpxchg64, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg64_relaxed */
+
+#ifndef atomic_read_acquire
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   return smp_load_acquire(&(v)->counter);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+#ifndef atomic_set_release
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   smp_store_release(&(v)->counter, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+#ifndef atomic_add_return_relaxed
+#define atomic_add_return_acquire atomic_add_return
+#define atomic_add_return_release atomic_add_return
+#define atomic_add_return_relaxed atomic_add_return
+#else /* atomic_add_return_relaxed */
+
+#ifndef atomic_add_return_acquire
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   int ret = atomic_add_return_relaxed(i, v);
+   __atomic_acquire_fence();
+   return ret;
+}
+#define 

[tip:locking/core] locking/atomics: Switch to generated atomic-long

[tip-bot for Mark Rutland]

Commit-ID:  acf81221804111bec24b399c9f44c28a5a3c9367
Gitweb: https://git.kernel.org/tip/acf81221804111bec24b399c9f44c28a5a3c9367
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:27 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Switch to generated atomic-long

As a step towards ensuring the atomic* APIs are consistent, let's switch
to wrappers generated by gen-atomic-long.h, using the same table that
gen-atomic-fallbacks.h uses to fill in gaps in the atomic_* and
atomic64_* APIs.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

Other than *_INIT() and *_cond_read_acquire(), all API functions are
implemented as static inline C functions, ensuring consistent type
promotion and/or truncation without requiring explicit casts to be
applied to parameters or return values.

Since we typedef atomic_long_t to either atomic_t or atomic64_t, we know
these types are equivalent, and don't require explicit casts between
them. However, as the try_cmpxchg*() functions take a pointer for the
'old' parameter, which may be an int or s64, an explicit cast is
generated for this.

There should be no functional change as a result of this patch (i.e.
existing code should not be affected). However, this introduces a number
of functions into the atomic_long_* API, bringing it into line with the
atomic_* and atomic64_* APIs.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: Arnd Bergmann 
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-4-mark.rutl...@arm.com
---
 include/asm-generic/atomic-long.h | 1173 ++---
 1 file changed, 958 insertions(+), 215 deletions(-)

diff --git a/include/asm-generic/atomic-long.h 
b/include/asm-generic/atomic-long.h
index 87d14476edc2..a833d385a70b 100644
--- a/include/asm-generic/atomic-long.h
+++ b/include/asm-generic/atomic-long.h
@@ -1,269 +1,1012 @@
-/* SPDX-License-Identifier: GPL-2.0 */
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-long.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
 #ifndef _ASM_GENERIC_ATOMIC_LONG_H
 #define _ASM_GENERIC_ATOMIC_LONG_H
-/*
- * Copyright (C) 2005 Silicon Graphics, Inc.
- * Christoph Lameter
- *
- * Allows to provide arch independent atomic definitions without the need to
- * edit all arch specific atomic.h files.
- */
 
 #include 
 
-/*
- * Suppport for atomic_long_t
- *
- * Casts for parameters are avoided for existing atomic functions in order to
- * avoid issues with cast-as-lval under gcc 4.x and other limitations that the
- * macros of a platform may have.
- */
+#ifdef CONFIG_64BIT
+typedef atomic64_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
+#define atomic_long_cond_read_acquire  atomic64_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic64_cond_read_relaxed
+#else
+typedef atomic_t atomic_long_t;
+#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
+#define atomic_long_cond_read_acquire  atomic_cond_read_acquire
+#define atomic_long_cond_read_relaxed  atomic_cond_read_relaxed
+#endif
 
-#if BITS_PER_LONG == 64
+#ifdef CONFIG_64BIT
 
-typedef atomic64_t atomic_long_t;
+static inline long
+atomic_long_read(const atomic_long_t *v)
+{
+   return atomic64_read(v);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC64_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic64 ## x
-#define ATOMIC_LONG_TYPE   s64
+static inline long
+atomic_long_read_acquire(const atomic_long_t *v)
+{
+   return atomic64_read_acquire(v);
+}
 
-#else
+static inline void
+atomic_long_set(atomic_long_t *v, long i)
+{
+   atomic64_set(v, i);
+}
 
-typedef atomic_t atomic_long_t;
+static inline void
+atomic_long_set_release(atomic_long_t *v, long i)
+{
+   atomic64_set_release(v, i);
+}
 
-#define ATOMIC_LONG_INIT(i)ATOMIC_INIT(i)
-#define ATOMIC_LONG_PFX(x) atomic ## x
-#define ATOMIC_LONG_TYPE   int
+static inline void
+atomic_long_add(long i, atomic_long_t *v)
+{
+   atomic64_add(i, v);
+}
 
-#endif
+static inline long
+atomic_long_add_return(long i, atomic_long_t *v)
+{
+   return atomic64_add_return(i, v);
+}
+
+static inline long
+atomic_long_add_return_acquire(long i, atomic_long_t *v)
+{
+   return 

[tip:locking/core] locking/atomics: Switch to generated fallbacks

[tip-bot for Mark Rutland]

Commit-ID:  1ff498091a527f636e8ce10b64d789371f8bebd9
Gitweb: https://git.kernel.org/tip/1ff498091a527f636e8ce10b64d789371f8bebd9
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:26 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:19 +0100

locking/atomics: Switch to generated fallbacks

As a step to ensuring the atomic* APIs are consistent, switch to fallbacks
generated by gen-atomic-fallback.sh.

These are checked in rather than generated with Kbuild, since:

* This allows inspection of the atomics with git grep and ctags on a
  pristine tree, which Linus strongly prefers being able to do.

* The fallbacks are not affected by machine details or configuration
  options, so it is not necessary to regenerate them to take these into
  account.

* These are included by files required *very* early in the build process
  (e.g. for generating bounds.h), and we'd rather not complicate the
  top-level Kbuild file with dependencies.

The new fallback header should be equivalent to the old fallbacks in
, but:

* It is formatted a little differently due to scripting ensuring things
  are more regular than they used to be.

* Fallbacks are now expanded in-place as static inline functions rather
  than macros.

* The prototypes for fallbacks are arragned consistently with the return
  type on a separate line to try to keep to a sensible line length.

There should be no functional change as a result of this patch.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-3-mark.rutl...@arm.com
---
 include/linux/atomic-fallback.h | 2294 +++
 include/linux/atomic.h  | 1241 +
 2 files changed, 2295 insertions(+), 1240 deletions(-)

diff --git a/include/linux/atomic-fallback.h b/include/linux/atomic-fallback.h
new file mode 100644
index ..1c02c0112fbb
--- /dev/null
+++ b/include/linux/atomic-fallback.h
@@ -0,0 +1,2294 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Generated by scripts/atomic/gen-atomic-fallback.sh
+// DO NOT MODIFY THIS FILE DIRECTLY
+
+#ifndef _LINUX_ATOMIC_FALLBACK_H
+#define _LINUX_ATOMIC_FALLBACK_H
+
+#ifndef xchg_relaxed
+#define xchg_relaxed   xchg
+#define xchg_acquire   xchg
+#define xchg_release   xchg
+#else /* xchg_relaxed */
+
+#ifndef xchg_acquire
+#define xchg_acquire(...) \
+   __atomic_op_acquire(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg_release
+#define xchg_release(...) \
+   __atomic_op_release(xchg, __VA_ARGS__)
+#endif
+
+#ifndef xchg
+#define xchg(...) \
+   __atomic_op_fence(xchg, __VA_ARGS__)
+#endif
+
+#endif /* xchg_relaxed */
+
+#ifndef cmpxchg_relaxed
+#define cmpxchg_relaxedcmpxchg
+#define cmpxchg_acquirecmpxchg
+#define cmpxchg_releasecmpxchg
+#else /* cmpxchg_relaxed */
+
+#ifndef cmpxchg_acquire
+#define cmpxchg_acquire(...) \
+   __atomic_op_acquire(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg_release
+#define cmpxchg_release(...) \
+   __atomic_op_release(cmpxchg, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg
+#define cmpxchg(...) \
+   __atomic_op_fence(cmpxchg, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg_relaxed */
+
+#ifndef cmpxchg64_relaxed
+#define cmpxchg64_relaxed  cmpxchg64
+#define cmpxchg64_acquire  cmpxchg64
+#define cmpxchg64_release  cmpxchg64
+#else /* cmpxchg64_relaxed */
+
+#ifndef cmpxchg64_acquire
+#define cmpxchg64_acquire(...) \
+   __atomic_op_acquire(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64_release
+#define cmpxchg64_release(...) \
+   __atomic_op_release(cmpxchg64, __VA_ARGS__)
+#endif
+
+#ifndef cmpxchg64
+#define cmpxchg64(...) \
+   __atomic_op_fence(cmpxchg64, __VA_ARGS__)
+#endif
+
+#endif /* cmpxchg64_relaxed */
+
+#ifndef atomic_read_acquire
+static inline int
+atomic_read_acquire(const atomic_t *v)
+{
+   return smp_load_acquire(&(v)->counter);
+}
+#define atomic_read_acquire atomic_read_acquire
+#endif
+
+#ifndef atomic_set_release
+static inline void
+atomic_set_release(atomic_t *v, int i)
+{
+   smp_store_release(&(v)->counter, i);
+}
+#define atomic_set_release atomic_set_release
+#endif
+
+#ifndef atomic_add_return_relaxed
+#define atomic_add_return_acquire atomic_add_return
+#define atomic_add_return_release atomic_add_return
+#define atomic_add_return_relaxed atomic_add_return
+#else /* atomic_add_return_relaxed */
+
+#ifndef atomic_add_return_acquire
+static inline int
+atomic_add_return_acquire(int i, atomic_t *v)
+{
+   int ret = atomic_add_return_relaxed(i, v);
+   __atomic_acquire_fence();
+   return ret;
+}
+#define 

[tip:locking/core] locking/atomics: Add common header generation files

[tip-bot for Mark Rutland]

Commit-ID:  d95152ec1193bf7597497a2ed8184f79a531323c
Gitweb: https://git.kernel.org/tip/d95152ec1193bf7597497a2ed8184f79a531323c
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:25 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:18 +0100

locking/atomics: Add common header generation files

To minimize repetition, to allow for future rework, and to ensure
regularity of the various atomic APIs, we'd like to automatically
generate (the bulk of) a number of headers related to atomics.

This patch adds the infrastructure to do so, leaving actual conversion
of headers to subsequent patches. This infrastructure consists of:

* atomics.tbl - a table describing the functions in the atomics API,
  with names, prototypes, and metadata describing the variants that
  exist (e.g fetch/return, acquire/release/relaxed). Note that the
  return type is dependent on the particular variant.

* atomic-tbl.sh - a library of routines useful for dealing with
  atomics.tbl (e.g. querying which variants exist, or generating
  argument/parameter lists for a given function variant).

* gen-atomic-fallback.sh - a script which generates a header of
  fallbacks, covering cases where architecture omit certain functions
  (e.g. omitting relaxed variants).

* gen-atomic-long.sh - a script which generates wrappers providing the
  atomic_long API atomic of the relevant atomic or atomic64 API,
  ensuring the APIs are consistent.

* gen-atomic-instrumented.sh - a script which generates atomic* wrappers
  atop of arch_atomic* functions, with automatically generated KASAN
  instrumentation.

* fallbacks/* - a set of fallback implementations for atomics, which
  should be used when no implementation of a given atomic is provided.
  These are used by gen-atomic-fallback.sh to generate fallbacks, and
  these are also used by other scripts to determine the set of optional
  atomics (as required to generate preprocessor guards correctly).

  Fallbacks may use the following variables:

  ${atomic} atomic prefix: atomic/atomic64/atomic_long, which can be
used to derive the atomic type, and to prefix functions

  ${int}integer type: int/s64/long

  ${pfx}variant prefix, e.g. fetch_

  ${name}   base function name, e.g. add

  ${sfx}variant suffix, e.g. _return

  ${order}  order suffix, e.g. _relaxed

  ${atomicname} full name, e.g. atomic64_fetch_add_relaxed

  ${ret}return type of the function, e.g. void

  ${retstmt}a return statement (with a trailing space), unless the
variant returns void

  ${params} parameter list for the function declaration, e.g.
"int i, atomic_t *v"

  ${args}   argument list for invoking the function, e.g. "i, v"

  ... for clarity, ${ret}, ${retstmt}, ${params}, and ${args} are
  open-coded for fallbacks where these do not vary, or are critical to
  understanding the logic of the fallback.

The MAINTAINERS entry for the atomic infrastructure is updated to cover
the new scripts.

There should be no functional change as a result of this patch.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-2-mark.rutl...@arm.com
---
 MAINTAINERS  |   1 +
 scripts/atomic/atomic-tbl.sh | 186 +++
 scripts/atomic/atomics.tbl   |  41 ++
 scripts/atomic/fallbacks/acquire |   9 ++
 scripts/atomic/fallbacks/add_negative|  16 +++
 scripts/atomic/fallbacks/add_unless  |  16 +++
 scripts/atomic/fallbacks/andnot  |   7 +
 scripts/atomic/fallbacks/dec |   7 +
 scripts/atomic/fallbacks/dec_and_test|  15 +++
 scripts/atomic/fallbacks/dec_if_positive |  15 +++
 scripts/atomic/fallbacks/dec_unless_positive |  14 ++
 scripts/atomic/fallbacks/fence   |  11 ++
 scripts/atomic/fallbacks/fetch_add_unless|  23 
 scripts/atomic/fallbacks/inc |   7 +
 scripts/atomic/fallbacks/inc_and_test|  15 +++
 scripts/atomic/fallbacks/inc_not_zero|  14 ++
 scripts/atomic/fallbacks/inc_unless_negative |  14 ++
 scripts/atomic/fallbacks/read_acquire|   7 +
 scripts/atomic/fallbacks/release |   8 ++
 scripts/atomic/fallbacks/set_release |   7 +
 scripts/atomic/fallbacks/sub_and_test|  16 +++
 scripts/atomic/fallbacks/try_cmpxchg |  11 ++
 scripts/atomic/gen-atomic-fallback.sh| 181 ++
 scripts/atomic/gen-atomic-instrumented.sh| 182 ++
 scripts/atomic/gen-atomic-long.sh| 101 

[tip:locking/core] locking/atomics: Add common header generation files

[tip-bot for Mark Rutland]

Commit-ID:  d95152ec1193bf7597497a2ed8184f79a531323c
Gitweb: https://git.kernel.org/tip/d95152ec1193bf7597497a2ed8184f79a531323c
Author: Mark Rutland 
AuthorDate: Tue, 4 Sep 2018 11:48:25 +0100
Committer:  Peter Zijlstra 
CommitDate: Thu, 1 Nov 2018 10:48:18 +0100

locking/atomics: Add common header generation files

To minimize repetition, to allow for future rework, and to ensure
regularity of the various atomic APIs, we'd like to automatically
generate (the bulk of) a number of headers related to atomics.

This patch adds the infrastructure to do so, leaving actual conversion
of headers to subsequent patches. This infrastructure consists of:

* atomics.tbl - a table describing the functions in the atomics API,
  with names, prototypes, and metadata describing the variants that
  exist (e.g fetch/return, acquire/release/relaxed). Note that the
  return type is dependent on the particular variant.

* atomic-tbl.sh - a library of routines useful for dealing with
  atomics.tbl (e.g. querying which variants exist, or generating
  argument/parameter lists for a given function variant).

* gen-atomic-fallback.sh - a script which generates a header of
  fallbacks, covering cases where architecture omit certain functions
  (e.g. omitting relaxed variants).

* gen-atomic-long.sh - a script which generates wrappers providing the
  atomic_long API atomic of the relevant atomic or atomic64 API,
  ensuring the APIs are consistent.

* gen-atomic-instrumented.sh - a script which generates atomic* wrappers
  atop of arch_atomic* functions, with automatically generated KASAN
  instrumentation.

* fallbacks/* - a set of fallback implementations for atomics, which
  should be used when no implementation of a given atomic is provided.
  These are used by gen-atomic-fallback.sh to generate fallbacks, and
  these are also used by other scripts to determine the set of optional
  atomics (as required to generate preprocessor guards correctly).

  Fallbacks may use the following variables:

  ${atomic} atomic prefix: atomic/atomic64/atomic_long, which can be
used to derive the atomic type, and to prefix functions

  ${int}integer type: int/s64/long

  ${pfx}variant prefix, e.g. fetch_

  ${name}   base function name, e.g. add

  ${sfx}variant suffix, e.g. _return

  ${order}  order suffix, e.g. _relaxed

  ${atomicname} full name, e.g. atomic64_fetch_add_relaxed

  ${ret}return type of the function, e.g. void

  ${retstmt}a return statement (with a trailing space), unless the
variant returns void

  ${params} parameter list for the function declaration, e.g.
"int i, atomic_t *v"

  ${args}   argument list for invoking the function, e.g. "i, v"

  ... for clarity, ${ret}, ${retstmt}, ${params}, and ${args} are
  open-coded for fallbacks where these do not vary, or are critical to
  understanding the logic of the fallback.

The MAINTAINERS entry for the atomic infrastructure is updated to cover
the new scripts.

There should be no functional change as a result of this patch.

Cc: linux-arm-ker...@lists.infradead.org
Cc: catalin.mari...@arm.com
Cc: Ingo Molnar 
Cc: Will Deacon 
Cc: linuxdriv...@attotech.com
Cc: dvyu...@google.com
Cc: Boqun Feng 
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: gli...@google.com
Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Signed-off-by: Peter Zijlstra (Intel) 
Link: http://lkml.kernel.org/r/20180904104830.2975-2-mark.rutl...@arm.com
---
 MAINTAINERS  |   1 +
 scripts/atomic/atomic-tbl.sh | 186 +++
 scripts/atomic/atomics.tbl   |  41 ++
 scripts/atomic/fallbacks/acquire |   9 ++
 scripts/atomic/fallbacks/add_negative|  16 +++
 scripts/atomic/fallbacks/add_unless  |  16 +++
 scripts/atomic/fallbacks/andnot  |   7 +
 scripts/atomic/fallbacks/dec |   7 +
 scripts/atomic/fallbacks/dec_and_test|  15 +++
 scripts/atomic/fallbacks/dec_if_positive |  15 +++
 scripts/atomic/fallbacks/dec_unless_positive |  14 ++
 scripts/atomic/fallbacks/fence   |  11 ++
 scripts/atomic/fallbacks/fetch_add_unless|  23 
 scripts/atomic/fallbacks/inc |   7 +
 scripts/atomic/fallbacks/inc_and_test|  15 +++
 scripts/atomic/fallbacks/inc_not_zero|  14 ++
 scripts/atomic/fallbacks/inc_unless_negative |  14 ++
 scripts/atomic/fallbacks/read_acquire|   7 +
 scripts/atomic/fallbacks/release |   8 ++
 scripts/atomic/fallbacks/set_release |   7 +
 scripts/atomic/fallbacks/sub_and_test|  16 +++
 scripts/atomic/fallbacks/try_cmpxchg |  11 ++
 scripts/atomic/gen-atomic-fallback.sh| 181 ++
 scripts/atomic/gen-atomic-instrumented.sh| 182 ++
 scripts/atomic/gen-atomic-long.sh| 101 

[tip:locking/core] locking/atomics: Rework ordering barriers

[tip-bot for Mark Rutland]

Commit-ID:  fd2efaa4eb5317c3a86357a83a7d456a1b86a0ac
Gitweb: https://git.kernel.org/tip/fd2efaa4eb5317c3a86357a83a7d456a1b86a0ac
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:11 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:59 +0200

locking/atomics: Rework ordering barriers

Currently architectures can override __atomic_op_*() to define the barriers
used before/after a relaxed atomic when used to build acquire/release/fence
variants.

This has the unfortunate property of requiring the architecture to define the
full wrapper for the atomics, rather than just the barriers they care about,
and gets in the way of generating atomics which can be easily read.

Instead, this patch has architectures define an optional set of barriers:

* __atomic_acquire_fence()
* __atomic_release_fence()
* __atomic_pre_full_fence()
* __atomic_post_full_fence()

... which  uses to build the wrappers.

It would be nice if we could undef these, along with the __atomic_op_*()
wrappers, but that would break the cmpxchg() wrappers, which are written
in preprocessor. Undefs would have been nice, but alas.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Andrea Parri 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: dvyu...@google.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-7-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/alpha/include/asm/atomic.h   |  8 
 arch/powerpc/include/asm/atomic.h | 17 +
 arch/riscv/include/asm/atomic.h   | 17 +
 include/linux/atomic.h| 38 ++
 4 files changed, 36 insertions(+), 44 deletions(-)

diff --git a/arch/alpha/include/asm/atomic.h b/arch/alpha/include/asm/atomic.h
index 4a6a8f58c9c9..150a1c5d6a2c 100644
--- a/arch/alpha/include/asm/atomic.h
+++ b/arch/alpha/include/asm/atomic.h
@@ -18,11 +18,11 @@
  * To ensure dependency ordering is preserved for the _relaxed and
  * _release atomics, an smp_read_barrier_depends() is unconditionally
  * inserted into the _relaxed variants, which are used to build the
- * barriered versions. To avoid redundant back-to-back fences, we can
- * define the _acquire and _fence versions explicitly.
+ * barriered versions. Avoid redundant back-to-back fences in the
+ * _acquire and _fence versions.
  */
-#define __atomic_op_acquire(op, args...)   op##_relaxed(args)
-#define __atomic_op_fence  __atomic_op_release
+#define __atomic_acquire_fence()
+#define __atomic_post_full_fence()
 
 #define ATOMIC_INIT(i) { (i) }
 #define ATOMIC64_INIT(i)   { (i) }
diff --git a/arch/powerpc/include/asm/atomic.h 
b/arch/powerpc/include/asm/atomic.h
index a0156cb43d1f..963abf8bf1c0 100644
--- a/arch/powerpc/include/asm/atomic.h
+++ b/arch/powerpc/include/asm/atomic.h
@@ -18,18 +18,11 @@
  * a "bne-" instruction at the end, so an isync is enough as a acquire barrier
  * on the platform without lwsync.
  */
-#define __atomic_op_acquire(op, args...)   \
-({ \
-   typeof(op##_relaxed(args)) __ret  = op##_relaxed(args); \
-   __asm__ __volatile__(PPC_ACQUIRE_BARRIER "" : : : "memory");\
-   __ret;  \
-})
-
-#define __atomic_op_release(op, args...)   \
-({ \
-   __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory");\
-   op##_relaxed(args); \
-})
+#define __atomic_acquire_fence()   \
+   __asm__ __volatile__(PPC_ACQUIRE_BARRIER "" : : : "memory")
+
+#define __atomic_release_fence()   \
+   __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory")
 
 static __inline__ int atomic_read(const atomic_t *v)
 {
diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index 512b89485790..c452359c9cb8 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -25,18 +25,11 @@
 
 #define ATOMIC_INIT(i) { (i) }
 
-#define __atomic_op_acquire(op, args...)   \
-({ \
-   typeof(op##_relaxed(args)) __ret  = op##_relaxed(args); \
-   __asm__ __volatile__(RISCV_ACQUIRE_BARRIER "" ::: "memory");\
-   __ret;  \
-})
-
-#define __atomic_op_release(op, args...) 

[tip:locking/core] locking/atomics: Rework ordering barriers

[tip-bot for Mark Rutland]

Commit-ID:  fd2efaa4eb5317c3a86357a83a7d456a1b86a0ac
Gitweb: https://git.kernel.org/tip/fd2efaa4eb5317c3a86357a83a7d456a1b86a0ac
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:11 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:59 +0200

locking/atomics: Rework ordering barriers

Currently architectures can override __atomic_op_*() to define the barriers
used before/after a relaxed atomic when used to build acquire/release/fence
variants.

This has the unfortunate property of requiring the architecture to define the
full wrapper for the atomics, rather than just the barriers they care about,
and gets in the way of generating atomics which can be easily read.

Instead, this patch has architectures define an optional set of barriers:

* __atomic_acquire_fence()
* __atomic_release_fence()
* __atomic_pre_full_fence()
* __atomic_post_full_fence()

... which  uses to build the wrappers.

It would be nice if we could undef these, along with the __atomic_op_*()
wrappers, but that would break the cmpxchg() wrappers, which are written
in preprocessor. Undefs would have been nice, but alas.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Andrea Parri 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: dvyu...@google.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-7-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/alpha/include/asm/atomic.h   |  8 
 arch/powerpc/include/asm/atomic.h | 17 +
 arch/riscv/include/asm/atomic.h   | 17 +
 include/linux/atomic.h| 38 ++
 4 files changed, 36 insertions(+), 44 deletions(-)

diff --git a/arch/alpha/include/asm/atomic.h b/arch/alpha/include/asm/atomic.h
index 4a6a8f58c9c9..150a1c5d6a2c 100644
--- a/arch/alpha/include/asm/atomic.h
+++ b/arch/alpha/include/asm/atomic.h
@@ -18,11 +18,11 @@
  * To ensure dependency ordering is preserved for the _relaxed and
  * _release atomics, an smp_read_barrier_depends() is unconditionally
  * inserted into the _relaxed variants, which are used to build the
- * barriered versions. To avoid redundant back-to-back fences, we can
- * define the _acquire and _fence versions explicitly.
+ * barriered versions. Avoid redundant back-to-back fences in the
+ * _acquire and _fence versions.
  */
-#define __atomic_op_acquire(op, args...)   op##_relaxed(args)
-#define __atomic_op_fence  __atomic_op_release
+#define __atomic_acquire_fence()
+#define __atomic_post_full_fence()
 
 #define ATOMIC_INIT(i) { (i) }
 #define ATOMIC64_INIT(i)   { (i) }
diff --git a/arch/powerpc/include/asm/atomic.h 
b/arch/powerpc/include/asm/atomic.h
index a0156cb43d1f..963abf8bf1c0 100644
--- a/arch/powerpc/include/asm/atomic.h
+++ b/arch/powerpc/include/asm/atomic.h
@@ -18,18 +18,11 @@
  * a "bne-" instruction at the end, so an isync is enough as a acquire barrier
  * on the platform without lwsync.
  */
-#define __atomic_op_acquire(op, args...)   \
-({ \
-   typeof(op##_relaxed(args)) __ret  = op##_relaxed(args); \
-   __asm__ __volatile__(PPC_ACQUIRE_BARRIER "" : : : "memory");\
-   __ret;  \
-})
-
-#define __atomic_op_release(op, args...)   \
-({ \
-   __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory");\
-   op##_relaxed(args); \
-})
+#define __atomic_acquire_fence()   \
+   __asm__ __volatile__(PPC_ACQUIRE_BARRIER "" : : : "memory")
+
+#define __atomic_release_fence()   \
+   __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory")
 
 static __inline__ int atomic_read(const atomic_t *v)
 {
diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index 512b89485790..c452359c9cb8 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -25,18 +25,11 @@
 
 #define ATOMIC_INIT(i) { (i) }
 
-#define __atomic_op_acquire(op, args...)   \
-({ \
-   typeof(op##_relaxed(args)) __ret  = op##_relaxed(args); \
-   __asm__ __volatile__(RISCV_ACQUIRE_BARRIER "" ::: "memory");\
-   __ret;  \
-})
-
-#define __atomic_op_release(op, args...) 

[tip:locking/core] locking/atomics: Instrument cmpxchg_double*()

[tip-bot for Mark Rutland]

Commit-ID:  4d2b25f630c731218d04f72580b4de68cb7a6e00
Gitweb: https://git.kernel.org/tip/4d2b25f630c731218d04f72580b4de68cb7a6e00
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:10 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:59 +0200

locking/atomics: Instrument cmpxchg_double*()

We currently don't instrument cmpxchg_double() and
cmpxchg_double_local() due to compilation issues reported in the past,
which are supposedly related to GCC bug 72873 [1], reported when GCC 7
was not yet released. This bug only applies to x86-64, and does not
apply to other architectures.

While the test case for GCC bug 72873 triggers issues with released
versions of GCC, the instrumented kernel code compiles fine for all
configurations I have tried, and it is unclear how the two cases
are/were related.

As we can't reproduce the kernel build failures, let's instrument
cmpxchg_double*() again. We can revisit the issue if build failures
reappear.

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Arnd Bergmann 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-6-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-instrumented.h | 21 -
 1 file changed, 8 insertions(+), 13 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 53481b6eacdf..0d4b1d3dbc1e 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -450,23 +450,18 @@ static __always_inline bool atomic64_add_negative(s64 i, 
atomic64_t *v)
arch_cmpxchg64_local(__ai_ptr, (old), (new));   \
 })
 
-/*
- * Originally we had the following code here:
- * __typeof__(p1) p1 = (p1);
- * kasan_check_write(p1, 2 * sizeof(*p1));
- * arch_cmpxchg_double(p1, (p2), (o1), (o2), (n1), (n2));
- * But it leads to compilation failures (see gcc issue 72873).
- * So for now it's left non-instrumented.
- * There are few callers of cmpxchg_double(), so it's not critical.
- */
 #define cmpxchg_double(p1, p2, o1, o2, n1, n2) \
 ({ \
-   arch_cmpxchg_double((p1), (p2), (o1), (o2), (n1), (n2));\
+   typeof(p1) __ai_p1 = (p1);  \
+   kasan_check_write(__ai_p1, 2 * sizeof(*__ai_p1));   \
+   arch_cmpxchg_double(__ai_p1, (p2), (o1), (o2), (n1), (n2)); \
 })
 
-#define cmpxchg_double_local(p1, p2, o1, o2, n1, n2)   \
-({ \
-   arch_cmpxchg_double_local((p1), (p2), (o1), (o2), (n1), (n2));  \
+#define cmpxchg_double_local(p1, p2, o1, o2, n1, n2)   
\
+({ 
\
+   typeof(p1) __ai_p1 = (p1);  
\
+   kasan_check_write(__ai_p1, 2 * sizeof(*__ai_p1));   
\
+   arch_cmpxchg_double_local(__ai_p1, (p2), (o1), (o2), (n1), (n2));   
\
 })
 
 #endif /* _LINUX_ATOMIC_INSTRUMENTED_H */

[tip:locking/core] locking/atomics: Instrument cmpxchg_double*()

[tip-bot for Mark Rutland]

Commit-ID:  4d2b25f630c731218d04f72580b4de68cb7a6e00
Gitweb: https://git.kernel.org/tip/4d2b25f630c731218d04f72580b4de68cb7a6e00
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:10 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:59 +0200

locking/atomics: Instrument cmpxchg_double*()

We currently don't instrument cmpxchg_double() and
cmpxchg_double_local() due to compilation issues reported in the past,
which are supposedly related to GCC bug 72873 [1], reported when GCC 7
was not yet released. This bug only applies to x86-64, and does not
apply to other architectures.

While the test case for GCC bug 72873 triggers issues with released
versions of GCC, the instrumented kernel code compiles fine for all
configurations I have tried, and it is unclear how the two cases
are/were related.

As we can't reproduce the kernel build failures, let's instrument
cmpxchg_double*() again. We can revisit the issue if build failures
reappear.

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Arnd Bergmann 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-6-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-instrumented.h | 21 -
 1 file changed, 8 insertions(+), 13 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 53481b6eacdf..0d4b1d3dbc1e 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -450,23 +450,18 @@ static __always_inline bool atomic64_add_negative(s64 i, 
atomic64_t *v)
arch_cmpxchg64_local(__ai_ptr, (old), (new));   \
 })
 
-/*
- * Originally we had the following code here:
- * __typeof__(p1) p1 = (p1);
- * kasan_check_write(p1, 2 * sizeof(*p1));
- * arch_cmpxchg_double(p1, (p2), (o1), (o2), (n1), (n2));
- * But it leads to compilation failures (see gcc issue 72873).
- * So for now it's left non-instrumented.
- * There are few callers of cmpxchg_double(), so it's not critical.
- */
 #define cmpxchg_double(p1, p2, o1, o2, n1, n2) \
 ({ \
-   arch_cmpxchg_double((p1), (p2), (o1), (o2), (n1), (n2));\
+   typeof(p1) __ai_p1 = (p1);  \
+   kasan_check_write(__ai_p1, 2 * sizeof(*__ai_p1));   \
+   arch_cmpxchg_double(__ai_p1, (p2), (o1), (o2), (n1), (n2)); \
 })
 
-#define cmpxchg_double_local(p1, p2, o1, o2, n1, n2)   \
-({ \
-   arch_cmpxchg_double_local((p1), (p2), (o1), (o2), (n1), (n2));  \
+#define cmpxchg_double_local(p1, p2, o1, o2, n1, n2)   
\
+({ 
\
+   typeof(p1) __ai_p1 = (p1);  
\
+   kasan_check_write(__ai_p1, 2 * sizeof(*__ai_p1));   
\
+   arch_cmpxchg_double_local(__ai_p1, (p2), (o1), (o2), (n1), (n2));   
\
 })
 
 #endif /* _LINUX_ATOMIC_INSTRUMENTED_H */

[tip:locking/core] locking/atomics: Simplify cmpxchg() instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  df79ed2c064363cdc7d2d896923c1885d4e30520
Gitweb: https://git.kernel.org/tip/df79ed2c064363cdc7d2d896923c1885d4e30520
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:08 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:58 +0200

locking/atomics: Simplify cmpxchg() instrumentation

Currently we define some fairly verbose wrappers for the cmpxchg()
family so that we can pass a pointer and size into kasan_check_write().

The wrappers duplicate the size-switching logic necessary in arch code,
and only work for scalar types. On some architectures, (cmp)xchg are
used on non-scalar types, and thus the instrumented wrappers need to be
able to handle this.

We could take the type-punning logic from {READ,WRITE}_ONCE(), but this
makes the wrappers even more verbose, and requires several local
variables in the macros.

Instead, let's simplify the wrappers into simple macros which:

* snapshot the pointer into a single local variable, called __ai_ptr to
  avoid conflicts with variables in the scope of the caller.

* call kasan_check_write() on __ai_ptr.

* invoke the relevant arch_*() function, passing the original arguments,
  bar __ai_ptr being substituted for ptr.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-instrumented.h | 100 +-
 1 file changed, 15 insertions(+), 85 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 3c64e95d5ed0..c7c3e4cdd942 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -408,109 +408,39 @@ static __always_inline bool atomic64_add_negative(s64 i, 
atomic64_t *v)
 }
 #endif
 
-static __always_inline unsigned long
-cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, int 
size)
-{
-   kasan_check_write(ptr, size);
-   switch (size) {
-   case 1:
-   return arch_cmpxchg((u8 *)ptr, (u8)old, (u8)new);
-   case 2:
-   return arch_cmpxchg((u16 *)ptr, (u16)old, (u16)new);
-   case 4:
-   return arch_cmpxchg((u32 *)ptr, (u32)old, (u32)new);
-   case 8:
-   BUILD_BUG_ON(sizeof(unsigned long) != 8);
-   return arch_cmpxchg((u64 *)ptr, (u64)old, (u64)new);
-   }
-   BUILD_BUG();
-   return 0;
-}
-
 #define cmpxchg(ptr, old, new) \
 ({ \
-   ((__typeof__(*(ptr)))cmpxchg_size((ptr), (unsigned long)(old),  \
-   (unsigned long)(new), sizeof(*(ptr; \
+   typeof(ptr) __ai_ptr = (ptr);   \
+   kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \
+   arch_cmpxchg(__ai_ptr, (old), (new));   \
 })
 
-static __always_inline unsigned long
-sync_cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new,
- int size)
-{
-   kasan_check_write(ptr, size);
-   switch (size) {
-   case 1:
-   return arch_sync_cmpxchg((u8 *)ptr, (u8)old, (u8)new);
-   case 2:
-   return arch_sync_cmpxchg((u16 *)ptr, (u16)old, (u16)new);
-   case 4:
-   return arch_sync_cmpxchg((u32 *)ptr, (u32)old, (u32)new);
-   case 8:
-   BUILD_BUG_ON(sizeof(unsigned long) != 8);
-   return arch_sync_cmpxchg((u64 *)ptr, (u64)old, (u64)new);
-   }
-   BUILD_BUG();
-   return 0;
-}
-
 #define sync_cmpxchg(ptr, old, new)\
 ({ \
-   ((__typeof__(*(ptr)))sync_cmpxchg_size((ptr),   \
-   (unsigned long)(old), (unsigned long)(new), \
-   sizeof(*(ptr;   \
+   typeof(ptr) __ai_ptr = (ptr);   \
+   kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \
+   arch_sync_cmpxchg(__ai_ptr, (old), (new));  \
 })
 
-static __always_inline unsigned long
-cmpxchg_local_size(volatile void *ptr, unsigned long old, unsigned long new,
-  int size)
-{
-   kasan_check_write(ptr, size);
-   switch (size) {
-   case 1:
-   return arch_cmpxchg_local((u8 *)ptr, (u8)old, 

[tip:locking/core] locking/atomics: Simplify cmpxchg() instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  df79ed2c064363cdc7d2d896923c1885d4e30520
Gitweb: https://git.kernel.org/tip/df79ed2c064363cdc7d2d896923c1885d4e30520
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:08 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:58 +0200

locking/atomics: Simplify cmpxchg() instrumentation

Currently we define some fairly verbose wrappers for the cmpxchg()
family so that we can pass a pointer and size into kasan_check_write().

The wrappers duplicate the size-switching logic necessary in arch code,
and only work for scalar types. On some architectures, (cmp)xchg are
used on non-scalar types, and thus the instrumented wrappers need to be
able to handle this.

We could take the type-punning logic from {READ,WRITE}_ONCE(), but this
makes the wrappers even more verbose, and requires several local
variables in the macros.

Instead, let's simplify the wrappers into simple macros which:

* snapshot the pointer into a single local variable, called __ai_ptr to
  avoid conflicts with variables in the scope of the caller.

* call kasan_check_write() on __ai_ptr.

* invoke the relevant arch_*() function, passing the original arguments,
  bar __ai_ptr being substituted for ptr.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/asm-generic/atomic-instrumented.h | 100 +-
 1 file changed, 15 insertions(+), 85 deletions(-)

diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index 3c64e95d5ed0..c7c3e4cdd942 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -408,109 +408,39 @@ static __always_inline bool atomic64_add_negative(s64 i, 
atomic64_t *v)
 }
 #endif
 
-static __always_inline unsigned long
-cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, int 
size)
-{
-   kasan_check_write(ptr, size);
-   switch (size) {
-   case 1:
-   return arch_cmpxchg((u8 *)ptr, (u8)old, (u8)new);
-   case 2:
-   return arch_cmpxchg((u16 *)ptr, (u16)old, (u16)new);
-   case 4:
-   return arch_cmpxchg((u32 *)ptr, (u32)old, (u32)new);
-   case 8:
-   BUILD_BUG_ON(sizeof(unsigned long) != 8);
-   return arch_cmpxchg((u64 *)ptr, (u64)old, (u64)new);
-   }
-   BUILD_BUG();
-   return 0;
-}
-
 #define cmpxchg(ptr, old, new) \
 ({ \
-   ((__typeof__(*(ptr)))cmpxchg_size((ptr), (unsigned long)(old),  \
-   (unsigned long)(new), sizeof(*(ptr; \
+   typeof(ptr) __ai_ptr = (ptr);   \
+   kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \
+   arch_cmpxchg(__ai_ptr, (old), (new));   \
 })
 
-static __always_inline unsigned long
-sync_cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new,
- int size)
-{
-   kasan_check_write(ptr, size);
-   switch (size) {
-   case 1:
-   return arch_sync_cmpxchg((u8 *)ptr, (u8)old, (u8)new);
-   case 2:
-   return arch_sync_cmpxchg((u16 *)ptr, (u16)old, (u16)new);
-   case 4:
-   return arch_sync_cmpxchg((u32 *)ptr, (u32)old, (u32)new);
-   case 8:
-   BUILD_BUG_ON(sizeof(unsigned long) != 8);
-   return arch_sync_cmpxchg((u64 *)ptr, (u64)old, (u64)new);
-   }
-   BUILD_BUG();
-   return 0;
-}
-
 #define sync_cmpxchg(ptr, old, new)\
 ({ \
-   ((__typeof__(*(ptr)))sync_cmpxchg_size((ptr),   \
-   (unsigned long)(old), (unsigned long)(new), \
-   sizeof(*(ptr;   \
+   typeof(ptr) __ai_ptr = (ptr);   \
+   kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \
+   arch_sync_cmpxchg(__ai_ptr, (old), (new));  \
 })
 
-static __always_inline unsigned long
-cmpxchg_local_size(volatile void *ptr, unsigned long old, unsigned long new,
-  int size)
-{
-   kasan_check_write(ptr, size);
-   switch (size) {
-   case 1:
-   return arch_cmpxchg_local((u8 *)ptr, (u8)old, 

[tip:locking/core] locking/atomics: Instrument xchg()

[tip-bot for Mark Rutland]

Commit-ID:  f9881cc43b118efc6f82fef2d121166113ee9f8e
Gitweb: https://git.kernel.org/tip/f9881cc43b118efc6f82fef2d121166113ee9f8e
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:09 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:59 +0200

locking/atomics: Instrument xchg()

While we instrument all of the (non-relaxed) atomic_*() functions and
cmpxchg(), we missed xchg().

Let's add instrumentation for xchg(), fixing up x86 to implement
arch_xchg().

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/x86/include/asm/atomic.h | 2 +-
 arch/x86/include/asm/atomic64_64.h| 2 +-
 arch/x86/include/asm/cmpxchg.h| 2 +-
 include/asm-generic/atomic-instrumented.h | 7 +++
 4 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h
index 823fd2f320cf..b143717b92b3 100644
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@ -202,7 +202,7 @@ static __always_inline bool 
arch_atomic_try_cmpxchg(atomic_t *v, int *old, int n
 
 static inline int arch_atomic_xchg(atomic_t *v, int new)
 {
-   return xchg(>counter, new);
+   return arch_xchg(>counter, new);
 }
 
 static inline void arch_atomic_and(int i, atomic_t *v)
diff --git a/arch/x86/include/asm/atomic64_64.h 
b/arch/x86/include/asm/atomic64_64.h
index 849f1c566a11..4343d9b4f30e 100644
--- a/arch/x86/include/asm/atomic64_64.h
+++ b/arch/x86/include/asm/atomic64_64.h
@@ -188,7 +188,7 @@ static __always_inline bool 
arch_atomic64_try_cmpxchg(atomic64_t *v, s64 *old, l
 
 static inline long arch_atomic64_xchg(atomic64_t *v, long new)
 {
-   return xchg(>counter, new);
+   return arch_xchg(>counter, new);
 }
 
 static inline void arch_atomic64_and(long i, atomic64_t *v)
diff --git a/arch/x86/include/asm/cmpxchg.h b/arch/x86/include/asm/cmpxchg.h
index e3efd8a06066..a55d79b233d3 100644
--- a/arch/x86/include/asm/cmpxchg.h
+++ b/arch/x86/include/asm/cmpxchg.h
@@ -75,7 +75,7 @@ extern void __add_wrong_size(void)
  * use "asm volatile" and "memory" clobbers to prevent gcc from moving
  * information around.
  */
-#define xchg(ptr, v)   __xchg_op((ptr), (v), xchg, "")
+#define arch_xchg(ptr, v)  __xchg_op((ptr), (v), xchg, "")
 
 /*
  * Atomic compare and exchange.  Compare OLD with MEM, if identical,
diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index c7c3e4cdd942..53481b6eacdf 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -408,6 +408,13 @@ static __always_inline bool atomic64_add_negative(s64 i, 
atomic64_t *v)
 }
 #endif
 
+#define xchg(ptr, new) \
+({ \
+   typeof(ptr) __ai_ptr = (ptr);   \
+   kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \
+   arch_xchg(__ai_ptr, (new)); \
+})
+
 #define cmpxchg(ptr, old, new) \
 ({ \
typeof(ptr) __ai_ptr = (ptr);   \

[tip:locking/core] locking/atomics: Instrument xchg()

[tip-bot for Mark Rutland]

Commit-ID:  f9881cc43b118efc6f82fef2d121166113ee9f8e
Gitweb: https://git.kernel.org/tip/f9881cc43b118efc6f82fef2d121166113ee9f8e
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:09 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:59 +0200

locking/atomics: Instrument xchg()

While we instrument all of the (non-relaxed) atomic_*() functions and
cmpxchg(), we missed xchg().

Let's add instrumentation for xchg(), fixing up x86 to implement
arch_xchg().

Signed-off-by: Mark Rutland 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/x86/include/asm/atomic.h | 2 +-
 arch/x86/include/asm/atomic64_64.h| 2 +-
 arch/x86/include/asm/cmpxchg.h| 2 +-
 include/asm-generic/atomic-instrumented.h | 7 +++
 4 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h
index 823fd2f320cf..b143717b92b3 100644
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@ -202,7 +202,7 @@ static __always_inline bool 
arch_atomic_try_cmpxchg(atomic_t *v, int *old, int n
 
 static inline int arch_atomic_xchg(atomic_t *v, int new)
 {
-   return xchg(>counter, new);
+   return arch_xchg(>counter, new);
 }
 
 static inline void arch_atomic_and(int i, atomic_t *v)
diff --git a/arch/x86/include/asm/atomic64_64.h 
b/arch/x86/include/asm/atomic64_64.h
index 849f1c566a11..4343d9b4f30e 100644
--- a/arch/x86/include/asm/atomic64_64.h
+++ b/arch/x86/include/asm/atomic64_64.h
@@ -188,7 +188,7 @@ static __always_inline bool 
arch_atomic64_try_cmpxchg(atomic64_t *v, s64 *old, l
 
 static inline long arch_atomic64_xchg(atomic64_t *v, long new)
 {
-   return xchg(>counter, new);
+   return arch_xchg(>counter, new);
 }
 
 static inline void arch_atomic64_and(long i, atomic64_t *v)
diff --git a/arch/x86/include/asm/cmpxchg.h b/arch/x86/include/asm/cmpxchg.h
index e3efd8a06066..a55d79b233d3 100644
--- a/arch/x86/include/asm/cmpxchg.h
+++ b/arch/x86/include/asm/cmpxchg.h
@@ -75,7 +75,7 @@ extern void __add_wrong_size(void)
  * use "asm volatile" and "memory" clobbers to prevent gcc from moving
  * information around.
  */
-#define xchg(ptr, v)   __xchg_op((ptr), (v), xchg, "")
+#define arch_xchg(ptr, v)  __xchg_op((ptr), (v), xchg, "")
 
 /*
  * Atomic compare and exchange.  Compare OLD with MEM, if identical,
diff --git a/include/asm-generic/atomic-instrumented.h 
b/include/asm-generic/atomic-instrumented.h
index c7c3e4cdd942..53481b6eacdf 100644
--- a/include/asm-generic/atomic-instrumented.h
+++ b/include/asm-generic/atomic-instrumented.h
@@ -408,6 +408,13 @@ static __always_inline bool atomic64_add_negative(s64 i, 
atomic64_t *v)
 }
 #endif
 
+#define xchg(ptr, new) \
+({ \
+   typeof(ptr) __ai_ptr = (ptr);   \
+   kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \
+   arch_xchg(__ai_ptr, (new)); \
+})
+
 #define cmpxchg(ptr, old, new) \
 ({ \
typeof(ptr) __ai_ptr = (ptr);   \

[tip:locking/core] locking/atomics/x86: Reduce arch_cmpxchg64*() instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  00d5551cc4eec0fc39c3871c25c613553acfb866
Gitweb: https://git.kernel.org/tip/00d5551cc4eec0fc39c3871c25c613553acfb866
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:07 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:58 +0200

locking/atomics/x86: Reduce arch_cmpxchg64*() instrumentation

Currently x86's arch_cmpxchg64() and arch_cmpxchg64_local() are
instrumented twice, as they call into instrumented atomics rather than
their arch_ equivalents.

A call to cmpxchg64() results in:

  cmpxchg64()
kasan_check_write()
arch_cmpxchg64()
  cmpxchg()
kasan_check_write()
arch_cmpxchg()

Let's fix this up and call the arch_ equivalents, resulting in:

  cmpxchg64()
kasan_check_write()
arch_cmpxchg64()
  arch_cmpxchg()

Signed-off-by: Mark Rutland 
Acked-by: Thomas Gleixner 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/x86/include/asm/cmpxchg_64.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/cmpxchg_64.h 
b/arch/x86/include/asm/cmpxchg_64.h
index bfca3b346c74..072e5459fe2f 100644
--- a/arch/x86/include/asm/cmpxchg_64.h
+++ b/arch/x86/include/asm/cmpxchg_64.h
@@ -10,13 +10,13 @@ static inline void set_64bit(volatile u64 *ptr, u64 val)
 #define arch_cmpxchg64(ptr, o, n)  \
 ({ \
BUILD_BUG_ON(sizeof(*(ptr)) != 8);  \
-   cmpxchg((ptr), (o), (n));   \
+   arch_cmpxchg((ptr), (o), (n));  \
 })
 
 #define arch_cmpxchg64_local(ptr, o, n)
\
 ({ \
BUILD_BUG_ON(sizeof(*(ptr)) != 8);  \
-   cmpxchg_local((ptr), (o), (n)); \
+   arch_cmpxchg_local((ptr), (o), (n));\
 })
 
 #define system_has_cmpxchg_double() boot_cpu_has(X86_FEATURE_CX16)

[tip:locking/core] locking/atomics/x86: Reduce arch_cmpxchg64*() instrumentation

[tip-bot for Mark Rutland]

Commit-ID:  00d5551cc4eec0fc39c3871c25c613553acfb866
Gitweb: https://git.kernel.org/tip/00d5551cc4eec0fc39c3871c25c613553acfb866
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 12:30:07 +0100
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Jul 2018 11:53:58 +0200

locking/atomics/x86: Reduce arch_cmpxchg64*() instrumentation

Currently x86's arch_cmpxchg64() and arch_cmpxchg64_local() are
instrumented twice, as they call into instrumented atomics rather than
their arch_ equivalents.

A call to cmpxchg64() results in:

  cmpxchg64()
kasan_check_write()
arch_cmpxchg64()
  cmpxchg()
kasan_check_write()
arch_cmpxchg()

Let's fix this up and call the arch_ equivalents, resulting in:

  cmpxchg64()
kasan_check_write()
arch_cmpxchg64()
  arch_cmpxchg()

Signed-off-by: Mark Rutland 
Acked-by: Thomas Gleixner 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Dmitry Vyukov 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: andy.shevche...@gmail.com
Cc: a...@arndb.de
Cc: aryabi...@virtuozzo.com
Cc: catalin.mari...@arm.com
Cc: gli...@google.com
Cc: linux-arm-ker...@lists.infradead.org
Cc: parri.and...@gmail.com
Cc: pe...@hurleysoftware.com
Link: http://lkml.kernel.org/r/20180716113017.3909-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 arch/x86/include/asm/cmpxchg_64.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/cmpxchg_64.h 
b/arch/x86/include/asm/cmpxchg_64.h
index bfca3b346c74..072e5459fe2f 100644
--- a/arch/x86/include/asm/cmpxchg_64.h
+++ b/arch/x86/include/asm/cmpxchg_64.h
@@ -10,13 +10,13 @@ static inline void set_64bit(volatile u64 *ptr, u64 val)
 #define arch_cmpxchg64(ptr, o, n)  \
 ({ \
BUILD_BUG_ON(sizeof(*(ptr)) != 8);  \
-   cmpxchg((ptr), (o), (n));   \
+   arch_cmpxchg((ptr), (o), (n));  \
 })
 
 #define arch_cmpxchg64_local(ptr, o, n)
\
 ({ \
BUILD_BUG_ON(sizeof(*(ptr)) != 8);  \
-   cmpxchg_local((ptr), (o), (n)); \
+   arch_cmpxchg_local((ptr), (o), (n));\
 })
 
 #define system_has_cmpxchg_double() boot_cpu_has(X86_FEATURE_CX16)

[tip:locking/core] tools/memory-model: Remove ACCESS_ONCE() from model

[tip-bot for Mark Rutland]

Commit-ID:  af41db5ef7c7cf0fee860f65cab59d0d188098d2
Gitweb: https://git.kernel.org/tip/af41db5ef7c7cf0fee860f65cab59d0d188098d2
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 11:05:57 -0700
Committer:  Ingo Molnar 
CommitDate: Tue, 17 Jul 2018 09:29:35 +0200

tools/memory-model: Remove ACCESS_ONCE() from model

Since commit:

  b899a850431e2dd0 ("compiler.h: Remove ACCESS_ONCE()")

... there has been no definition of ACCESS_ONCE() in the kernel tree,
and it has been necessary to use READ_ONCE() or WRITE_ONCE() instead.

Correspondingly, let's remove ACCESS_ONCE() from the kernel memory
model.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Andrea Parri 
Cc: Akira Yokosawa 
Cc: Alan Stern 
Cc: Boqun Feng 
Cc: David Howells 
Cc: Jade Alglave 
Cc: Linus Torvalds 
Cc: Luc Maranget 
Cc: Nicholas Piggin 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: linux-a...@vger.kernel.org
Cc: parri.and...@gmail.com
Link: http://lkml.kernel.org/r/20180716180605.16115-6-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/memory-model/linux-kernel.bell | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/memory-model/linux-kernel.bell 
b/tools/memory-model/linux-kernel.bell
index 64f5740e0e75..b84fb2f67109 100644
--- a/tools/memory-model/linux-kernel.bell
+++ b/tools/memory-model/linux-kernel.bell
@@ -13,7 +13,7 @@
 
 "Linux-kernel memory consistency model"
 
-enum Accesses = 'once (*READ_ONCE,WRITE_ONCE,ACCESS_ONCE*) ||
+enum Accesses = 'once (*READ_ONCE,WRITE_ONCE*) ||
'release (*smp_store_release*) ||
'acquire (*smp_load_acquire*) ||
'noreturn (* R of non-return RMW *)

[tip:locking/core] tools/memory-model: Remove ACCESS_ONCE() from model

[tip-bot for Mark Rutland]

Commit-ID:  af41db5ef7c7cf0fee860f65cab59d0d188098d2
Gitweb: https://git.kernel.org/tip/af41db5ef7c7cf0fee860f65cab59d0d188098d2
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 11:05:57 -0700
Committer:  Ingo Molnar 
CommitDate: Tue, 17 Jul 2018 09:29:35 +0200

tools/memory-model: Remove ACCESS_ONCE() from model

Since commit:

  b899a850431e2dd0 ("compiler.h: Remove ACCESS_ONCE()")

... there has been no definition of ACCESS_ONCE() in the kernel tree,
and it has been necessary to use READ_ONCE() or WRITE_ONCE() instead.

Correspondingly, let's remove ACCESS_ONCE() from the kernel memory
model.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Andrea Parri 
Cc: Akira Yokosawa 
Cc: Alan Stern 
Cc: Boqun Feng 
Cc: David Howells 
Cc: Jade Alglave 
Cc: Linus Torvalds 
Cc: Luc Maranget 
Cc: Nicholas Piggin 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: linux-a...@vger.kernel.org
Cc: parri.and...@gmail.com
Link: http://lkml.kernel.org/r/20180716180605.16115-6-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/memory-model/linux-kernel.bell | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/memory-model/linux-kernel.bell 
b/tools/memory-model/linux-kernel.bell
index 64f5740e0e75..b84fb2f67109 100644
--- a/tools/memory-model/linux-kernel.bell
+++ b/tools/memory-model/linux-kernel.bell
@@ -13,7 +13,7 @@
 
 "Linux-kernel memory consistency model"
 
-enum Accesses = 'once (*READ_ONCE,WRITE_ONCE,ACCESS_ONCE*) ||
+enum Accesses = 'once (*READ_ONCE,WRITE_ONCE*) ||
'release (*smp_store_release*) ||
'acquire (*smp_load_acquire*) ||
'noreturn (* R of non-return RMW *)

[tip:locking/core] tools/memory-model: Remove ACCESS_ONCE() from recipes

[tip-bot for Mark Rutland]

Commit-ID:  5bde06b63af2af8c2326270aae041067eef89245
Gitweb: https://git.kernel.org/tip/5bde06b63af2af8c2326270aae041067eef89245
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 11:05:56 -0700
Committer:  Ingo Molnar 
CommitDate: Tue, 17 Jul 2018 09:29:34 +0200

tools/memory-model: Remove ACCESS_ONCE() from recipes

Since commit:

  b899a850431e2dd0 ("compiler.h: Remove ACCESS_ONCE()")

... there has been no definition of ACCESS_ONCE() in the kernel tree,
and it has been necessary to use READ_ONCE() or WRITE_ONCE() instead.

Let's update the exmaples in recipes.txt likewise for consistency, using
READ_ONCE() for reads.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Andrea Parri 
Cc: Akira Yokosawa 
Cc: Alan Stern 
Cc: Boqun Feng 
Cc: David Howells 
Cc: Jade Alglave 
Cc: Linus Torvalds 
Cc: Luc Maranget 
Cc: Nicholas Piggin 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: linux-a...@vger.kernel.org
Cc: parri.and...@gmail.com
Link: http://lkml.kernel.org/r/20180716180605.16115-5-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/memory-model/Documentation/recipes.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/memory-model/Documentation/recipes.txt 
b/tools/memory-model/Documentation/recipes.txt
index ee4309a87fc4..1fea8ef2b184 100644
--- a/tools/memory-model/Documentation/recipes.txt
+++ b/tools/memory-model/Documentation/recipes.txt
@@ -322,9 +322,9 @@ the following write-side code fragment:
 And the xlog_valid_lsn() function in fs/xfs/xfs_log_priv.h contains
 the corresponding read-side code fragment:
 
-   cur_cycle = ACCESS_ONCE(log->l_curr_cycle);
+   cur_cycle = READ_ONCE(log->l_curr_cycle);
smp_rmb();
-   cur_block = ACCESS_ONCE(log->l_curr_block);
+   cur_block = READ_ONCE(log->l_curr_block);
 
 Alternatively, consider the following comment in function
 perf_output_put_handle() in kernel/events/ring_buffer.c:

[tip:locking/core] tools/memory-model: Remove ACCESS_ONCE() from recipes

[tip-bot for Mark Rutland]

Commit-ID:  5bde06b63af2af8c2326270aae041067eef89245
Gitweb: https://git.kernel.org/tip/5bde06b63af2af8c2326270aae041067eef89245
Author: Mark Rutland 
AuthorDate: Mon, 16 Jul 2018 11:05:56 -0700
Committer:  Ingo Molnar 
CommitDate: Tue, 17 Jul 2018 09:29:34 +0200

tools/memory-model: Remove ACCESS_ONCE() from recipes

Since commit:

  b899a850431e2dd0 ("compiler.h: Remove ACCESS_ONCE()")

... there has been no definition of ACCESS_ONCE() in the kernel tree,
and it has been necessary to use READ_ONCE() or WRITE_ONCE() instead.

Let's update the exmaples in recipes.txt likewise for consistency, using
READ_ONCE() for reads.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Andrea Parri 
Cc: Akira Yokosawa 
Cc: Alan Stern 
Cc: Boqun Feng 
Cc: David Howells 
Cc: Jade Alglave 
Cc: Linus Torvalds 
Cc: Luc Maranget 
Cc: Nicholas Piggin 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: linux-a...@vger.kernel.org
Cc: parri.and...@gmail.com
Link: http://lkml.kernel.org/r/20180716180605.16115-5-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/memory-model/Documentation/recipes.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/memory-model/Documentation/recipes.txt 
b/tools/memory-model/Documentation/recipes.txt
index ee4309a87fc4..1fea8ef2b184 100644
--- a/tools/memory-model/Documentation/recipes.txt
+++ b/tools/memory-model/Documentation/recipes.txt
@@ -322,9 +322,9 @@ the following write-side code fragment:
 And the xlog_valid_lsn() function in fs/xfs/xfs_log_priv.h contains
 the corresponding read-side code fragment:
 
-   cur_cycle = ACCESS_ONCE(log->l_curr_cycle);
+   cur_cycle = READ_ONCE(log->l_curr_cycle);
smp_rmb();
-   cur_block = ACCESS_ONCE(log->l_curr_block);
+   cur_block = READ_ONCE(log->l_curr_block);
 
 Alternatively, consider the following comment in function
 perf_output_put_handle() in kernel/events/ring_buffer.c:

[tip:locking/core] locking/refcount: Always allow checked forms

[tip-bot for Mark Rutland]

Commit-ID:  afed7bcf9487bb28e2e2b016a195085c07416c0b
Gitweb: https://git.kernel.org/tip/afed7bcf9487bb28e2e2b016a195085c07416c0b
Author: Mark Rutland 
AuthorDate: Wed, 11 Jul 2018 10:36:07 +0100
Committer:  Ingo Molnar 
CommitDate: Fri, 13 Jul 2018 15:23:25 +0200

locking/refcount: Always allow checked forms

In many cases, it would be useful to be able to use the full
sanity-checked refcount helpers regardless of CONFIG_REFCOUNT_FULL,
as this would help to avoid duplicate warnings where callers try to
sanity-check refcount manipulation.

This patch refactors things such that the full refcount helpers were
always built, as refcount_${op}_checked(), such that they can be used
regardless of CONFIG_REFCOUNT_FULL. This will allow code which *always*
wants a checked refcount to opt-in, avoiding the need to duplicate the
logic for warnings.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: David Sterba 
Acked-by: Kees Cook 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: http://lkml.kernel.org/r/20180711093607.1644-1-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/refcount.h | 27 +---
 lib/refcount.c   | 53 +++-
 2 files changed, 45 insertions(+), 35 deletions(-)

diff --git a/include/linux/refcount.h b/include/linux/refcount.h
index c36addd27dd5..53c5eca24d83 100644
--- a/include/linux/refcount.h
+++ b/include/linux/refcount.h
@@ -43,17 +43,30 @@ static inline unsigned int refcount_read(const refcount_t 
*r)
return atomic_read(>refs);
 }
 
+extern __must_check bool refcount_add_not_zero_checked(unsigned int i, 
refcount_t *r);
+extern void refcount_add_checked(unsigned int i, refcount_t *r);
+
+extern __must_check bool refcount_inc_not_zero_checked(refcount_t *r);
+extern void refcount_inc_checked(refcount_t *r);
+
+extern __must_check bool refcount_sub_and_test_checked(unsigned int i, 
refcount_t *r);
+
+extern __must_check bool refcount_dec_and_test_checked(refcount_t *r);
+extern void refcount_dec_checked(refcount_t *r);
+
 #ifdef CONFIG_REFCOUNT_FULL
-extern __must_check bool refcount_add_not_zero(unsigned int i, refcount_t *r);
-extern void refcount_add(unsigned int i, refcount_t *r);
 
-extern __must_check bool refcount_inc_not_zero(refcount_t *r);
-extern void refcount_inc(refcount_t *r);
+#define refcount_add_not_zero  refcount_add_not_zero_checked
+#define refcount_add   refcount_add_checked
+
+#define refcount_inc_not_zero  refcount_inc_not_zero_checked
+#define refcount_inc   refcount_inc_checked
+
+#define refcount_sub_and_test  refcount_sub_and_test_checked
 
-extern __must_check bool refcount_sub_and_test(unsigned int i, refcount_t *r);
+#define refcount_dec_and_test  refcount_dec_and_test_checked
+#define refcount_dec   refcount_dec_checked
 
-extern __must_check bool refcount_dec_and_test(refcount_t *r);
-extern void refcount_dec(refcount_t *r);
 #else
 # ifdef CONFIG_ARCH_HAS_REFCOUNT
 #  include 
diff --git a/lib/refcount.c b/lib/refcount.c
index 4bd842f20749..5c4aaefc0682 100644
--- a/lib/refcount.c
+++ b/lib/refcount.c
@@ -40,10 +40,8 @@
 #include 
 #include 
 
-#ifdef CONFIG_REFCOUNT_FULL
-
 /**
- * refcount_add_not_zero - add a value to a refcount unless it is 0
+ * refcount_add_not_zero_checked - add a value to a refcount unless it is 0
  * @i: the value to add to the refcount
  * @r: the refcount
  *
@@ -60,7 +58,7 @@
  *
  * Return: false if the passed refcount is 0, true otherwise
  */
-bool refcount_add_not_zero(unsigned int i, refcount_t *r)
+bool refcount_add_not_zero_checked(unsigned int i, refcount_t *r)
 {
unsigned int new, val = atomic_read(>refs);
 
@@ -81,10 +79,10 @@ bool refcount_add_not_zero(unsigned int i, refcount_t *r)
 
return true;
 }
-EXPORT_SYMBOL(refcount_add_not_zero);
+EXPORT_SYMBOL(refcount_add_not_zero_checked);
 
 /**
- * refcount_add - add a value to a refcount
+ * refcount_add_checked - add a value to a refcount
  * @i: the value to add to the refcount
  * @r: the refcount
  *
@@ -99,14 +97,14 @@ EXPORT_SYMBOL(refcount_add_not_zero);
  * cases, refcount_inc(), or one of its variants, should instead be used to
  * increment a reference count.
  */
-void refcount_add(unsigned int i, refcount_t *r)
+void refcount_add_checked(unsigned int i, refcount_t *r)
 {
-   WARN_ONCE(!refcount_add_not_zero(i, r), "refcount_t: addition on 0; 
use-after-free.\n");
+   WARN_ONCE(!refcount_add_not_zero_checked(i, r), "refcount_t: addition 
on 0; use-after-free.\n");
 }
-EXPORT_SYMBOL(refcount_add);
+EXPORT_SYMBOL(refcount_add_checked);
 
 /**
- * refcount_inc_not_zero - increment a refcount unless it is 0
+ * refcount_inc_not_zero_checked - increment a refcount unless it is 0
  * @r: the refcount to increment
  *
  * Similar to atomic_inc_not_zero(), but will saturate at UINT_MAX and WARN.
@@ -117,7 +115,7 @@ 

[tip:locking/core] locking/refcount: Always allow checked forms

[tip-bot for Mark Rutland]

Commit-ID:  afed7bcf9487bb28e2e2b016a195085c07416c0b
Gitweb: https://git.kernel.org/tip/afed7bcf9487bb28e2e2b016a195085c07416c0b
Author: Mark Rutland 
AuthorDate: Wed, 11 Jul 2018 10:36:07 +0100
Committer:  Ingo Molnar 
CommitDate: Fri, 13 Jul 2018 15:23:25 +0200

locking/refcount: Always allow checked forms

In many cases, it would be useful to be able to use the full
sanity-checked refcount helpers regardless of CONFIG_REFCOUNT_FULL,
as this would help to avoid duplicate warnings where callers try to
sanity-check refcount manipulation.

This patch refactors things such that the full refcount helpers were
always built, as refcount_${op}_checked(), such that they can be used
regardless of CONFIG_REFCOUNT_FULL. This will allow code which *always*
wants a checked refcount to opt-in, avoiding the need to duplicate the
logic for warnings.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: David Sterba 
Acked-by: Kees Cook 
Acked-by: Will Deacon 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: http://lkml.kernel.org/r/20180711093607.1644-1-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/refcount.h | 27 +---
 lib/refcount.c   | 53 +++-
 2 files changed, 45 insertions(+), 35 deletions(-)

diff --git a/include/linux/refcount.h b/include/linux/refcount.h
index c36addd27dd5..53c5eca24d83 100644
--- a/include/linux/refcount.h
+++ b/include/linux/refcount.h
@@ -43,17 +43,30 @@ static inline unsigned int refcount_read(const refcount_t 
*r)
return atomic_read(>refs);
 }
 
+extern __must_check bool refcount_add_not_zero_checked(unsigned int i, 
refcount_t *r);
+extern void refcount_add_checked(unsigned int i, refcount_t *r);
+
+extern __must_check bool refcount_inc_not_zero_checked(refcount_t *r);
+extern void refcount_inc_checked(refcount_t *r);
+
+extern __must_check bool refcount_sub_and_test_checked(unsigned int i, 
refcount_t *r);
+
+extern __must_check bool refcount_dec_and_test_checked(refcount_t *r);
+extern void refcount_dec_checked(refcount_t *r);
+
 #ifdef CONFIG_REFCOUNT_FULL
-extern __must_check bool refcount_add_not_zero(unsigned int i, refcount_t *r);
-extern void refcount_add(unsigned int i, refcount_t *r);
 
-extern __must_check bool refcount_inc_not_zero(refcount_t *r);
-extern void refcount_inc(refcount_t *r);
+#define refcount_add_not_zero  refcount_add_not_zero_checked
+#define refcount_add   refcount_add_checked
+
+#define refcount_inc_not_zero  refcount_inc_not_zero_checked
+#define refcount_inc   refcount_inc_checked
+
+#define refcount_sub_and_test  refcount_sub_and_test_checked
 
-extern __must_check bool refcount_sub_and_test(unsigned int i, refcount_t *r);
+#define refcount_dec_and_test  refcount_dec_and_test_checked
+#define refcount_dec   refcount_dec_checked
 
-extern __must_check bool refcount_dec_and_test(refcount_t *r);
-extern void refcount_dec(refcount_t *r);
 #else
 # ifdef CONFIG_ARCH_HAS_REFCOUNT
 #  include 
diff --git a/lib/refcount.c b/lib/refcount.c
index 4bd842f20749..5c4aaefc0682 100644
--- a/lib/refcount.c
+++ b/lib/refcount.c
@@ -40,10 +40,8 @@
 #include 
 #include 
 
-#ifdef CONFIG_REFCOUNT_FULL
-
 /**
- * refcount_add_not_zero - add a value to a refcount unless it is 0
+ * refcount_add_not_zero_checked - add a value to a refcount unless it is 0
  * @i: the value to add to the refcount
  * @r: the refcount
  *
@@ -60,7 +58,7 @@
  *
  * Return: false if the passed refcount is 0, true otherwise
  */
-bool refcount_add_not_zero(unsigned int i, refcount_t *r)
+bool refcount_add_not_zero_checked(unsigned int i, refcount_t *r)
 {
unsigned int new, val = atomic_read(>refs);
 
@@ -81,10 +79,10 @@ bool refcount_add_not_zero(unsigned int i, refcount_t *r)
 
return true;
 }
-EXPORT_SYMBOL(refcount_add_not_zero);
+EXPORT_SYMBOL(refcount_add_not_zero_checked);
 
 /**
- * refcount_add - add a value to a refcount
+ * refcount_add_checked - add a value to a refcount
  * @i: the value to add to the refcount
  * @r: the refcount
  *
@@ -99,14 +97,14 @@ EXPORT_SYMBOL(refcount_add_not_zero);
  * cases, refcount_inc(), or one of its variants, should instead be used to
  * increment a reference count.
  */
-void refcount_add(unsigned int i, refcount_t *r)
+void refcount_add_checked(unsigned int i, refcount_t *r)
 {
-   WARN_ONCE(!refcount_add_not_zero(i, r), "refcount_t: addition on 0; 
use-after-free.\n");
+   WARN_ONCE(!refcount_add_not_zero_checked(i, r), "refcount_t: addition 
on 0; use-after-free.\n");
 }
-EXPORT_SYMBOL(refcount_add);
+EXPORT_SYMBOL(refcount_add_checked);
 
 /**
- * refcount_inc_not_zero - increment a refcount unless it is 0
+ * refcount_inc_not_zero_checked - increment a refcount unless it is 0
  * @r: the refcount to increment
  *
  * Similar to atomic_inc_not_zero(), but will saturate at UINT_MAX and WARN.
@@ -117,7 +115,7 @@ 

[tip:efi/urgent] efi/libstub/arm64: Handle randomized TEXT_OFFSET

[tip-bot for Mark Rutland]

Commit-ID:  4f74d72aa7067e75af92fbab077e6d7d0210be66
Gitweb: https://git.kernel.org/tip/4f74d72aa7067e75af92fbab077e6d7d0210be66
Author: Mark Rutland 
AuthorDate: Fri, 18 May 2018 16:08:41 +0200
Committer:  Ingo Molnar 
CommitDate: Sat, 19 May 2018 08:07:56 +0200

efi/libstub/arm64: Handle randomized TEXT_OFFSET

When CONFIG_RANDOMIZE_TEXT_OFFSET=y, TEXT_OFFSET is an arbitrary
multiple of PAGE_SIZE in the interval [0, 2MB).

The EFI stub does not account for the potential misalignment of
TEXT_OFFSET relative to EFI_KIMG_ALIGN, and produces a randomized
physical offset which is always a round multiple of EFI_KIMG_ALIGN.
This may result in statically allocated objects whose alignment exceeds
PAGE_SIZE to appear misaligned in memory. This has been observed to
result in spurious stack overflow reports and failure to make use of
the IRQ stacks, and theoretically could result in a number of other
issues.

We can OR in the low bits of TEXT_OFFSET to ensure that we have the
necessary offset (and hence preserve the misalignment of TEXT_OFFSET
relative to EFI_KIMG_ALIGN), so let's do that.

Reported-by: Kim Phillips 
Tested-by: Kim Phillips 
[ardb: clarify comment and commit log, drop unneeded parens]
Signed-off-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Fixes: 6f26b3671184c36d ("arm64: kaslr: increase randomization granularity")
Link: http://lkml.kernel.org/r/20180518140841.9731-2-ard.biesheu...@linaro.org
Signed-off-by: Ingo Molnar 
---
 drivers/firmware/efi/libstub/arm64-stub.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/drivers/firmware/efi/libstub/arm64-stub.c 
b/drivers/firmware/efi/libstub/arm64-stub.c
index b9bd827caa22..1b4d465cc5d9 100644
--- a/drivers/firmware/efi/libstub/arm64-stub.c
+++ b/drivers/firmware/efi/libstub/arm64-stub.c
@@ -97,6 +97,16 @@ efi_status_t handle_kernel_image(efi_system_table_t 
*sys_table_arg,
u32 offset = !IS_ENABLED(CONFIG_DEBUG_ALIGN_RODATA) ?
 (phys_seed >> 32) & mask : TEXT_OFFSET;
 
+   /*
+* With CONFIG_RANDOMIZE_TEXT_OFFSET=y, TEXT_OFFSET may not
+* be a multiple of EFI_KIMG_ALIGN, and we must ensure that
+* we preserve the misalignment of 'offset' relative to
+* EFI_KIMG_ALIGN so that statically allocated objects whose
+* alignment exceeds PAGE_SIZE appear correctly aligned in
+* memory.
+*/
+   offset |= TEXT_OFFSET % EFI_KIMG_ALIGN;
+
/*
 * If KASLR is enabled, and we have some randomness available,
 * locate the kernel at a randomized offset in physical memory.

[tip:efi/urgent] efi/libstub/arm64: Handle randomized TEXT_OFFSET

[tip-bot for Mark Rutland]

Commit-ID:  4f74d72aa7067e75af92fbab077e6d7d0210be66
Gitweb: https://git.kernel.org/tip/4f74d72aa7067e75af92fbab077e6d7d0210be66
Author: Mark Rutland 
AuthorDate: Fri, 18 May 2018 16:08:41 +0200
Committer:  Ingo Molnar 
CommitDate: Sat, 19 May 2018 08:07:56 +0200

efi/libstub/arm64: Handle randomized TEXT_OFFSET

When CONFIG_RANDOMIZE_TEXT_OFFSET=y, TEXT_OFFSET is an arbitrary
multiple of PAGE_SIZE in the interval [0, 2MB).

The EFI stub does not account for the potential misalignment of
TEXT_OFFSET relative to EFI_KIMG_ALIGN, and produces a randomized
physical offset which is always a round multiple of EFI_KIMG_ALIGN.
This may result in statically allocated objects whose alignment exceeds
PAGE_SIZE to appear misaligned in memory. This has been observed to
result in spurious stack overflow reports and failure to make use of
the IRQ stacks, and theoretically could result in a number of other
issues.

We can OR in the low bits of TEXT_OFFSET to ensure that we have the
necessary offset (and hence preserve the misalignment of TEXT_OFFSET
relative to EFI_KIMG_ALIGN), so let's do that.

Reported-by: Kim Phillips 
Tested-by: Kim Phillips 
[ardb: clarify comment and commit log, drop unneeded parens]
Signed-off-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Fixes: 6f26b3671184c36d ("arm64: kaslr: increase randomization granularity")
Link: http://lkml.kernel.org/r/20180518140841.9731-2-ard.biesheu...@linaro.org
Signed-off-by: Ingo Molnar 
---
 drivers/firmware/efi/libstub/arm64-stub.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/drivers/firmware/efi/libstub/arm64-stub.c 
b/drivers/firmware/efi/libstub/arm64-stub.c
index b9bd827caa22..1b4d465cc5d9 100644
--- a/drivers/firmware/efi/libstub/arm64-stub.c
+++ b/drivers/firmware/efi/libstub/arm64-stub.c
@@ -97,6 +97,16 @@ efi_status_t handle_kernel_image(efi_system_table_t 
*sys_table_arg,
u32 offset = !IS_ENABLED(CONFIG_DEBUG_ALIGN_RODATA) ?
 (phys_seed >> 32) & mask : TEXT_OFFSET;
 
+   /*
+* With CONFIG_RANDOMIZE_TEXT_OFFSET=y, TEXT_OFFSET may not
+* be a multiple of EFI_KIMG_ALIGN, and we must ensure that
+* we preserve the misalignment of 'offset' relative to
+* EFI_KIMG_ALIGN so that statically allocated objects whose
+* alignment exceeds PAGE_SIZE appear correctly aligned in
+* memory.
+*/
+   offset |= TEXT_OFFSET % EFI_KIMG_ALIGN;
+
/*
 * If KASLR is enabled, and we have some randomness available,
 * locate the kernel at a randomized offset in physical memory.

[tip:perf/urgent] tools headers: Restore READ_ONCE() C++ compatibility

[tip-bot for Mark Rutland]

Commit-ID:  4d3b57da1593c66835d8e3a757e4751b35493fb8
Gitweb: https://git.kernel.org/tip/4d3b57da1593c66835d8e3a757e4751b35493fb8
Author: Mark Rutland 
AuthorDate: Wed, 4 Apr 2018 17:34:45 +0100
Committer:  Arnaldo Carvalho de Melo 
CommitDate: Thu, 12 Apr 2018 09:30:09 -0300

tools headers: Restore READ_ONCE() C++ compatibility

Our userspace  defines READ_ONCE() in a way that clang
doesn't like, as we have an anonymous union in which neither field is
initialized.

WRITE_ONCE() is fine since it initializes the __val field. For
READ_ONCE() we can keep clang and GCC happy with a dummy initialization
of the __c field, so let's do that.

At the same time, let's split READ_ONCE() and WRITE_ONCE() over several
lines for legibility, as we do in the in-kernel .

Reported-by: Li Zhijian 
Reported-by: Sandipan Das 
Tested-by: Sandipan Das 
Signed-off-by: Mark Rutland 
Fixes: 6aa7de059173a986 ("locking/atomics: COCCINELLE/treewide: Convert trivial 
ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE()")
Link: http://lkml.kernel.org/r/20180404163445.16492-1-mark.rutl...@arm.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/include/linux/compiler.h | 20 +++-
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/include/linux/compiler.h b/tools/include/linux/compiler.h
index 04e32f965ad7..1827c2f973f9 100644
--- a/tools/include/linux/compiler.h
+++ b/tools/include/linux/compiler.h
@@ -151,11 +151,21 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
  * required ordering.
  */
 
-#define READ_ONCE(x) \
-   ({ union { typeof(x) __val; char __c[1]; } __u; __read_once_size(&(x), 
__u.__c, sizeof(x)); __u.__val; })
-
-#define WRITE_ONCE(x, val) \
-   ({ union { typeof(x) __val; char __c[1]; } __u = { .__val = (val) }; 
__write_once_size(&(x), __u.__c, sizeof(x)); __u.__val; })
+#define READ_ONCE(x)   \
+({ \
+   union { typeof(x) __val; char __c[1]; } __u =   \
+   { .__c = { 0 } };   \
+   __read_once_size(&(x), __u.__c, sizeof(x)); \
+   __u.__val;  \
+})
+
+#define WRITE_ONCE(x, val) \
+({ \
+   union { typeof(x) __val; char __c[1]; } __u =   \
+   { .__val = (val) }; \
+   __write_once_size(&(x), __u.__c, sizeof(x));\
+   __u.__val;  \
+})
 
 
 #ifndef __fallthrough

[tip:perf/urgent] tools headers: Restore READ_ONCE() C++ compatibility

[tip-bot for Mark Rutland]

Commit-ID:  4d3b57da1593c66835d8e3a757e4751b35493fb8
Gitweb: https://git.kernel.org/tip/4d3b57da1593c66835d8e3a757e4751b35493fb8
Author: Mark Rutland 
AuthorDate: Wed, 4 Apr 2018 17:34:45 +0100
Committer:  Arnaldo Carvalho de Melo 
CommitDate: Thu, 12 Apr 2018 09:30:09 -0300

tools headers: Restore READ_ONCE() C++ compatibility

Our userspace  defines READ_ONCE() in a way that clang
doesn't like, as we have an anonymous union in which neither field is
initialized.

WRITE_ONCE() is fine since it initializes the __val field. For
READ_ONCE() we can keep clang and GCC happy with a dummy initialization
of the __c field, so let's do that.

At the same time, let's split READ_ONCE() and WRITE_ONCE() over several
lines for legibility, as we do in the in-kernel .

Reported-by: Li Zhijian 
Reported-by: Sandipan Das 
Tested-by: Sandipan Das 
Signed-off-by: Mark Rutland 
Fixes: 6aa7de059173a986 ("locking/atomics: COCCINELLE/treewide: Convert trivial 
ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE()")
Link: http://lkml.kernel.org/r/20180404163445.16492-1-mark.rutl...@arm.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/include/linux/compiler.h | 20 +++-
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/include/linux/compiler.h b/tools/include/linux/compiler.h
index 04e32f965ad7..1827c2f973f9 100644
--- a/tools/include/linux/compiler.h
+++ b/tools/include/linux/compiler.h
@@ -151,11 +151,21 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
  * required ordering.
  */
 
-#define READ_ONCE(x) \
-   ({ union { typeof(x) __val; char __c[1]; } __u; __read_once_size(&(x), 
__u.__c, sizeof(x)); __u.__val; })
-
-#define WRITE_ONCE(x, val) \
-   ({ union { typeof(x) __val; char __c[1]; } __u = { .__val = (val) }; 
__write_once_size(&(x), __u.__c, sizeof(x)); __u.__val; })
+#define READ_ONCE(x)   \
+({ \
+   union { typeof(x) __val; char __c[1]; } __u =   \
+   { .__c = { 0 } };   \
+   __read_once_size(&(x), __u.__c, sizeof(x)); \
+   __u.__val;  \
+})
+
+#define WRITE_ONCE(x, val) \
+({ \
+   union { typeof(x) __val; char __c[1]; } __u =   \
+   { .__val = (val) }; \
+   __write_once_size(&(x), __u.__c, sizeof(x));\
+   __u.__val;  \
+})
 
 
 #ifndef __fallthrough

[tip:perf/core] perf/core: Clear sibling list of detached events

[tip-bot for Mark Rutland]

Commit-ID:  24868367cdcac447232ebcb2aa06e1bf91291586
Gitweb: https://git.kernel.org/tip/24868367cdcac447232ebcb2aa06e1bf91291586
Author: Mark Rutland 
AuthorDate: Fri, 16 Mar 2018 12:51:40 +
Committer:  Thomas Gleixner 
CommitDate: Fri, 16 Mar 2018 20:44:32 +0100

perf/core: Clear sibling list of detached events

When perf_group_dettach() is called on a group leader, it updates each
sibling's group_leader field to point to that sibling, effectively
upgrading each siblnig to a group leader. After perf_group_detach has
completed, the caller may free the leader event.

We only remove siblings from the group leader's sibling_list when the
leader has a non-empty group_node. This was fine prior to commit:

  8343aae66167df67 ("perf/core: Remove perf_event::group_entry")

... as the sibling's sibling_list would be empty. However, now that we
use the sibling_list field as both the list head and the list entry,
this leaves each sibling with a non-empty sibling list, including the
stale leader event.

If perf_group_detach() is subsequently called on a sibling, it will
appear to be a group leader, and we'll walk the sibling_list,
potentially dereferencing these stale events. In 0day testing, this has
been observed to result in kernel panics.

Let's avoid this by always removing siblings from the sibling list when
we promote them to leaders.

Fixes: 8343aae66167df67 ("perf/core: Remove perf_event::group_entry")
Signed-off-by: Mark Rutland 
Signed-off-by: Thomas Gleixner 
Cc: vincent.wea...@maine.edu
Cc: Peter Zijlstra 
Cc: torva...@linux-foundation.org
Cc: Alexey Budankov 
Cc: valery.cherepenni...@intel.com
Cc: linux-tip-comm...@vger.kernel.org
Cc: eran...@google.com
Cc: a...@redhat.com
Cc: alexander.shish...@linux.intel.com
Cc: davi...@google.com
Cc: kan.li...@intel.com
Cc: dmitry.proho...@intel.com
Cc: Jiri Olsa 
Link: 
https://lkml.kernel.org/r/20180316131741.3svgr64yibc6v...@lakrids.cambridge.arm.com
---
 kernel/events/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 4d7a460d6669..2776a660db15 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1906,12 +1906,12 @@ static void perf_group_detach(struct perf_event *event)
list_for_each_entry_safe(sibling, tmp, >sibling_list, 
sibling_list) {
 
sibling->group_leader = sibling;
+   list_del_init(>sibling_list);
 
/* Inherit group flags from the previous leader */
sibling->group_caps = event->group_caps;
 
if (!RB_EMPTY_NODE(>group_node)) {
-   list_del_init(>sibling_list);
add_event_to_groups(sibling, event->ctx);
 
if (sibling->state == PERF_EVENT_STATE_ACTIVE) {

[tip:perf/core] perf/core: Clear sibling list of detached events

[tip-bot for Mark Rutland]

Commit-ID:  24868367cdcac447232ebcb2aa06e1bf91291586
Gitweb: https://git.kernel.org/tip/24868367cdcac447232ebcb2aa06e1bf91291586
Author: Mark Rutland 
AuthorDate: Fri, 16 Mar 2018 12:51:40 +
Committer:  Thomas Gleixner 
CommitDate: Fri, 16 Mar 2018 20:44:32 +0100

perf/core: Clear sibling list of detached events

When perf_group_dettach() is called on a group leader, it updates each
sibling's group_leader field to point to that sibling, effectively
upgrading each siblnig to a group leader. After perf_group_detach has
completed, the caller may free the leader event.

We only remove siblings from the group leader's sibling_list when the
leader has a non-empty group_node. This was fine prior to commit:

  8343aae66167df67 ("perf/core: Remove perf_event::group_entry")

... as the sibling's sibling_list would be empty. However, now that we
use the sibling_list field as both the list head and the list entry,
this leaves each sibling with a non-empty sibling list, including the
stale leader event.

If perf_group_detach() is subsequently called on a sibling, it will
appear to be a group leader, and we'll walk the sibling_list,
potentially dereferencing these stale events. In 0day testing, this has
been observed to result in kernel panics.

Let's avoid this by always removing siblings from the sibling list when
we promote them to leaders.

Fixes: 8343aae66167df67 ("perf/core: Remove perf_event::group_entry")
Signed-off-by: Mark Rutland 
Signed-off-by: Thomas Gleixner 
Cc: vincent.wea...@maine.edu
Cc: Peter Zijlstra 
Cc: torva...@linux-foundation.org
Cc: Alexey Budankov 
Cc: valery.cherepenni...@intel.com
Cc: linux-tip-comm...@vger.kernel.org
Cc: eran...@google.com
Cc: a...@redhat.com
Cc: alexander.shish...@linux.intel.com
Cc: davi...@google.com
Cc: kan.li...@intel.com
Cc: dmitry.proho...@intel.com
Cc: Jiri Olsa 
Link: 
https://lkml.kernel.org/r/20180316131741.3svgr64yibc6v...@lakrids.cambridge.arm.com
---
 kernel/events/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 4d7a460d6669..2776a660db15 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1906,12 +1906,12 @@ static void perf_group_detach(struct perf_event *event)
list_for_each_entry_safe(sibling, tmp, >sibling_list, 
sibling_list) {
 
sibling->group_leader = sibling;
+   list_del_init(>sibling_list);
 
/* Inherit group flags from the previous leader */
sibling->group_caps = event->group_caps;
 
if (!RB_EMPTY_NODE(>group_node)) {
-   list_del_init(>sibling_list);
add_event_to_groups(sibling, event->ctx);
 
if (sibling->state == PERF_EVENT_STATE_ACTIVE) {

[tip:perf/core] perf/core: Clear sibling list of detached events

[tip-bot for Mark Rutland]

Commit-ID:  bbb68468641547d56c83012670bcaf77f3dacd64
Gitweb: https://git.kernel.org/tip/bbb68468641547d56c83012670bcaf77f3dacd64
Author: Mark Rutland 
AuthorDate: Fri, 16 Mar 2018 12:51:40 +
Committer:  Thomas Gleixner 
CommitDate: Fri, 16 Mar 2018 15:34:24 +0100

perf/core: Clear sibling list of detached events

When perf_group_dettach() is called on a group leader, it updates each
sibling's group_leader field to point to that sibling, effectively
upgrading each siblnig to a group leader. After perf_group_detach has
completed, the caller may free the leader event.

We only remove siblings from the group leader's sibling_list when the
leader has a non-empty group_node. This was fine prior to commit:

  8343aae66167df67 ("perf/core: Remove perf_event::group_entry")

... as the sibling's sibling_list would be empty. However, now that we
use the sibling_list field as both the list head and the list entry,
this leaves each sibling with a non-empty sibling list, including the
stale leader event.

If perf_group_detach() is subsequently called on a sibling, it will
appear to be a group leader, and we'll walk the sibling_list,
potentially dereferencing these stale events. In 0day testing, this has
been observed to result in kernel panics.

Let's avoid this by always removing siblings from the sibling list when
we promote them to leaders.

Fixes: 8343aae66167df67 ("perf/core: Remove perf_event::group_entry")
Signed-off-by: Mark Rutland 
Signed-off-by: Thomas Gleixner 
Cc: vincent.wea...@maine.edu
Cc: Peter Zijlstra 
Cc: torva...@linux-foundation.org
Cc: Alexey Budankov 
Cc: valery.cherepenni...@intel.com
Cc: linux-tip-comm...@vger.kernel.org
Cc: eran...@google.com
Cc: a...@redhat.com
Cc: alexander.shish...@linux.intel.com
Cc: davi...@google.com
Cc: kan.li...@intel.com
Cc: dmitry.proho...@intel.com
Cc: Jiri Olsa 
Link: 
https://lkml.kernel.org/r/20180316131741.3svgr64yibc6v...@lakrids.cambridge.arm.com
---
 kernel/events/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 4d7a460d6669..2776a660db15 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1906,12 +1906,12 @@ static void perf_group_detach(struct perf_event *event)
list_for_each_entry_safe(sibling, tmp, >sibling_list, 
sibling_list) {
 
sibling->group_leader = sibling;
+   list_del_init(>sibling_list);
 
/* Inherit group flags from the previous leader */
sibling->group_caps = event->group_caps;
 
if (!RB_EMPTY_NODE(>group_node)) {
-   list_del_init(>sibling_list);
add_event_to_groups(sibling, event->ctx);
 
if (sibling->state == PERF_EVENT_STATE_ACTIVE) {

[tip:perf/core] perf/core: Clear sibling list of detached events

[tip-bot for Mark Rutland]

Commit-ID:  bbb68468641547d56c83012670bcaf77f3dacd64
Gitweb: https://git.kernel.org/tip/bbb68468641547d56c83012670bcaf77f3dacd64
Author: Mark Rutland 
AuthorDate: Fri, 16 Mar 2018 12:51:40 +
Committer:  Thomas Gleixner 
CommitDate: Fri, 16 Mar 2018 15:34:24 +0100

perf/core: Clear sibling list of detached events

When perf_group_dettach() is called on a group leader, it updates each
sibling's group_leader field to point to that sibling, effectively
upgrading each siblnig to a group leader. After perf_group_detach has
completed, the caller may free the leader event.

We only remove siblings from the group leader's sibling_list when the
leader has a non-empty group_node. This was fine prior to commit:

  8343aae66167df67 ("perf/core: Remove perf_event::group_entry")

... as the sibling's sibling_list would be empty. However, now that we
use the sibling_list field as both the list head and the list entry,
this leaves each sibling with a non-empty sibling list, including the
stale leader event.

If perf_group_detach() is subsequently called on a sibling, it will
appear to be a group leader, and we'll walk the sibling_list,
potentially dereferencing these stale events. In 0day testing, this has
been observed to result in kernel panics.

Let's avoid this by always removing siblings from the sibling list when
we promote them to leaders.

Fixes: 8343aae66167df67 ("perf/core: Remove perf_event::group_entry")
Signed-off-by: Mark Rutland 
Signed-off-by: Thomas Gleixner 
Cc: vincent.wea...@maine.edu
Cc: Peter Zijlstra 
Cc: torva...@linux-foundation.org
Cc: Alexey Budankov 
Cc: valery.cherepenni...@intel.com
Cc: linux-tip-comm...@vger.kernel.org
Cc: eran...@google.com
Cc: a...@redhat.com
Cc: alexander.shish...@linux.intel.com
Cc: davi...@google.com
Cc: kan.li...@intel.com
Cc: dmitry.proho...@intel.com
Cc: Jiri Olsa 
Link: 
https://lkml.kernel.org/r/20180316131741.3svgr64yibc6v...@lakrids.cambridge.arm.com
---
 kernel/events/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 4d7a460d6669..2776a660db15 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1906,12 +1906,12 @@ static void perf_group_detach(struct perf_event *event)
list_for_each_entry_safe(sibling, tmp, >sibling_list, 
sibling_list) {
 
sibling->group_leader = sibling;
+   list_del_init(>sibling_list);
 
/* Inherit group flags from the previous leader */
sibling->group_caps = event->group_caps;
 
if (!RB_EMPTY_NODE(>group_node)) {
-   list_del_init(>sibling_list);
add_event_to_groups(sibling, event->ctx);
 
if (sibling->state == PERF_EVENT_STATE_ACTIVE) {

[tip:efi/core] efi/arm*: Only register page tables when they exist

[tip-bot for Mark Rutland]

Commit-ID:  6b31a2fa1e8f7bc6c2a474b4a12dad7a145cf83d
Gitweb: https://git.kernel.org/tip/6b31a2fa1e8f7bc6c2a474b4a12dad7a145cf83d
Author: Mark Rutland 
AuthorDate: Thu, 8 Mar 2018 08:00:09 +
Committer:  Ingo Molnar 
CommitDate: Fri, 9 Mar 2018 08:58:21 +0100

efi/arm*: Only register page tables when they exist

Currently the arm/arm64 runtime code registers the runtime servies
pagetables with ptdump regardless of whether runtime services page
tables have been created.

As efi_mm.pgd is NULL in these cases, attempting to dump the efi page
tables results in a NULL pointer dereference in the ptdump code:

/sys/kernel/debug# cat efi_page_tables
[  479.522600] Unable to handle kernel NULL pointer dereference at virtual 
address 
[  479.522715] Mem abort info:
[  479.522764]   ESR = 0x9606
[  479.522850]   Exception class = DABT (current EL), IL = 32 bits
[  479.522899]   SET = 0, FnV = 0
[  479.522937]   EA = 0, S1PTW = 0
[  479.528200] Data abort info:
[  479.528230]   ISV = 0, ISS = 0x0006
[  479.528317]   CM = 0, WnR = 0
[  479.528317] user pgtable: 4k pages, 48-bit VAs, pgd = 64ab0cb0
[  479.528449] [] *pgd=fbbe4003, *pud=fb66e003, 
*pmd=
[  479.528600] Internal error: Oops: 9606 [#1] PREEMPT SMP
[  479.528664] Modules linked in:
[  479.528699] CPU: 0 PID: 2457 Comm: cat Not tainted 
4.15.0-rc3-00065-g2ad2ee7ecb5c-dirty #7
[  479.528799] Hardware name: FVP Base (DT)
[  479.528899] pstate: 0049 (nzcv daif +PAN -UAO)
[  479.528941] pc : walk_pgd.isra.1+0x20/0x1d0
[  479.529011] lr : ptdump_walk_pgd+0x30/0x50
[  479.529105] sp : 0bf4bc20
[  479.529185] x29: 0bf4bc20 x28: 9d22e000
[  479.529271] x27: 0002 x26: 80007b4c63c0
[  479.529358] x25: 014000c0 x24: 80007c098900
[  479.529445] x23: 0bf4beb8 x22: 
[  479.529532] x21: 0bf4bd70 x20: 0001
[  479.529618] x19: 0bf4bcb0 x18: 
[  479.529760] x17: 0041a1c8 x16: 082139d8
[  479.529800] x15: 9d3c6030 x14: 9d2527f4
[  479.529924] x13: 03f3 x12: 0038
[  479.53] x11: 0003 x10: 0101010101010101
[  479.530099] x9 : 17e94050 x8 : 003f
[  479.530226] x7 :  x6 : 
[  479.530313] x5 : 0001 x4 : 
[  479.530416] x3 : 09069fd8 x2 : 
[  479.530500] x1 :  x0 : 
[  479.530599] Process cat (pid: 2457, stack limit = 0x5d1b0e6f)
[  479.530660] Call trace:
[  479.530746]  walk_pgd.isra.1+0x20/0x1d0
[  479.530833]  ptdump_walk_pgd+0x30/0x50
[  479.530907]  ptdump_show+0x10/0x20
[  479.530920]  seq_read+0xc8/0x470
[  479.531023]  full_proxy_read+0x60/0x90
[  479.531100]  __vfs_read+0x18/0x100
[  479.531180]  vfs_read+0x88/0x160
[  479.531267]  SyS_read+0x48/0xb0
[  479.531299]  el0_svc_naked+0x20/0x24
[  479.531400] Code: 91400420 f90033a0 a90707a2 f9403fa0 (f940)
[  479.531499] ---[ end trace bfe8e28d8acb2b67 ]---
Segmentation fault

Let's avoid this problem by only registering the tables after their
successful creation, which is also less confusing when EFI runtime
services are not in use.

Reported-by: Will Deacon 
Signed-off-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Acked-by: Will Deacon 
Cc: Linus Torvalds 
Cc: Matt Fleming 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Link: http://lkml.kernel.org/r/20180308080020.22828-2-ard.biesheu...@linaro.org
Signed-off-by: Ingo Molnar 
---
 drivers/firmware/efi/arm-runtime.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/firmware/efi/arm-runtime.c 
b/drivers/firmware/efi/arm-runtime.c
index 1cc41c3d6315..86a1ad17a32e 100644
--- a/drivers/firmware/efi/arm-runtime.c
+++ b/drivers/firmware/efi/arm-runtime.c
@@ -54,6 +54,9 @@ static struct ptdump_info efi_ptdump_info = {
 
 static int __init ptdump_init(void)
 {
+   if (!efi_enabled(EFI_RUNTIME_SERVICES))
+   return 0;
+
return ptdump_debugfs_register(_ptdump_info, "efi_page_tables");
 }
 device_initcall(ptdump_init);

[tip:efi/core] efi/arm*: Only register page tables when they exist

[tip-bot for Mark Rutland]

Commit-ID:  6b31a2fa1e8f7bc6c2a474b4a12dad7a145cf83d
Gitweb: https://git.kernel.org/tip/6b31a2fa1e8f7bc6c2a474b4a12dad7a145cf83d
Author: Mark Rutland 
AuthorDate: Thu, 8 Mar 2018 08:00:09 +
Committer:  Ingo Molnar 
CommitDate: Fri, 9 Mar 2018 08:58:21 +0100

efi/arm*: Only register page tables when they exist

Currently the arm/arm64 runtime code registers the runtime servies
pagetables with ptdump regardless of whether runtime services page
tables have been created.

As efi_mm.pgd is NULL in these cases, attempting to dump the efi page
tables results in a NULL pointer dereference in the ptdump code:

/sys/kernel/debug# cat efi_page_tables
[  479.522600] Unable to handle kernel NULL pointer dereference at virtual 
address 
[  479.522715] Mem abort info:
[  479.522764]   ESR = 0x9606
[  479.522850]   Exception class = DABT (current EL), IL = 32 bits
[  479.522899]   SET = 0, FnV = 0
[  479.522937]   EA = 0, S1PTW = 0
[  479.528200] Data abort info:
[  479.528230]   ISV = 0, ISS = 0x0006
[  479.528317]   CM = 0, WnR = 0
[  479.528317] user pgtable: 4k pages, 48-bit VAs, pgd = 64ab0cb0
[  479.528449] [] *pgd=fbbe4003, *pud=fb66e003, 
*pmd=
[  479.528600] Internal error: Oops: 9606 [#1] PREEMPT SMP
[  479.528664] Modules linked in:
[  479.528699] CPU: 0 PID: 2457 Comm: cat Not tainted 
4.15.0-rc3-00065-g2ad2ee7ecb5c-dirty #7
[  479.528799] Hardware name: FVP Base (DT)
[  479.528899] pstate: 0049 (nzcv daif +PAN -UAO)
[  479.528941] pc : walk_pgd.isra.1+0x20/0x1d0
[  479.529011] lr : ptdump_walk_pgd+0x30/0x50
[  479.529105] sp : 0bf4bc20
[  479.529185] x29: 0bf4bc20 x28: 9d22e000
[  479.529271] x27: 0002 x26: 80007b4c63c0
[  479.529358] x25: 014000c0 x24: 80007c098900
[  479.529445] x23: 0bf4beb8 x22: 
[  479.529532] x21: 0bf4bd70 x20: 0001
[  479.529618] x19: 0bf4bcb0 x18: 
[  479.529760] x17: 0041a1c8 x16: 082139d8
[  479.529800] x15: 9d3c6030 x14: 9d2527f4
[  479.529924] x13: 03f3 x12: 0038
[  479.53] x11: 0003 x10: 0101010101010101
[  479.530099] x9 : 17e94050 x8 : 003f
[  479.530226] x7 :  x6 : 
[  479.530313] x5 : 0001 x4 : 
[  479.530416] x3 : 09069fd8 x2 : 
[  479.530500] x1 :  x0 : 
[  479.530599] Process cat (pid: 2457, stack limit = 0x5d1b0e6f)
[  479.530660] Call trace:
[  479.530746]  walk_pgd.isra.1+0x20/0x1d0
[  479.530833]  ptdump_walk_pgd+0x30/0x50
[  479.530907]  ptdump_show+0x10/0x20
[  479.530920]  seq_read+0xc8/0x470
[  479.531023]  full_proxy_read+0x60/0x90
[  479.531100]  __vfs_read+0x18/0x100
[  479.531180]  vfs_read+0x88/0x160
[  479.531267]  SyS_read+0x48/0xb0
[  479.531299]  el0_svc_naked+0x20/0x24
[  479.531400] Code: 91400420 f90033a0 a90707a2 f9403fa0 (f940)
[  479.531499] ---[ end trace bfe8e28d8acb2b67 ]---
Segmentation fault

Let's avoid this problem by only registering the tables after their
successful creation, which is also less confusing when EFI runtime
services are not in use.

Reported-by: Will Deacon 
Signed-off-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Acked-by: Will Deacon 
Cc: Linus Torvalds 
Cc: Matt Fleming 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Link: http://lkml.kernel.org/r/20180308080020.22828-2-ard.biesheu...@linaro.org
Signed-off-by: Ingo Molnar 
---
 drivers/firmware/efi/arm-runtime.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/firmware/efi/arm-runtime.c 
b/drivers/firmware/efi/arm-runtime.c
index 1cc41c3d6315..86a1ad17a32e 100644
--- a/drivers/firmware/efi/arm-runtime.c
+++ b/drivers/firmware/efi/arm-runtime.c
@@ -54,6 +54,9 @@ static struct ptdump_info efi_ptdump_info = {
 
 static int __init ptdump_init(void)
 {
+   if (!efi_enabled(EFI_RUNTIME_SERVICES))
+   return 0;
+
return ptdump_debugfs_register(_ptdump_info, "efi_page_tables");
 }
 device_initcall(ptdump_init);

[tip:x86/pti] Documentation: Document array_index_nospec

[tip-bot for Mark Rutland]

Commit-ID:  f84a56f73dddaeac1dba8045b007f742f61cd2da
Gitweb: https://git.kernel.org/tip/f84a56f73dddaeac1dba8045b007f742f61cd2da
Author: Mark Rutland 
AuthorDate: Mon, 29 Jan 2018 17:02:16 -0800
Committer:  Thomas Gleixner 
CommitDate: Tue, 30 Jan 2018 21:54:28 +0100

Documentation: Document array_index_nospec

Document the rationale and usage of the new array_index_nospec() helper.

Signed-off-by: Mark Rutland 
Signed-off-by: Will Deacon 
Signed-off-by: Dan Williams 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Kees Cook 
Cc: linux-a...@vger.kernel.org
Cc: Jonathan Corbet 
Cc: Peter Zijlstra 
Cc: gre...@linuxfoundation.org
Cc: kernel-harden...@lists.openwall.com
Cc: torva...@linux-foundation.org
Cc: a...@linux.intel.com
Link: 
https://lkml.kernel.org/r/151727413645.33451.15878817161436755393.st...@dwillia2-desk3.amr.corp.intel.com

---
 Documentation/speculation.txt | 90 +++
 1 file changed, 90 insertions(+)

diff --git a/Documentation/speculation.txt b/Documentation/speculation.txt
new file mode 100644
index 000..e9e6cba
--- /dev/null
+++ b/Documentation/speculation.txt
@@ -0,0 +1,90 @@
+This document explains potential effects of speculation, and how undesirable
+effects can be mitigated portably using common APIs.
+
+===
+Speculation
+===
+
+To improve performance and minimize average latencies, many contemporary CPUs
+employ speculative execution techniques such as branch prediction, performing
+work which may be discarded at a later stage.
+
+Typically speculative execution cannot be observed from architectural state,
+such as the contents of registers. However, in some cases it is possible to
+observe its impact on microarchitectural state, such as the presence or
+absence of data in caches. Such state may form side-channels which can be
+observed to extract secret information.
+
+For example, in the presence of branch prediction, it is possible for bounds
+checks to be ignored by code which is speculatively executed. Consider the
+following code:
+
+   int load_array(int *array, unsigned int index)
+   {
+   if (index >= MAX_ARRAY_ELEMS)
+   return 0;
+   else
+   return array[index];
+   }
+
+Which, on arm64, may be compiled to an assembly sequence such as:
+
+   CMP , #MAX_ARRAY_ELEMS
+   B.LTless
+   MOV , #0
+   RET
+  less:
+   LDR , [, ]
+   RET
+
+It is possible that a CPU mis-predicts the conditional branch, and
+speculatively loads array[index], even if index >= MAX_ARRAY_ELEMS. This
+value will subsequently be discarded, but the speculated load may affect
+microarchitectural state which can be subsequently measured.
+
+More complex sequences involving multiple dependent memory accesses may
+result in sensitive information being leaked. Consider the following
+code, building on the prior example:
+
+   int load_dependent_arrays(int *arr1, int *arr2, int index)
+   {
+   int val1, val2,
+
+   val1 = load_array(arr1, index);
+   val2 = load_array(arr2, val1);
+
+   return val2;
+   }
+
+Under speculation, the first call to load_array() may return the value
+of an out-of-bounds address, while the second call will influence
+microarchitectural state dependent on this value. This may provide an
+arbitrary read primitive.
+
+
+Mitigating speculation side-channels
+
+
+The kernel provides a generic API to ensure that bounds checks are
+respected even under speculation. Architectures which are affected by
+speculation-based side-channels are expected to implement these
+primitives.
+
+The array_index_nospec() helper in  can be used to
+prevent information from being leaked via side-channels.
+
+A call to array_index_nospec(index, size) returns a sanitized index
+value that is bounded to [0, size) even under cpu speculation
+conditions.
+
+This can be used to protect the earlier load_array() example:
+
+   int load_array(int *array, unsigned int index)
+   {
+   if (index >= MAX_ARRAY_ELEMS)
+   return 0;
+   else {
+   index = array_index_nospec(index, MAX_ARRAY_ELEMS);
+   return array[index];
+   }
+   }

[tip:x86/pti] Documentation: Document array_index_nospec

[tip-bot for Mark Rutland]

Commit-ID:  f84a56f73dddaeac1dba8045b007f742f61cd2da
Gitweb: https://git.kernel.org/tip/f84a56f73dddaeac1dba8045b007f742f61cd2da
Author: Mark Rutland 
AuthorDate: Mon, 29 Jan 2018 17:02:16 -0800
Committer:  Thomas Gleixner 
CommitDate: Tue, 30 Jan 2018 21:54:28 +0100

Documentation: Document array_index_nospec

Document the rationale and usage of the new array_index_nospec() helper.

Signed-off-by: Mark Rutland 
Signed-off-by: Will Deacon 
Signed-off-by: Dan Williams 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Kees Cook 
Cc: linux-a...@vger.kernel.org
Cc: Jonathan Corbet 
Cc: Peter Zijlstra 
Cc: gre...@linuxfoundation.org
Cc: kernel-harden...@lists.openwall.com
Cc: torva...@linux-foundation.org
Cc: a...@linux.intel.com
Link: 
https://lkml.kernel.org/r/151727413645.33451.15878817161436755393.st...@dwillia2-desk3.amr.corp.intel.com

---
 Documentation/speculation.txt | 90 +++
 1 file changed, 90 insertions(+)

diff --git a/Documentation/speculation.txt b/Documentation/speculation.txt
new file mode 100644
index 000..e9e6cba
--- /dev/null
+++ b/Documentation/speculation.txt
@@ -0,0 +1,90 @@
+This document explains potential effects of speculation, and how undesirable
+effects can be mitigated portably using common APIs.
+
+===
+Speculation
+===
+
+To improve performance and minimize average latencies, many contemporary CPUs
+employ speculative execution techniques such as branch prediction, performing
+work which may be discarded at a later stage.
+
+Typically speculative execution cannot be observed from architectural state,
+such as the contents of registers. However, in some cases it is possible to
+observe its impact on microarchitectural state, such as the presence or
+absence of data in caches. Such state may form side-channels which can be
+observed to extract secret information.
+
+For example, in the presence of branch prediction, it is possible for bounds
+checks to be ignored by code which is speculatively executed. Consider the
+following code:
+
+   int load_array(int *array, unsigned int index)
+   {
+   if (index >= MAX_ARRAY_ELEMS)
+   return 0;
+   else
+   return array[index];
+   }
+
+Which, on arm64, may be compiled to an assembly sequence such as:
+
+   CMP , #MAX_ARRAY_ELEMS
+   B.LTless
+   MOV , #0
+   RET
+  less:
+   LDR , [, ]
+   RET
+
+It is possible that a CPU mis-predicts the conditional branch, and
+speculatively loads array[index], even if index >= MAX_ARRAY_ELEMS. This
+value will subsequently be discarded, but the speculated load may affect
+microarchitectural state which can be subsequently measured.
+
+More complex sequences involving multiple dependent memory accesses may
+result in sensitive information being leaked. Consider the following
+code, building on the prior example:
+
+   int load_dependent_arrays(int *arr1, int *arr2, int index)
+   {
+   int val1, val2,
+
+   val1 = load_array(arr1, index);
+   val2 = load_array(arr2, val1);
+
+   return val2;
+   }
+
+Under speculation, the first call to load_array() may return the value
+of an out-of-bounds address, while the second call will influence
+microarchitectural state dependent on this value. This may provide an
+arbitrary read primitive.
+
+
+Mitigating speculation side-channels
+
+
+The kernel provides a generic API to ensure that bounds checks are
+respected even under speculation. Architectures which are affected by
+speculation-based side-channels are expected to implement these
+primitives.
+
+The array_index_nospec() helper in  can be used to
+prevent information from being leaked via side-channels.
+
+A call to array_index_nospec(index, size) returns a sanitized index
+value that is bounded to [0, size) even under cpu speculation
+conditions.
+
+This can be used to protect the earlier load_array() example:
+
+   int load_array(int *array, unsigned int index)
+   {
+   if (index >= MAX_ARRAY_ELEMS)
+   return 0;
+   else {
+   index = array_index_nospec(index, MAX_ARRAY_ELEMS);
+   return array[index];
+   }
+   }

[tip:locking/urgent] checkpatch: Remove ACCESS_ONCE() warning

[tip-bot for Mark Rutland]

Commit-ID:  8cb562b1d56fad42cbee44bdc9bc64cea41a0a8c
Gitweb: https://git.kernel.org/tip/8cb562b1d56fad42cbee44bdc9bc64cea41a0a8c
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:24 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:10 +0100

checkpatch: Remove ACCESS_ONCE() warning

Now that ACCESS_ONCE() has been excised from the kernel, any uses will
result in a build error, and we no longer need to whine about it in
checkpatch.

This patch removes the newly redundant warning.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Acked-by: Joe Perches 
Cc: Andy Whitcroft 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@redhat.com
Link: http://lkml.kernel.org/r/20171127103824.36526-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 scripts/checkpatch.pl | 22 --
 1 file changed, 22 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 040aa79..31031f1 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -6233,28 +6233,6 @@ sub process {
}
}
 
-# whine about ACCESS_ONCE
-   if ($^V && $^V ge 5.10.0 &&
-   $line =~ 
/\bACCESS_ONCE\s*$balanced_parens\s*(=(?!=))?\s*($FuncArg)?/) {
-   my $par = $1;
-   my $eq = $2;
-   my $fun = $3;
-   $par =~ s/^\(\s*(.*)\s*\)$/$1/;
-   if (defined($eq)) {
-   if (WARN("PREFER_WRITE_ONCE",
-"Prefer WRITE_ONCE(, ) over 
ACCESS_ONCE() = \n" . $herecurr) &&
-   $fix) {
-   $fixed[$fixlinenr] =~ 
s/\bACCESS_ONCE\s*\(\s*\Q$par\E\s*\)\s*$eq\s*\Q$fun\E/WRITE_ONCE($par, $fun)/;
-   }
-   } else {
-   if (WARN("PREFER_READ_ONCE",
-"Prefer READ_ONCE() over 
ACCESS_ONCE()\n" . $herecurr) &&
-   $fix) {
-   $fixed[$fixlinenr] =~ 
s/\bACCESS_ONCE\s*\(\s*\Q$par\E\s*\)/READ_ONCE($par)/;
-   }
-   }
-   }
-
 # check for mutex_trylock_recursive usage
if ($line =~ /mutex_trylock_recursive/) {
ERROR("LOCKING",

[tip:locking/urgent] checkpatch: Remove ACCESS_ONCE() warning

[tip-bot for Mark Rutland]

Commit-ID:  8cb562b1d56fad42cbee44bdc9bc64cea41a0a8c
Gitweb: https://git.kernel.org/tip/8cb562b1d56fad42cbee44bdc9bc64cea41a0a8c
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:24 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:10 +0100

checkpatch: Remove ACCESS_ONCE() warning

Now that ACCESS_ONCE() has been excised from the kernel, any uses will
result in a build error, and we no longer need to whine about it in
checkpatch.

This patch removes the newly redundant warning.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Acked-by: Joe Perches 
Cc: Andy Whitcroft 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@redhat.com
Link: http://lkml.kernel.org/r/20171127103824.36526-5-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 scripts/checkpatch.pl | 22 --
 1 file changed, 22 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 040aa79..31031f1 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -6233,28 +6233,6 @@ sub process {
}
}
 
-# whine about ACCESS_ONCE
-   if ($^V && $^V ge 5.10.0 &&
-   $line =~ 
/\bACCESS_ONCE\s*$balanced_parens\s*(=(?!=))?\s*($FuncArg)?/) {
-   my $par = $1;
-   my $eq = $2;
-   my $fun = $3;
-   $par =~ s/^\(\s*(.*)\s*\)$/$1/;
-   if (defined($eq)) {
-   if (WARN("PREFER_WRITE_ONCE",
-"Prefer WRITE_ONCE(, ) over 
ACCESS_ONCE() = \n" . $herecurr) &&
-   $fix) {
-   $fixed[$fixlinenr] =~ 
s/\bACCESS_ONCE\s*\(\s*\Q$par\E\s*\)\s*$eq\s*\Q$fun\E/WRITE_ONCE($par, $fun)/;
-   }
-   } else {
-   if (WARN("PREFER_READ_ONCE",
-"Prefer READ_ONCE() over 
ACCESS_ONCE()\n" . $herecurr) &&
-   $fix) {
-   $fixed[$fixlinenr] =~ 
s/\bACCESS_ONCE\s*\(\s*\Q$par\E\s*\)/READ_ONCE($par)/;
-   }
-   }
-   }
-
 # check for mutex_trylock_recursive usage
if ($line =~ /mutex_trylock_recursive/) {
ERROR("LOCKING",

[tip:locking/urgent] compiler.h: Remove ACCESS_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  b899a850431e2dd0943205a63a68573f3e312d0d
Gitweb: https://git.kernel.org/tip/b899a850431e2dd0943205a63a68573f3e312d0d
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:23 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:10 +0100

compiler.h: Remove ACCESS_ONCE()

There are no longer any kernelspace uses of ACCESS_ONCE(), so we can
remove the definition from .

This patch removes the ACCESS_ONCE() definition, and updates comments
which referred to it. At the same time, some inconsistent and redundant
whitespace is removed from comments.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Cc: Arnaldo Carvalho de Melo 
Cc: Joe Perches 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@canonical.com
Link: http://lkml.kernel.org/r/20171127103824.36526-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/compiler.h | 47 +++
 1 file changed, 11 insertions(+), 36 deletions(-)

diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 188ed9f..52e611a 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -220,21 +220,21 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
 /*
  * Prevent the compiler from merging or refetching reads or writes. The
  * compiler is also forbidden from reordering successive instances of
- * READ_ONCE, WRITE_ONCE and ACCESS_ONCE (see below), but only when the
- * compiler is aware of some particular ordering.  One way to make the
- * compiler aware of ordering is to put the two invocations of READ_ONCE,
- * WRITE_ONCE or ACCESS_ONCE() in different C statements.
+ * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
+ * particular ordering. One way to make the compiler aware of ordering is to
+ * put the two invocations of READ_ONCE or WRITE_ONCE in different C
+ * statements.
  *
- * In contrast to ACCESS_ONCE these two macros will also work on aggregate
- * data types like structs or unions. If the size of the accessed data
- * type exceeds the word size of the machine (e.g., 32 bits or 64 bits)
- * READ_ONCE() and WRITE_ONCE() will fall back to memcpy(). There's at
- * least two memcpy()s: one for the __builtin_memcpy() and then one for
- * the macro doing the copy of variable - '__u' allocated on the stack.
+ * These two macros will also work on aggregate data types like structs or
+ * unions. If the size of the accessed data type exceeds the word size of
+ * the machine (e.g., 32 bits or 64 bits) READ_ONCE() and WRITE_ONCE() will
+ * fall back to memcpy(). There's at least two memcpy()s: one for the
+ * __builtin_memcpy() and then one for the macro doing the copy of variable
+ * - '__u' allocated on the stack.
  *
  * Their two major use cases are: (1) Mediating communication between
  * process-level code and irq/NMI handlers, all running on the same CPU,
- * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
+ * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
  * mutilate accesses that either do not require ordering or that interact
  * with an explicit memory barrier or atomic instruction that provides the
  * required ordering.
@@ -327,29 +327,4 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
compiletime_assert(__native_word(t),\
"Need native word sized stores/loads for atomicity.")
 
-/*
- * Prevent the compiler from merging or refetching accesses.  The compiler
- * is also forbidden from reordering successive instances of ACCESS_ONCE(),
- * but only when the compiler is aware of some particular ordering.  One way
- * to make the compiler aware of ordering is to put the two invocations of
- * ACCESS_ONCE() in different C statements.
- *
- * ACCESS_ONCE will only work on scalar types. For union types, ACCESS_ONCE
- * on a union member will work as long as the size of the member matches the
- * size of the union and the size is smaller than word size.
- *
- * The major use cases of ACCESS_ONCE used to be (1) Mediating communication
- * between process-level code and irq/NMI handlers, all running on the same 
CPU,
- * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
- * mutilate accesses that either do not require ordering or that interact
- * with an explicit memory barrier or atomic instruction that provides the
- * required ordering.
- *
- * If possible use READ_ONCE()/WRITE_ONCE() instead.
- */
-#define __ACCESS_ONCE(x) ({ \
-__maybe_unused typeof(x) __var = (__force typeof(x)) 0; \
-   (volatile typeof(x) *)&(x); })
-#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
-
 #endif /* 

[tip:locking/urgent] compiler.h: Remove ACCESS_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  b899a850431e2dd0943205a63a68573f3e312d0d
Gitweb: https://git.kernel.org/tip/b899a850431e2dd0943205a63a68573f3e312d0d
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:23 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:10 +0100

compiler.h: Remove ACCESS_ONCE()

There are no longer any kernelspace uses of ACCESS_ONCE(), so we can
remove the definition from .

This patch removes the ACCESS_ONCE() definition, and updates comments
which referred to it. At the same time, some inconsistent and redundant
whitespace is removed from comments.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Cc: Arnaldo Carvalho de Melo 
Cc: Joe Perches 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@canonical.com
Link: http://lkml.kernel.org/r/20171127103824.36526-4-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/compiler.h | 47 +++
 1 file changed, 11 insertions(+), 36 deletions(-)

diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 188ed9f..52e611a 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -220,21 +220,21 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
 /*
  * Prevent the compiler from merging or refetching reads or writes. The
  * compiler is also forbidden from reordering successive instances of
- * READ_ONCE, WRITE_ONCE and ACCESS_ONCE (see below), but only when the
- * compiler is aware of some particular ordering.  One way to make the
- * compiler aware of ordering is to put the two invocations of READ_ONCE,
- * WRITE_ONCE or ACCESS_ONCE() in different C statements.
+ * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
+ * particular ordering. One way to make the compiler aware of ordering is to
+ * put the two invocations of READ_ONCE or WRITE_ONCE in different C
+ * statements.
  *
- * In contrast to ACCESS_ONCE these two macros will also work on aggregate
- * data types like structs or unions. If the size of the accessed data
- * type exceeds the word size of the machine (e.g., 32 bits or 64 bits)
- * READ_ONCE() and WRITE_ONCE() will fall back to memcpy(). There's at
- * least two memcpy()s: one for the __builtin_memcpy() and then one for
- * the macro doing the copy of variable - '__u' allocated on the stack.
+ * These two macros will also work on aggregate data types like structs or
+ * unions. If the size of the accessed data type exceeds the word size of
+ * the machine (e.g., 32 bits or 64 bits) READ_ONCE() and WRITE_ONCE() will
+ * fall back to memcpy(). There's at least two memcpy()s: one for the
+ * __builtin_memcpy() and then one for the macro doing the copy of variable
+ * - '__u' allocated on the stack.
  *
  * Their two major use cases are: (1) Mediating communication between
  * process-level code and irq/NMI handlers, all running on the same CPU,
- * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
+ * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
  * mutilate accesses that either do not require ordering or that interact
  * with an explicit memory barrier or atomic instruction that provides the
  * required ordering.
@@ -327,29 +327,4 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
compiletime_assert(__native_word(t),\
"Need native word sized stores/loads for atomicity.")
 
-/*
- * Prevent the compiler from merging or refetching accesses.  The compiler
- * is also forbidden from reordering successive instances of ACCESS_ONCE(),
- * but only when the compiler is aware of some particular ordering.  One way
- * to make the compiler aware of ordering is to put the two invocations of
- * ACCESS_ONCE() in different C statements.
- *
- * ACCESS_ONCE will only work on scalar types. For union types, ACCESS_ONCE
- * on a union member will work as long as the size of the member matches the
- * size of the union and the size is smaller than word size.
- *
- * The major use cases of ACCESS_ONCE used to be (1) Mediating communication
- * between process-level code and irq/NMI handlers, all running on the same 
CPU,
- * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
- * mutilate accesses that either do not require ordering or that interact
- * with an explicit memory barrier or atomic instruction that provides the
- * required ordering.
- *
- * If possible use READ_ONCE()/WRITE_ONCE() instead.
- */
-#define __ACCESS_ONCE(x) ({ \
-__maybe_unused typeof(x) __var = (__force typeof(x)) 0; \
-   (volatile typeof(x) *)&(x); })
-#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
-
 #endif /* __LINUX_COMPILER_H */

[tip:locking/urgent] tools/include: Remove ACCESS_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  2a22f692bbe0a7933acbd50045479ffc0fdf11f7
Gitweb: https://git.kernel.org/tip/2a22f692bbe0a7933acbd50045479ffc0fdf11f7
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:22 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:10 +0100

tools/include: Remove ACCESS_ONCE()

There are no longer any usersapce uses of ACCESS_ONCE(), so we can
remove the definition from our userspace , which is
only used by tools in the kernel directory (i.e. it isn't a uapi
header).

This patch removes the ACCESS_ONCE() definition, and updates comments
which referred to it. At the same time, some inconsistent and redundant
whitespace is removed from comments.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Cc: Arnaldo Carvalho de Melo 
Cc: Joe Perches 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@canonical.com
Link: http://lkml.kernel.org/r/20171127103824.36526-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 tools/include/linux/compiler.h | 21 +
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/tools/include/linux/compiler.h b/tools/include/linux/compiler.h
index 07fd03c..04e32f9 100644
--- a/tools/include/linux/compiler.h
+++ b/tools/include/linux/compiler.h
@@ -84,8 +84,6 @@
 
 #define uninitialized_var(x) x = *(&(x))
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
-
 #include 
 
 /*
@@ -135,20 +133,19 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
 /*
  * Prevent the compiler from merging or refetching reads or writes. The
  * compiler is also forbidden from reordering successive instances of
- * READ_ONCE, WRITE_ONCE and ACCESS_ONCE (see below), but only when the
- * compiler is aware of some particular ordering.  One way to make the
- * compiler aware of ordering is to put the two invocations of READ_ONCE,
- * WRITE_ONCE or ACCESS_ONCE() in different C statements.
+ * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
+ * particular ordering. One way to make the compiler aware of ordering is to
+ * put the two invocations of READ_ONCE or WRITE_ONCE in different C
+ * statements.
  *
- * In contrast to ACCESS_ONCE these two macros will also work on aggregate
- * data types like structs or unions. If the size of the accessed data
- * type exceeds the word size of the machine (e.g., 32 bits or 64 bits)
- * READ_ONCE() and WRITE_ONCE()  will fall back to memcpy and print a
- * compile-time warning.
+ * These two macros will also work on aggregate data types like structs or
+ * unions. If the size of the accessed data type exceeds the word size of
+ * the machine (e.g., 32 bits or 64 bits) READ_ONCE() and WRITE_ONCE() will
+ * fall back to memcpy and print a compile-time warning.
  *
  * Their two major use cases are: (1) Mediating communication between
  * process-level code and irq/NMI handlers, all running on the same CPU,
- * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
+ * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
  * mutilate accesses that either do not require ordering or that interact
  * with an explicit memory barrier or atomic instruction that provides the
  * required ordering.

[tip:locking/urgent] tools/include: Remove ACCESS_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  2a22f692bbe0a7933acbd50045479ffc0fdf11f7
Gitweb: https://git.kernel.org/tip/2a22f692bbe0a7933acbd50045479ffc0fdf11f7
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:22 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:10 +0100

tools/include: Remove ACCESS_ONCE()

There are no longer any usersapce uses of ACCESS_ONCE(), so we can
remove the definition from our userspace , which is
only used by tools in the kernel directory (i.e. it isn't a uapi
header).

This patch removes the ACCESS_ONCE() definition, and updates comments
which referred to it. At the same time, some inconsistent and redundant
whitespace is removed from comments.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Cc: Arnaldo Carvalho de Melo 
Cc: Joe Perches 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@canonical.com
Link: http://lkml.kernel.org/r/20171127103824.36526-3-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 tools/include/linux/compiler.h | 21 +
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/tools/include/linux/compiler.h b/tools/include/linux/compiler.h
index 07fd03c..04e32f9 100644
--- a/tools/include/linux/compiler.h
+++ b/tools/include/linux/compiler.h
@@ -84,8 +84,6 @@
 
 #define uninitialized_var(x) x = *(&(x))
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
-
 #include 
 
 /*
@@ -135,20 +133,19 @@ static __always_inline void __write_once_size(volatile 
void *p, void *res, int s
 /*
  * Prevent the compiler from merging or refetching reads or writes. The
  * compiler is also forbidden from reordering successive instances of
- * READ_ONCE, WRITE_ONCE and ACCESS_ONCE (see below), but only when the
- * compiler is aware of some particular ordering.  One way to make the
- * compiler aware of ordering is to put the two invocations of READ_ONCE,
- * WRITE_ONCE or ACCESS_ONCE() in different C statements.
+ * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
+ * particular ordering. One way to make the compiler aware of ordering is to
+ * put the two invocations of READ_ONCE or WRITE_ONCE in different C
+ * statements.
  *
- * In contrast to ACCESS_ONCE these two macros will also work on aggregate
- * data types like structs or unions. If the size of the accessed data
- * type exceeds the word size of the machine (e.g., 32 bits or 64 bits)
- * READ_ONCE() and WRITE_ONCE()  will fall back to memcpy and print a
- * compile-time warning.
+ * These two macros will also work on aggregate data types like structs or
+ * unions. If the size of the accessed data type exceeds the word size of
+ * the machine (e.g., 32 bits or 64 bits) READ_ONCE() and WRITE_ONCE() will
+ * fall back to memcpy and print a compile-time warning.
  *
  * Their two major use cases are: (1) Mediating communication between
  * process-level code and irq/NMI handlers, all running on the same CPU,
- * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
+ * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
  * mutilate accesses that either do not require ordering or that interact
  * with an explicit memory barrier or atomic instruction that provides the
  * required ordering.

[tip:locking/urgent] tools/perf: Convert ACCESS_ONCE() to READ_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  f971e511cb7d6f1b3730248cf2967d3ccdd8874c
Gitweb: https://git.kernel.org/tip/f971e511cb7d6f1b3730248cf2967d3ccdd8874c
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:21 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:09 +0100

tools/perf: Convert ACCESS_ONCE() to READ_ONCE()

Recently there was a treewide conversion of ACCESS_ONCE() to
{READ,WRITE}_ONCE(), but a new use was introduced concurrently by
commit:

  1695849735752d2a ("perf mmap: Move perf_mmap and methods to separate 
mmap.[ch] files")

Let's convert this over to READ_ONCE() so that we can remove the
ACCESS_ONCE() definitions in subsequent patches.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Reviewed-by: Paul E. McKenney 
Cc: Arnaldo Carvalho de Melo 
Cc: Joe Perches 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@canonical.com
Link: http://lkml.kernel.org/r/20171127103824.36526-2-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 tools/perf/util/mmap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/mmap.h b/tools/perf/util/mmap.h
index efd78b8..3a5cb5a 100644
--- a/tools/perf/util/mmap.h
+++ b/tools/perf/util/mmap.h
@@ -70,7 +70,7 @@ void perf_mmap__read_catchup(struct perf_mmap *md);
 static inline u64 perf_mmap__read_head(struct perf_mmap *mm)
 {
struct perf_event_mmap_page *pc = mm->base;
-   u64 head = ACCESS_ONCE(pc->data_head);
+   u64 head = READ_ONCE(pc->data_head);
rmb();
return head;
 }

[tip:locking/urgent] tools/perf: Convert ACCESS_ONCE() to READ_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  f971e511cb7d6f1b3730248cf2967d3ccdd8874c
Gitweb: https://git.kernel.org/tip/f971e511cb7d6f1b3730248cf2967d3ccdd8874c
Author: Mark Rutland 
AuthorDate: Mon, 27 Nov 2017 10:38:21 +
Committer:  Ingo Molnar 
CommitDate: Tue, 12 Dec 2017 13:22:09 +0100

tools/perf: Convert ACCESS_ONCE() to READ_ONCE()

Recently there was a treewide conversion of ACCESS_ONCE() to
{READ,WRITE}_ONCE(), but a new use was introduced concurrently by
commit:

  1695849735752d2a ("perf mmap: Move perf_mmap and methods to separate 
mmap.[ch] files")

Let's convert this over to READ_ONCE() so that we can remove the
ACCESS_ONCE() definitions in subsequent patches.

Tested-by: Paul E. McKenney 
Signed-off-by: Mark Rutland 
Reviewed-by: Paul E. McKenney 
Cc: Arnaldo Carvalho de Melo 
Cc: Joe Perches 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: a...@canonical.com
Link: http://lkml.kernel.org/r/20171127103824.36526-2-mark.rutl...@arm.com
Signed-off-by: Ingo Molnar 
---
 tools/perf/util/mmap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/mmap.h b/tools/perf/util/mmap.h
index efd78b8..3a5cb5a 100644
--- a/tools/perf/util/mmap.h
+++ b/tools/perf/util/mmap.h
@@ -70,7 +70,7 @@ void perf_mmap__read_catchup(struct perf_mmap *md);
 static inline u64 perf_mmap__read_head(struct perf_mmap *mm)
 {
struct perf_event_mmap_page *pc = mm->base;
-   u64 head = ACCESS_ONCE(pc->data_head);
+   u64 head = READ_ONCE(pc->data_head);
rmb();
return head;
 }

[tip:locking/core] locking/atomics, net/netlink/netfilter: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  14cd5d4a0125f643350e7fa12f5384f1fc2d3e9d
Gitweb: https://git.kernel.org/tip/14cd5d4a0125f643350e7fa12f5384f1fc2d3e9d
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:17 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:59 +0200

locking/atomics, net/netlink/netfilter: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts netlink and netfilter code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: David S. Miller 
Cc: Florian Westphal 
Cc: Jozsef Kadlecsik 
Cc: Linus Torvalds 
Cc: Pablo Neira Ayuso 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-7-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/genetlink.h   | 2 +-
 include/linux/netfilter/nfnetlink.h | 2 +-
 include/linux/rtnetlink.h   | 2 +-
 include/net/netfilter/nf_tables.h   | 4 ++--
 net/netfilter/ipvs/ip_vs_sync.c | 2 +-
 net/netfilter/nfnetlink_queue.c | 4 ++--
 6 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/linux/genetlink.h b/include/linux/genetlink.h
index a4c61cb..0e694cf 100644
--- a/include/linux/genetlink.h
+++ b/include/linux/genetlink.h
@@ -30,7 +30,7 @@ extern wait_queue_head_t genl_sk_destructing_waitq;
  * @p: The pointer to read, prior to dereferencing
  *
  * Return the value of the specified RCU-protected pointer, but omit
- * both the smp_read_barrier_depends() and the ACCESS_ONCE(), because
+ * both the smp_read_barrier_depends() and the READ_ONCE(), because
  * caller holds genl mutex.
  */
 #define genl_dereference(p)\
diff --git a/include/linux/netfilter/nfnetlink.h 
b/include/linux/netfilter/nfnetlink.h
index 41d04e9..0f47a4a 100644
--- a/include/linux/netfilter/nfnetlink.h
+++ b/include/linux/netfilter/nfnetlink.h
@@ -66,7 +66,7 @@ static inline bool lockdep_nfnl_is_held(__u8 subsys_id)
  * @ss: The nfnetlink subsystem ID
  *
  * Return the value of the specified RCU-protected pointer, but omit
- * both the smp_read_barrier_depends() and the ACCESS_ONCE(), because
+ * both the smp_read_barrier_depends() and the READ_ONCE(), because
  * caller holds the NFNL subsystem mutex.
  */
 #define nfnl_dereference(p, ss)\
diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
index dea59c8..765f7b9 100644
--- a/include/linux/rtnetlink.h
+++ b/include/linux/rtnetlink.h
@@ -67,7 +67,7 @@ static inline bool lockdep_rtnl_is_held(void)
  * @p: The pointer to read, prior to dereferencing
  *
  * Return the value of the specified RCU-protected pointer, but omit
- * both the smp_read_barrier_depends() and the ACCESS_ONCE(), because
+ * both the smp_read_barrier_depends() and the READ_ONCE(), because
  * caller holds RTNL.
  */
 #define rtnl_dereference(p)\
diff --git a/include/net/netfilter/nf_tables.h 
b/include/net/netfilter/nf_tables.h
index 0f5b12a..5c68e27 100644
--- a/include/net/netfilter/nf_tables.h
+++ b/include/net/netfilter/nf_tables.h
@@ -1164,8 +1164,8 @@ static inline u8 nft_genmask_next(const struct net *net)
 
 static inline u8 nft_genmask_cur(const struct net *net)
 {
-   /* Use ACCESS_ONCE() to prevent refetching the value for atomicity */
-   return 1 << ACCESS_ONCE(net->nft.gencursor);
+   /* Use READ_ONCE() to prevent refetching the value for atomicity */
+   return 1 << READ_ONCE(net->nft.gencursor);
 }
 
 #define NFT_GENMASK_ANY((1 << 0) | (1 << 1))
diff --git a/net/netfilter/ipvs/ip_vs_sync.c 

[tip:locking/core] locking/atomics, net/netlink/netfilter: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  14cd5d4a0125f643350e7fa12f5384f1fc2d3e9d
Gitweb: https://git.kernel.org/tip/14cd5d4a0125f643350e7fa12f5384f1fc2d3e9d
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:17 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:59 +0200

locking/atomics, net/netlink/netfilter: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts netlink and netfilter code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: David S. Miller 
Cc: Florian Westphal 
Cc: Jozsef Kadlecsik 
Cc: Linus Torvalds 
Cc: Pablo Neira Ayuso 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-7-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/genetlink.h   | 2 +-
 include/linux/netfilter/nfnetlink.h | 2 +-
 include/linux/rtnetlink.h   | 2 +-
 include/net/netfilter/nf_tables.h   | 4 ++--
 net/netfilter/ipvs/ip_vs_sync.c | 2 +-
 net/netfilter/nfnetlink_queue.c | 4 ++--
 6 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/linux/genetlink.h b/include/linux/genetlink.h
index a4c61cb..0e694cf 100644
--- a/include/linux/genetlink.h
+++ b/include/linux/genetlink.h
@@ -30,7 +30,7 @@ extern wait_queue_head_t genl_sk_destructing_waitq;
  * @p: The pointer to read, prior to dereferencing
  *
  * Return the value of the specified RCU-protected pointer, but omit
- * both the smp_read_barrier_depends() and the ACCESS_ONCE(), because
+ * both the smp_read_barrier_depends() and the READ_ONCE(), because
  * caller holds genl mutex.
  */
 #define genl_dereference(p)\
diff --git a/include/linux/netfilter/nfnetlink.h 
b/include/linux/netfilter/nfnetlink.h
index 41d04e9..0f47a4a 100644
--- a/include/linux/netfilter/nfnetlink.h
+++ b/include/linux/netfilter/nfnetlink.h
@@ -66,7 +66,7 @@ static inline bool lockdep_nfnl_is_held(__u8 subsys_id)
  * @ss: The nfnetlink subsystem ID
  *
  * Return the value of the specified RCU-protected pointer, but omit
- * both the smp_read_barrier_depends() and the ACCESS_ONCE(), because
+ * both the smp_read_barrier_depends() and the READ_ONCE(), because
  * caller holds the NFNL subsystem mutex.
  */
 #define nfnl_dereference(p, ss)\
diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
index dea59c8..765f7b9 100644
--- a/include/linux/rtnetlink.h
+++ b/include/linux/rtnetlink.h
@@ -67,7 +67,7 @@ static inline bool lockdep_rtnl_is_held(void)
  * @p: The pointer to read, prior to dereferencing
  *
  * Return the value of the specified RCU-protected pointer, but omit
- * both the smp_read_barrier_depends() and the ACCESS_ONCE(), because
+ * both the smp_read_barrier_depends() and the READ_ONCE(), because
  * caller holds RTNL.
  */
 #define rtnl_dereference(p)\
diff --git a/include/net/netfilter/nf_tables.h 
b/include/net/netfilter/nf_tables.h
index 0f5b12a..5c68e27 100644
--- a/include/net/netfilter/nf_tables.h
+++ b/include/net/netfilter/nf_tables.h
@@ -1164,8 +1164,8 @@ static inline u8 nft_genmask_next(const struct net *net)
 
 static inline u8 nft_genmask_cur(const struct net *net)
 {
-   /* Use ACCESS_ONCE() to prevent refetching the value for atomicity */
-   return 1 << ACCESS_ONCE(net->nft.gencursor);
+   /* Use READ_ONCE() to prevent refetching the value for atomicity */
+   return 1 << READ_ONCE(net->nft.gencursor);
 }
 
 #define NFT_GENMASK_ANY((1 << 0) | (1 << 1))
diff --git a/net/netfilter/ipvs/ip_vs_sync.c b/net/netfilter/ipvs/ip_vs_sync.c
index 0e5b64a..1cfffd4 100644
--- a/net/netfilter/ipvs/ip_vs_sync.c
+++ b/net/netfilter/ipvs/ip_vs_sync.c
@@ -457,7 +457,7 @@ static inline bool in_persistence(struct ip_vs_conn *cp)
 static int ip_vs_sync_conn_needed(struct netns_ipvs *ipvs,
 

[tip:locking/core] locking/atomics, selftests/powerpc: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  564cbc87937d5de1b5b666b0ed68122d6c74fa07
Gitweb: https://git.kernel.org/tip/564cbc87937d5de1b5b666b0ed68122d6c74fa07
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:21 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:02 +0200

locking/atomics, selftests/powerpc: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7).

To avoid fragility if/when that transformation occurs, and to align with
the preferred usage of {READ,WRITE}_ONCE(), this patch updates the DSCR
selftest code to use READ_ONCE() rather than ACCESS_ONCE(). There should
be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Michael Ellerman 
Cc: Benjamin Herrenschmidt 
Cc: Linus Torvalds 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Shuah Khan 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-11-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/testing/selftests/powerpc/dscr/dscr.h  | 2 +-
 tools/testing/selftests/powerpc/dscr/dscr_default_test.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/powerpc/dscr/dscr.h 
b/tools/testing/selftests/powerpc/dscr/dscr.h
index 18ea223b..cdb840b 100644
--- a/tools/testing/selftests/powerpc/dscr/dscr.h
+++ b/tools/testing/selftests/powerpc/dscr/dscr.h
@@ -39,7 +39,7 @@
 #define rmb()  asm volatile("lwsync":::"memory")
 #define wmb()  asm volatile("lwsync":::"memory")
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
+#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
 
 /* Prilvilege state DSCR access */
 inline unsigned long get_dscr(void)
diff --git a/tools/testing/selftests/powerpc/dscr/dscr_default_test.c 
b/tools/testing/selftests/powerpc/dscr/dscr_default_test.c
index df17c3b..9e1a37e 100644
--- a/tools/testing/selftests/powerpc/dscr/dscr_default_test.c
+++ b/tools/testing/selftests/powerpc/dscr/dscr_default_test.c
@@ -27,7 +27,7 @@ static void *do_test(void *in)
unsigned long d, cur_dscr, cur_dscr_usr;
unsigned long s1, s2;
 
-   s1 = ACCESS_ONCE(sequence);
+   s1 = READ_ONCE(sequence);
if (s1 & 1)
continue;
rmb();

[tip:locking/core] locking/atomics, selftests/powerpc: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  564cbc87937d5de1b5b666b0ed68122d6c74fa07
Gitweb: https://git.kernel.org/tip/564cbc87937d5de1b5b666b0ed68122d6c74fa07
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:21 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:02 +0200

locking/atomics, selftests/powerpc: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7).

To avoid fragility if/when that transformation occurs, and to align with
the preferred usage of {READ,WRITE}_ONCE(), this patch updates the DSCR
selftest code to use READ_ONCE() rather than ACCESS_ONCE(). There should
be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Michael Ellerman 
Cc: Benjamin Herrenschmidt 
Cc: Linus Torvalds 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Shuah Khan 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-11-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/testing/selftests/powerpc/dscr/dscr.h  | 2 +-
 tools/testing/selftests/powerpc/dscr/dscr_default_test.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/powerpc/dscr/dscr.h 
b/tools/testing/selftests/powerpc/dscr/dscr.h
index 18ea223b..cdb840b 100644
--- a/tools/testing/selftests/powerpc/dscr/dscr.h
+++ b/tools/testing/selftests/powerpc/dscr/dscr.h
@@ -39,7 +39,7 @@
 #define rmb()  asm volatile("lwsync":::"memory")
 #define wmb()  asm volatile("lwsync":::"memory")
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
+#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
 
 /* Prilvilege state DSCR access */
 inline unsigned long get_dscr(void)
diff --git a/tools/testing/selftests/powerpc/dscr/dscr_default_test.c 
b/tools/testing/selftests/powerpc/dscr/dscr_default_test.c
index df17c3b..9e1a37e 100644
--- a/tools/testing/selftests/powerpc/dscr/dscr_default_test.c
+++ b/tools/testing/selftests/powerpc/dscr/dscr_default_test.c
@@ -27,7 +27,7 @@ static void *do_test(void *in)
unsigned long d, cur_dscr, cur_dscr_usr;
unsigned long s1, s2;
 
-   s1 = ACCESS_ONCE(sequence);
+   s1 = READ_ONCE(sequence);
if (s1 & 1)
continue;
rmb();

[tip:locking/core] locking/atomics, workqueue: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  c95491ed6d6a958743d82bea1d053819988da418
Gitweb: https://git.kernel.org/tip/c95491ed6d6a958743d82bea1d053819988da418
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:22 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:03 +0200

locking/atomics, workqueue: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the workqueue code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Tejun Heo 
Cc: Lai Jiangshan 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-12-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 kernel/workqueue.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 64d0edf..39831b2 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -4647,7 +4647,7 @@ static void rebind_workers(struct worker_pool *pool)
 * concurrency management.  Note that when or whether
 * @worker clears REBOUND doesn't affect correctness.
 *
-* ACCESS_ONCE() is necessary because @worker->flags may be
+* WRITE_ONCE() is necessary because @worker->flags may be
 * tested without holding any lock in
 * wq_worker_waking_up().  Without it, NOT_RUNNING test may
 * fail incorrectly leading to premature concurrency
@@ -4656,7 +4656,7 @@ static void rebind_workers(struct worker_pool *pool)
WARN_ON_ONCE(!(worker_flags & WORKER_UNBOUND));
worker_flags |= WORKER_REBOUND;
worker_flags &= ~WORKER_UNBOUND;
-   ACCESS_ONCE(worker->flags) = worker_flags;
+   WRITE_ONCE(worker->flags, worker_flags);
}
 
spin_unlock_irq(>lock);

[tip:locking/core] locking/atomics, workqueue: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  c95491ed6d6a958743d82bea1d053819988da418
Gitweb: https://git.kernel.org/tip/c95491ed6d6a958743d82bea1d053819988da418
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:22 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:03 +0200

locking/atomics, workqueue: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the workqueue code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Acked-by: Tejun Heo 
Cc: Lai Jiangshan 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-12-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 kernel/workqueue.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 64d0edf..39831b2 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -4647,7 +4647,7 @@ static void rebind_workers(struct worker_pool *pool)
 * concurrency management.  Note that when or whether
 * @worker clears REBOUND doesn't affect correctness.
 *
-* ACCESS_ONCE() is necessary because @worker->flags may be
+* WRITE_ONCE() is necessary because @worker->flags may be
 * tested without holding any lock in
 * wq_worker_waking_up().  Without it, NOT_RUNNING test may
 * fail incorrectly leading to premature concurrency
@@ -4656,7 +4656,7 @@ static void rebind_workers(struct worker_pool *pool)
WARN_ON_ONCE(!(worker_flags & WORKER_UNBOUND));
worker_flags |= WORKER_REBOUND;
worker_flags &= ~WORKER_UNBOUND;
-   ACCESS_ONCE(worker->flags) = worker_flags;
+   WRITE_ONCE(worker->flags, worker_flags);
}
 
spin_unlock_irq(>lock);

[tip:locking/core] locking/atomics, samples/mic/mpssd/mpssd.c: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  94bbc9c1a0b0967509f8ad91323245aad30572f7
Gitweb: https://git.kernel.org/tip/94bbc9c1a0b0967509f8ad91323245aad30572f7
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:20 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:01 +0200

locking/atomics, samples/mic/mpssd/mpssd.c: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7).

To avoid fragility if/when that transformation occurs, and to align with
the preferred usage of {READ,WRITE}_ONCE(), this patch updates the MPSSD
sample code to use READ_ONCE() rather than ACCESS_ONCE(). There should
be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Shuah Khan 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-10-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 samples/mic/mpssd/mpssd.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/samples/mic/mpssd/mpssd.c b/samples/mic/mpssd/mpssd.c
index 49db1de..f42ce55 100644
--- a/samples/mic/mpssd/mpssd.c
+++ b/samples/mic/mpssd/mpssd.c
@@ -65,7 +65,7 @@ static struct mic_info mic_list;
 /* to align the pointer to the (next) page boundary */
 #define PAGE_ALIGN(addr)_ALIGN(addr, PAGE_SIZE)
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
+#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
 
 #define GSO_ENABLED1
 #define MAX_GSO_SIZE   (64 * 1024)
@@ -382,7 +382,7 @@ disp_iovec(struct mic_info *mic, struct mic_copy_desc *copy,
 
 static inline __u16 read_avail_idx(struct mic_vring *vr)
 {
-   return ACCESS_ONCE(vr->info->avail_idx);
+   return READ_ONCE(vr->info->avail_idx);
 }
 
 static inline void txrx_prepare(int type, bool tx, struct mic_vring *vr,
@@ -523,7 +523,7 @@ spin_for_descriptors(struct mic_info *mic, struct mic_vring 
*vr)
 {
__u16 avail_idx = read_avail_idx(vr);
 
-   while (avail_idx == le16toh(ACCESS_ONCE(vr->vr.avail->idx))) {
+   while (avail_idx == le16toh(READ_ONCE(vr->vr.avail->idx))) {
 #ifdef DEBUG
mpsslog("%s %s waiting for desc avail %d info_avail %d\n",
mic->name, __func__,

[tip:locking/core] locking/atomics, samples/mic/mpssd/mpssd.c: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  94bbc9c1a0b0967509f8ad91323245aad30572f7
Gitweb: https://git.kernel.org/tip/94bbc9c1a0b0967509f8ad91323245aad30572f7
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:20 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:01 +0200

locking/atomics, samples/mic/mpssd/mpssd.c: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7).

To avoid fragility if/when that transformation occurs, and to align with
the preferred usage of {READ,WRITE}_ONCE(), this patch updates the MPSSD
sample code to use READ_ONCE() rather than ACCESS_ONCE(). There should
be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Shuah Khan 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-10-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 samples/mic/mpssd/mpssd.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/samples/mic/mpssd/mpssd.c b/samples/mic/mpssd/mpssd.c
index 49db1de..f42ce55 100644
--- a/samples/mic/mpssd/mpssd.c
+++ b/samples/mic/mpssd/mpssd.c
@@ -65,7 +65,7 @@ static struct mic_info mic_list;
 /* to align the pointer to the (next) page boundary */
 #define PAGE_ALIGN(addr)_ALIGN(addr, PAGE_SIZE)
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
+#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
 
 #define GSO_ENABLED1
 #define MAX_GSO_SIZE   (64 * 1024)
@@ -382,7 +382,7 @@ disp_iovec(struct mic_info *mic, struct mic_copy_desc *copy,
 
 static inline __u16 read_avail_idx(struct mic_vring *vr)
 {
-   return ACCESS_ONCE(vr->info->avail_idx);
+   return READ_ONCE(vr->info->avail_idx);
 }
 
 static inline void txrx_prepare(int type, bool tx, struct mic_vring *vr,
@@ -523,7 +523,7 @@ spin_for_descriptors(struct mic_info *mic, struct mic_vring 
*vr)
 {
__u16 avail_idx = read_avail_idx(vr);
 
-   while (avail_idx == le16toh(ACCESS_ONCE(vr->vr.avail->idx))) {
+   while (avail_idx == le16toh(READ_ONCE(vr->vr.avail->idx))) {
 #ifdef DEBUG
mpsslog("%s %s waiting for desc avail %d info_avail %d\n",
mic->name, __func__,

[tip:locking/core] locking/atomics, rcutorture/formal: Prepare for ACCESS_ONCE() removal

[tip-bot for Mark Rutland]

Commit-ID:  5cd38016d0c88e3f67528c1a7380a6d5c90859e9
Gitweb: https://git.kernel.org/tip/5cd38016d0c88e3f67528c1a7380a6d5c90859e9
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:23 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:04 +0200

locking/atomics, rcutorture/formal: Prepare for ACCESS_ONCE() removal

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7).

To avoid fragility if/when that transformation occurs, this patch
reworks the definitions of {READ,WRITE}_ONCE() in the rcutorture formal
tests, and removes the unused ACCESS_ONCE() helper. There should be no
functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-13-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h 
b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h
index 6687acc..cc27b9e 100644
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h
@@ -34,8 +34,7 @@
 #define rs_smp_mb() do {} while (0)
 #endif
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *) &(x))
-#define READ_ONCE(x) ACCESS_ONCE(x)
-#define WRITE_ONCE(x, val) (ACCESS_ONCE(x) = (val))
+#define READ_ONCE(x) (*(volatile typeof(x) *) &(x))
+#define WRITE_ONCE(x) ((*(volatile typeof(x) *) &(x)) = (val))
 
 #endif

[tip:locking/core] locking/atomics, rcutorture/formal: Prepare for ACCESS_ONCE() removal

[tip-bot for Mark Rutland]

Commit-ID:  5cd38016d0c88e3f67528c1a7380a6d5c90859e9
Gitweb: https://git.kernel.org/tip/5cd38016d0c88e3f67528c1a7380a6d5c90859e9
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:23 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:04 +0200

locking/atomics, rcutorture/formal: Prepare for ACCESS_ONCE() removal

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b71d9f7).

To avoid fragility if/when that transformation occurs, this patch
reworks the definitions of {READ,WRITE}_ONCE() in the rcutorture formal
tests, and removes the unused ACCESS_ONCE() helper. There should be no
functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-13-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h 
b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h
index 6687acc..cc27b9e 100644
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/barriers.h
@@ -34,8 +34,7 @@
 #define rs_smp_mb() do {} while (0)
 #endif
 
-#define ACCESS_ONCE(x) (*(volatile typeof(x) *) &(x))
-#define READ_ONCE(x) ACCESS_ONCE(x)
-#define WRITE_ONCE(x, val) (ACCESS_ONCE(x) = (val))
+#define READ_ONCE(x) (*(volatile typeof(x) *) &(x))
+#define WRITE_ONCE(x) ((*(volatile typeof(x) *) &(x)) = (val))
 
 #endif

[tip:locking/core] locking/atomics, net/average: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  ef4d9af62f47e3070b00c3307a4d8eb5092bb9a2
Gitweb: https://git.kernel.org/tip/ef4d9af62f47e3070b00c3307a4d8eb5092bb9a2
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:19 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:00 +0200

locking/atomics, net/average: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't pick up some uses, including those
in . As a preparatory step, this patch converts the
file to use {READ,WRITE}_ONCE() consistently.

At the same time, this patch addds missing includes necessary for
{READ,WRITE}_ONCE(), *BUG_ON*(), and ilog2().


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Reviewed-by: Johannes Berg 
Cc: David S. Miller 
Cc: Johannes Berg 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-9-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/average.h | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/include/linux/average.h b/include/linux/average.h
index 7ddaf34..3f46229 100644
--- a/include/linux/average.h
+++ b/include/linux/average.h
@@ -1,6 +1,10 @@
 #ifndef _LINUX_AVERAGE_H
 #define _LINUX_AVERAGE_H
 
+#include 
+#include 
+#include 
+
 /*
  * Exponentially weighted moving average (EWMA)
  *
@@ -48,7 +52,7 @@
static inline void ewma_##name##_add(struct ewma_##name *e, \
 unsigned long val) \
{   \
-   unsigned long internal = ACCESS_ONCE(e->internal);  \
+   unsigned long internal = READ_ONCE(e->internal);\
unsigned long weight_rcp = ilog2(_weight_rcp);  \
unsigned long precision = _precision;   \
\
@@ -57,10 +61,10 @@
BUILD_BUG_ON((_precision) > 30);\
BUILD_BUG_ON_NOT_POWER_OF_2(_weight_rcp);   \
\
-   ACCESS_ONCE(e->internal) = internal ?   \
+   WRITE_ONCE(e->internal, internal ?  \
(((internal << weight_rcp) - internal) +\
(val << precision)) >> weight_rcp : \
-   (val << precision); \
+   (val << precision));\
}
 
 #endif /* _LINUX_AVERAGE_H */

[tip:locking/core] locking/atomics, net/average: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  ef4d9af62f47e3070b00c3307a4d8eb5092bb9a2
Gitweb: https://git.kernel.org/tip/ef4d9af62f47e3070b00c3307a4d8eb5092bb9a2
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:19 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:00 +0200

locking/atomics, net/average: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't pick up some uses, including those
in . As a preparatory step, this patch converts the
file to use {READ,WRITE}_ONCE() consistently.

At the same time, this patch addds missing includes necessary for
{READ,WRITE}_ONCE(), *BUG_ON*(), and ilog2().


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Reviewed-by: Johannes Berg 
Cc: David S. Miller 
Cc: Johannes Berg 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-9-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 include/linux/average.h | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/include/linux/average.h b/include/linux/average.h
index 7ddaf34..3f46229 100644
--- a/include/linux/average.h
+++ b/include/linux/average.h
@@ -1,6 +1,10 @@
 #ifndef _LINUX_AVERAGE_H
 #define _LINUX_AVERAGE_H
 
+#include 
+#include 
+#include 
+
 /*
  * Exponentially weighted moving average (EWMA)
  *
@@ -48,7 +52,7 @@
static inline void ewma_##name##_add(struct ewma_##name *e, \
 unsigned long val) \
{   \
-   unsigned long internal = ACCESS_ONCE(e->internal);  \
+   unsigned long internal = READ_ONCE(e->internal);\
unsigned long weight_rcp = ilog2(_weight_rcp);  \
unsigned long precision = _precision;   \
\
@@ -57,10 +61,10 @@
BUILD_BUG_ON((_precision) > 30);\
BUILD_BUG_ON_NOT_POWER_OF_2(_weight_rcp);   \
\
-   ACCESS_ONCE(e->internal) = internal ?   \
+   WRITE_ONCE(e->internal, internal ?  \
(((internal << weight_rcp) - internal) +\
(val << precision)) >> weight_rcp : \
-   (val << precision); \
+   (val << precision));\
}
 
 #endif /* _LINUX_AVERAGE_H */

[tip:locking/core] locking/atomics, net/ipv4/tcp_input.c: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  a9da6f29baf6b61fc94f9768638ee54e9bff173c
Gitweb: https://git.kernel.org/tip/a9da6f29baf6b61fc94f9768638ee54e9bff173c
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:18 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:00 +0200

locking/atomics, net/ipv4/tcp_input.c: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the IPv4 TCP input code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: David S. Miller 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-8-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 net/ipv4/tcp_input.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 7eec338..74480e0 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -815,12 +815,12 @@ static void tcp_update_pacing_rate(struct sock *sk)
if (likely(tp->srtt_us))
do_div(rate, tp->srtt_us);
 
-   /* ACCESS_ONCE() is needed because sch_fq fetches sk_pacing_rate
+   /* WRITE_ONCE() is needed because sch_fq fetches sk_pacing_rate
 * without any lock. We want to make sure compiler wont store
 * intermediate values in this location.
 */
-   ACCESS_ONCE(sk->sk_pacing_rate) = min_t(u64, rate,
-   sk->sk_max_pacing_rate);
+   WRITE_ONCE(sk->sk_pacing_rate, min_t(u64, rate,
+sk->sk_max_pacing_rate));
 }
 
 /* Calculate rto without backoff.  This is the second half of Van Jacobson's

[tip:locking/core] locking/atomics, net/ipv4/tcp_input.c: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  a9da6f29baf6b61fc94f9768638ee54e9bff173c
Gitweb: https://git.kernel.org/tip/a9da6f29baf6b61fc94f9768638ee54e9bff173c
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:18 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:01:00 +0200

locking/atomics, net/ipv4/tcp_input.c: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the IPv4 TCP input code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: David S. Miller 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-8-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 net/ipv4/tcp_input.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 7eec338..74480e0 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -815,12 +815,12 @@ static void tcp_update_pacing_rate(struct sock *sk)
if (likely(tp->srtt_us))
do_div(rate, tp->srtt_us);
 
-   /* ACCESS_ONCE() is needed because sch_fq fetches sk_pacing_rate
+   /* WRITE_ONCE() is needed because sch_fq fetches sk_pacing_rate
 * without any lock. We want to make sure compiler wont store
 * intermediate values in this location.
 */
-   ACCESS_ONCE(sk->sk_pacing_rate) = min_t(u64, rate,
-   sk->sk_max_pacing_rate);
+   WRITE_ONCE(sk->sk_pacing_rate, min_t(u64, rate,
+sk->sk_max_pacing_rate));
 }
 
 /* Calculate rto without backoff.  This is the second half of Van Jacobson's

[tip:locking/core] locking/atomics, fs/ncpfs: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  fd7048adb7a5f3c44154675e0e128fe0f2e16bef
Gitweb: https://git.kernel.org/tip/fd7048adb7a5f3c44154675e0e128fe0f2e16bef
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:15 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:58 +0200

locking/atomics, fs/ncpfs: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

The NCPFS code has some stale comments regarding ACCESS_ONCE() uses
which were removed a long time ago.

Let's remove the stale comments.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Alexander Viro 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Petr Vandrovec 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-5-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 fs/ncpfs/dir.c | 9 -
 1 file changed, 9 deletions(-)

diff --git a/fs/ncpfs/dir.c b/fs/ncpfs/dir.c
index 088f524..72cfaa2 100644
--- a/fs/ncpfs/dir.c
+++ b/fs/ncpfs/dir.c
@@ -119,10 +119,6 @@ static inline int ncp_case_sensitive(const struct inode *i)
 /*
  * Note: leave the hash unchanged if the directory
  * is case-sensitive.
- *
- * Accessing the parent inode can be racy under RCU pathwalking.
- * Use ACCESS_ONCE() to make sure we use _one_ particular inode,
- * the callers will handle races.
  */
 static int 
 ncp_hash_dentry(const struct dentry *dentry, struct qstr *this)
@@ -147,11 +143,6 @@ ncp_hash_dentry(const struct dentry *dentry, struct qstr 
*this)
return 0;
 }
 
-/*
- * Accessing the parent inode can be racy under RCU pathwalking.
- * Use ACCESS_ONCE() to make sure we use _one_ particular inode,
- * the callers will handle races.
- */
 static int
 ncp_compare_dentry(const struct dentry *dentry,
unsigned int len, const char *str, const struct qstr *name)

[tip:locking/core] locking/atomics, fs/ncpfs: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  fd7048adb7a5f3c44154675e0e128fe0f2e16bef
Gitweb: https://git.kernel.org/tip/fd7048adb7a5f3c44154675e0e128fe0f2e16bef
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:15 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:58 +0200

locking/atomics, fs/ncpfs: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

The NCPFS code has some stale comments regarding ACCESS_ONCE() uses
which were removed a long time ago.

Let's remove the stale comments.

Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Alexander Viro 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Petr Vandrovec 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-5-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 fs/ncpfs/dir.c | 9 -
 1 file changed, 9 deletions(-)

diff --git a/fs/ncpfs/dir.c b/fs/ncpfs/dir.c
index 088f524..72cfaa2 100644
--- a/fs/ncpfs/dir.c
+++ b/fs/ncpfs/dir.c
@@ -119,10 +119,6 @@ static inline int ncp_case_sensitive(const struct inode *i)
 /*
  * Note: leave the hash unchanged if the directory
  * is case-sensitive.
- *
- * Accessing the parent inode can be racy under RCU pathwalking.
- * Use ACCESS_ONCE() to make sure we use _one_ particular inode,
- * the callers will handle races.
  */
 static int 
 ncp_hash_dentry(const struct dentry *dentry, struct qstr *this)
@@ -147,11 +143,6 @@ ncp_hash_dentry(const struct dentry *dentry, struct qstr 
*this)
return 0;
 }
 
-/*
- * Accessing the parent inode can be racy under RCU pathwalking.
- * Use ACCESS_ONCE() to make sure we use _one_ particular inode,
- * the callers will handle races.
- */
 static int
 ncp_compare_dentry(const struct dentry *dentry,
unsigned int len, const char *str, const struct qstr *name)

[tip:locking/core] locking/atomics, firmware/ivc: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  eeafcc5a5925d0819dad462eac1d42fda5fbe36f
Gitweb: https://git.kernel.org/tip/eeafcc5a5925d0819dad462eac1d42fda5fbe36f
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:13 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:57 +0200

locking/atomics, firmware/ivc: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

workqueue: kill off ACCESS_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the Tegra IVC code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Jonathan Hunter 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thierry Reding 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-3-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 drivers/firmware/tegra/ivc.c | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/firmware/tegra/ivc.c b/drivers/firmware/tegra/ivc.c
index a01461d..00de793 100644
--- a/drivers/firmware/tegra/ivc.c
+++ b/drivers/firmware/tegra/ivc.c
@@ -99,11 +99,11 @@ static inline bool tegra_ivc_empty(struct tegra_ivc *ivc,
 {
/*
 * This function performs multiple checks on the same values with
-* security implications, so create snapshots with ACCESS_ONCE() to
+* security implications, so create snapshots with READ_ONCE() to
 * ensure that these checks use the same values.
 */
-   u32 tx = ACCESS_ONCE(header->tx.count);
-   u32 rx = ACCESS_ONCE(header->rx.count);
+   u32 tx = READ_ONCE(header->tx.count);
+   u32 rx = READ_ONCE(header->rx.count);
 
/*
 * Perform an over-full check to prevent denial of service attacks
@@ -124,8 +124,8 @@ static inline bool tegra_ivc_empty(struct tegra_ivc *ivc,
 static inline bool tegra_ivc_full(struct tegra_ivc *ivc,
  struct tegra_ivc_header *header)
 {
-   u32 tx = ACCESS_ONCE(header->tx.count);
-   u32 rx = ACCESS_ONCE(header->rx.count);
+   u32 tx = READ_ONCE(header->tx.count);
+   u32 rx = READ_ONCE(header->rx.count);
 
/*
 * Invalid cases where the counters indicate that the queue is over
@@ -137,8 +137,8 @@ static inline bool tegra_ivc_full(struct tegra_ivc *ivc,
 static inline u32 tegra_ivc_available(struct tegra_ivc *ivc,
  struct tegra_ivc_header *header)
 {
-   u32 tx = ACCESS_ONCE(header->tx.count);
-   u32 rx = ACCESS_ONCE(header->rx.count);
+   u32 tx = READ_ONCE(header->tx.count);
+   u32 rx = READ_ONCE(header->rx.count);
 
/*
 * This function isn't expected to be used in scenarios where an
@@ -151,8 +151,8 @@ static inline u32 tegra_ivc_available(struct tegra_ivc *ivc,
 
 static inline void tegra_ivc_advance_tx(struct tegra_ivc *ivc)
 {
-   ACCESS_ONCE(ivc->tx.channel->tx.count) =
-   ACCESS_ONCE(ivc->tx.channel->tx.count) + 1;
+   WRITE_ONCE(ivc->tx.channel->tx.count,
+  READ_ONCE(ivc->tx.channel->tx.count) + 1);
 
if (ivc->tx.position == ivc->num_frames - 1)
ivc->tx.position = 0;
@@ -162,8 +162,8 @@ static inline void tegra_ivc_advance_tx(struct tegra_ivc 
*ivc)
 
 static inline void tegra_ivc_advance_rx(struct tegra_ivc *ivc)
 {
-   ACCESS_ONCE(ivc->rx.channel->rx.count) =
-   ACCESS_ONCE(ivc->rx.channel->rx.count) + 1;
+   WRITE_ONCE(ivc->rx.channel->rx.count,
+  READ_ONCE(ivc->rx.channel->rx.count) + 1);
 
if (ivc->rx.position == ivc->num_frames - 1)
ivc->rx.position = 0;
@@ -428,7 +428,7 @@ int tegra_ivc_notified(struct tegra_ivc *ivc)
 
/* Copy 

[tip:locking/core] locking/atomics, firmware/ivc: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  eeafcc5a5925d0819dad462eac1d42fda5fbe36f
Gitweb: https://git.kernel.org/tip/eeafcc5a5925d0819dad462eac1d42fda5fbe36f
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:13 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:57 +0200

locking/atomics, firmware/ivc: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

workqueue: kill off ACCESS_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the Tegra IVC code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Jonathan Hunter 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thierry Reding 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-3-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 drivers/firmware/tegra/ivc.c | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/firmware/tegra/ivc.c b/drivers/firmware/tegra/ivc.c
index a01461d..00de793 100644
--- a/drivers/firmware/tegra/ivc.c
+++ b/drivers/firmware/tegra/ivc.c
@@ -99,11 +99,11 @@ static inline bool tegra_ivc_empty(struct tegra_ivc *ivc,
 {
/*
 * This function performs multiple checks on the same values with
-* security implications, so create snapshots with ACCESS_ONCE() to
+* security implications, so create snapshots with READ_ONCE() to
 * ensure that these checks use the same values.
 */
-   u32 tx = ACCESS_ONCE(header->tx.count);
-   u32 rx = ACCESS_ONCE(header->rx.count);
+   u32 tx = READ_ONCE(header->tx.count);
+   u32 rx = READ_ONCE(header->rx.count);
 
/*
 * Perform an over-full check to prevent denial of service attacks
@@ -124,8 +124,8 @@ static inline bool tegra_ivc_empty(struct tegra_ivc *ivc,
 static inline bool tegra_ivc_full(struct tegra_ivc *ivc,
  struct tegra_ivc_header *header)
 {
-   u32 tx = ACCESS_ONCE(header->tx.count);
-   u32 rx = ACCESS_ONCE(header->rx.count);
+   u32 tx = READ_ONCE(header->tx.count);
+   u32 rx = READ_ONCE(header->rx.count);
 
/*
 * Invalid cases where the counters indicate that the queue is over
@@ -137,8 +137,8 @@ static inline bool tegra_ivc_full(struct tegra_ivc *ivc,
 static inline u32 tegra_ivc_available(struct tegra_ivc *ivc,
  struct tegra_ivc_header *header)
 {
-   u32 tx = ACCESS_ONCE(header->tx.count);
-   u32 rx = ACCESS_ONCE(header->rx.count);
+   u32 tx = READ_ONCE(header->tx.count);
+   u32 rx = READ_ONCE(header->rx.count);
 
/*
 * This function isn't expected to be used in scenarios where an
@@ -151,8 +151,8 @@ static inline u32 tegra_ivc_available(struct tegra_ivc *ivc,
 
 static inline void tegra_ivc_advance_tx(struct tegra_ivc *ivc)
 {
-   ACCESS_ONCE(ivc->tx.channel->tx.count) =
-   ACCESS_ONCE(ivc->tx.channel->tx.count) + 1;
+   WRITE_ONCE(ivc->tx.channel->tx.count,
+  READ_ONCE(ivc->tx.channel->tx.count) + 1);
 
if (ivc->tx.position == ivc->num_frames - 1)
ivc->tx.position = 0;
@@ -162,8 +162,8 @@ static inline void tegra_ivc_advance_tx(struct tegra_ivc 
*ivc)
 
 static inline void tegra_ivc_advance_rx(struct tegra_ivc *ivc)
 {
-   ACCESS_ONCE(ivc->rx.channel->rx.count) =
-   ACCESS_ONCE(ivc->rx.channel->rx.count) + 1;
+   WRITE_ONCE(ivc->rx.channel->rx.count,
+  READ_ONCE(ivc->rx.channel->rx.count) + 1);
 
if (ivc->rx.position == ivc->num_frames - 1)
ivc->rx.position = 0;
@@ -428,7 +428,7 @@ int tegra_ivc_notified(struct tegra_ivc *ivc)
 
/* Copy the receiver's state out of shared memory. */
tegra_ivc_invalidate(ivc, ivc->rx.phys + offset);
-   state = ACCESS_ONCE(ivc->rx.channel->tx.state);
+   state = READ_ONCE(ivc->rx.channel->tx.state);
 
if 

[tip:locking/core] locking/atomics, media/dvb_ringbuffer: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  24fbd6e04e4e4793ec9dd9c2e007b0b38d2ec3ab
Gitweb: https://git.kernel.org/tip/24fbd6e04e4e4793ec9dd9c2e007b0b38d2ec3ab
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:16 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:58 +0200

locking/atomics, media/dvb_ringbuffer: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the DVB ringbuffer code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Linus Torvalds 
Cc: Mauro Carvalho Chehab 
Cc: Peter Zijlstra 
Cc: Sakari Ailus 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-6-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 drivers/media/dvb-core/dvb_ringbuffer.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/media/dvb-core/dvb_ringbuffer.c 
b/drivers/media/dvb-core/dvb_ringbuffer.c
index 2322af1..5301162 100644
--- a/drivers/media/dvb-core/dvb_ringbuffer.c
+++ b/drivers/media/dvb-core/dvb_ringbuffer.c
@@ -66,12 +66,12 @@ ssize_t dvb_ringbuffer_free(struct dvb_ringbuffer *rbuf)
 {
ssize_t free;
 
-   /* ACCESS_ONCE() to load read pointer on writer side
+   /* READ_ONCE() to load read pointer on writer side
 * this pairs with smp_store_release() in dvb_ringbuffer_read(),
 * dvb_ringbuffer_read_user(), dvb_ringbuffer_flush(),
 * or dvb_ringbuffer_reset()
 */
-   free = ACCESS_ONCE(rbuf->pread) - rbuf->pwrite;
+   free = READ_ONCE(rbuf->pread) - rbuf->pwrite;
if (free <= 0)
free += rbuf->size;
return free-1;
@@ -143,7 +143,7 @@ ssize_t dvb_ringbuffer_read_user(struct dvb_ringbuffer 
*rbuf, u8 __user *buf, si
todo -= split;
/* smp_store_release() for read pointer update to ensure
 * that buf is not overwritten until read is complete,
-* this pairs with ACCESS_ONCE() in dvb_ringbuffer_free()
+* this pairs with READ_ONCE() in dvb_ringbuffer_free()
 */
smp_store_release(>pread, 0);
}
@@ -168,7 +168,7 @@ void dvb_ringbuffer_read(struct dvb_ringbuffer *rbuf, u8 
*buf, size_t len)
todo -= split;
/* smp_store_release() for read pointer update to ensure
 * that buf is not overwritten until read is complete,
-* this pairs with ACCESS_ONCE() in dvb_ringbuffer_free()
+* this pairs with READ_ONCE() in dvb_ringbuffer_free()
 */
smp_store_release(>pread, 0);
}

[tip:locking/core] locking/atomics, media/dvb_ringbuffer: Convert ACCESS_ONCE() to READ_ONCE()/WRITE_ONCE()

[tip-bot for Mark Rutland]

Commit-ID:  24fbd6e04e4e4793ec9dd9c2e007b0b38d2ec3ab
Gitweb: https://git.kernel.org/tip/24fbd6e04e4e4793ec9dd9c2e007b0b38d2ec3ab
Author: Mark Rutland 
AuthorDate: Mon, 23 Oct 2017 14:07:16 -0700
Committer:  Ingo Molnar 
CommitDate: Wed, 25 Oct 2017 11:00:58 +0200

locking/atomics, media/dvb_ringbuffer: Convert ACCESS_ONCE() to 
READ_ONCE()/WRITE_ONCE()

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.

However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.

It's possible to transform the bulk of kernel code using the Coccinelle
script below. However, this doesn't handle comments, leaving references
to ACCESS_ONCE() instances which have been removed. As a preparatory
step, this patch converts the DVB ringbuffer code and comments to use
{READ,WRITE}_ONCE() consistently.


virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)


Signed-off-by: Mark Rutland 
Signed-off-by: Paul E. McKenney 
Cc: Linus Torvalds 
Cc: Mauro Carvalho Chehab 
Cc: Peter Zijlstra 
Cc: Sakari Ailus 
Cc: Thomas Gleixner 
Cc: da...@davemloft.net
Cc: linux-a...@vger.kernel.org
Cc: m...@ellerman.id.au
Cc: sh...@kernel.org
Cc: snit...@redhat.com
Cc: thor.tha...@linux.intel.com
Cc: t...@kernel.org
Cc: v...@zeniv.linux.org.uk
Cc: will.dea...@arm.com
Link: 
http://lkml.kernel.org/r/1508792849-3115-6-git-send-email-paul...@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar 
---
 drivers/media/dvb-core/dvb_ringbuffer.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/media/dvb-core/dvb_ringbuffer.c 
b/drivers/media/dvb-core/dvb_ringbuffer.c
index 2322af1..5301162 100644
--- a/drivers/media/dvb-core/dvb_ringbuffer.c
+++ b/drivers/media/dvb-core/dvb_ringbuffer.c
@@ -66,12 +66,12 @@ ssize_t dvb_ringbuffer_free(struct dvb_ringbuffer *rbuf)
 {
ssize_t free;
 
-   /* ACCESS_ONCE() to load read pointer on writer side
+   /* READ_ONCE() to load read pointer on writer side
 * this pairs with smp_store_release() in dvb_ringbuffer_read(),
 * dvb_ringbuffer_read_user(), dvb_ringbuffer_flush(),
 * or dvb_ringbuffer_reset()
 */
-   free = ACCESS_ONCE(rbuf->pread) - rbuf->pwrite;
+   free = READ_ONCE(rbuf->pread) - rbuf->pwrite;
if (free <= 0)
free += rbuf->size;
return free-1;
@@ -143,7 +143,7 @@ ssize_t dvb_ringbuffer_read_user(struct dvb_ringbuffer 
*rbuf, u8 __user *buf, si
todo -= split;
/* smp_store_release() for read pointer update to ensure
 * that buf is not overwritten until read is complete,
-* this pairs with ACCESS_ONCE() in dvb_ringbuffer_free()
+* this pairs with READ_ONCE() in dvb_ringbuffer_free()
 */
smp_store_release(>pread, 0);
}
@@ -168,7 +168,7 @@ void dvb_ringbuffer_read(struct dvb_ringbuffer *rbuf, u8 
*buf, size_t len)
todo -= split;
/* smp_store_release() for read pointer update to ensure
 * that buf is not overwritten until read is complete,
-* this pairs with ACCESS_ONCE() in dvb_ringbuffer_free()
+* this pairs with READ_ONCE() in dvb_ringbuffer_free()
 */
smp_store_release(>pread, 0);
}