Since the implementation of P0528R3, "The Curious Case of Padding Bits,
Featuring Atomic Compare-and-Exchange", in r13-2548-g157236dbd62164, the
atomic<T> clears the padding bits of T (if any) in construction and each
store operation. The atomic<T>::compare_exchange_(weak_strong) (and
__atomic_impl::__compare_exchange<false>) implementation assumes above,
to provide shorter (in terms of instructions) implementation.
However, as indicated in the issues, this assumption may not hold when
the store operation is performed in the TU compiled with GCC 12 or eariel.
In consequence program that worked correctly when compiled with GCC 12
(while not required by standard to do so), may trigger infinite loop when
some of TUs are recompiled with GCC 13.
This patch introduces and _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT and
corresponding --enable-atomic-cmpxchg-compat configuration option, that
forces atomic<T> to use implementation that handles non-zero padding
bits (already used for atomic_ref).
libstdc++-v3/ChangeLog:
libstdc++/PR126230
* acinclude.m4 (GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT): Define.
* config.h.in: Regenerate.
* configure: Regenerate.
* configure.ac: Call GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT.
* doc/html/manual/using_macros.html: Regenerate.
* doc/xml/manual/using.xml
(_GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT): Add entry.
* include/bits/atomic_base.h (__atomic_impl::__compare_exchange):
Rename _AtomicRef template parameter to _NonZeroPadding. And
default it to true if _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT is
defined.
* testsuite/29_atomics/atomic/126230.cc: New test.
* testsuite/29_atomics/atomic/126230_compat.cc: Test above
with _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT defined.
---
In the end I haven't followed Jakub's suggestion to mark the
__atomic_compare_exchange path returning true as likely, in this patch,
as I would like to do separate measurments on it (we would impact
the pre-existing uses of atomic_ref).
For the test, I am reinterpret casting the atomic, and wrritting
over it object reprsentation. While not portable, I haven't have a
better idea how to test this.
Tested on x86_64-linux locally. Also checked the
--enable-atomic-cmpxchg-compat=yes.
Testing full build and boostrap powerpc64-linux.
OK for trunk? I assume we want to backport this up to 14?
libstdc++-v3/acinclude.m4 | 19 ++++++++++
libstdc++-v3/config.h.in | 3 ++
libstdc++-v3/configure | 38 ++++++++++++++++---
libstdc++-v3/configure.ac | 3 ++
.../doc/html/manual/using_macros.html | 9 +++++
libstdc++-v3/doc/xml/manual/using.xml | 13 +++++++
libstdc++-v3/include/bits/atomic_base.h | 8 +++-
.../testsuite/29_atomics/atomic/126230.cc | 36 ++++++++++++++++++
.../29_atomics/atomic/126230_compat.cc | 5 +++
9 files changed, 126 insertions(+), 8 deletions(-)
create mode 100644 libstdc++-v3/testsuite/29_atomics/atomic/126230.cc
create mode 100644 libstdc++-v3/testsuite/29_atomics/atomic/126230_compat.cc
diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4
index 8c44e618cb8..5b36939a2f8 100644
--- a/libstdc++-v3/acinclude.m4
+++ b/libstdc++-v3/acinclude.m4
@@ -5888,6 +5888,25 @@ AC_DEFUN([GLIBCXX_CHECK_STDIO_LOCKING], [
AC_LANG_POP([C++])
])
+dnl
+dnl Forces atomic::compare_exchange_weak and atomic::compare_exchange_strong
+dnl to use slower implementation that handles non-zero padding bits inside
+dnl the stored object, that may be produced by TUs compiled by GCC 12 or
eariel.
+dnl Set to yes, for programs linked against binaries compiled with such GCC
+dnl version, that use atomic with types containing padding bits.
+dnl
+dnl --enable-atomic-cmpxchg-compat=yes turns compatibility mode and defines
+dnl _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT.
+dnl Default is no.
+dnl
+AC_DEFUN([GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT], [
+ GLIBCXX_ENABLE(atomic-cmpxchg-compat,no,,
+ [enable compatibility mode for atomic::compare_exchange], [permit yes|no])
+ if test $enable_atomic_cmpxchg_compat = yes; then
+ AC_DEFINE(_GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT, 1,
+ [Define to enable compatibility mode for atomic::compare_exchange])
+ fi
+])
# Macros from the top-level gcc directory.
m4_include([../config/gc++filt.m4])
diff --git a/libstdc++-v3/config.h.in b/libstdc++-v3/config.h.in
index 4cfb9ba26be..a08c42d19e8 100644
--- a/libstdc++-v3/config.h.in
+++ b/libstdc++-v3/config.h.in
@@ -658,6 +658,9 @@
/* Define to use concept checking code from the boost libraries. */
#undef _GLIBCXX_CONCEPT_CHECKS
+/* Define to enable compatibility mode for atomic::compare_exchange */
+#undef _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT
+
/* Define to 1 if a fully dynamic basic_string is wanted, 0 to disable,
undefined for platform defaults */
#undef _GLIBCXX_FULLY_DYNAMIC_STRING
diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure
index 577640fcb75..84adff7bb9e 100755
--- a/libstdc++-v3/configure
+++ b/libstdc++-v3/configure
@@ -971,6 +971,7 @@ enable_libstdcxx_backtrace
enable_libstdcxx_static_eh_pool
with_libstdcxx_eh_pool_obj_count
with_libstdcxx_zoneinfo
+enable_atomic_cmpxchg_compat
enable_cet
with_gxx_include_dir
enable_version_specific_runtime_libs
@@ -1687,6 +1688,9 @@ Optional Features:
--enable-libstdcxx-static-eh-pool
use a fixed-size static buffer for allocating
exceptions if malloc fails
+ --enable-atomic-cmpxchg-compat
+ enable compatibility mode for
+ atomic::compare_exchange [default=no]
--enable-cet enable Intel CET in target libraries [default=auto]
--enable-version-specific-runtime-libs
Specify that runtime libraries should be installed
@@ -12679,7 +12683,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 12682 "configure"
+#line 12686 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -12785,7 +12789,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 12788 "configure"
+#line 12792 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -16475,7 +16479,7 @@ $as_echo "$glibcxx_cv_atomic_word" >&6; }
# Fake what AC_COMPILE_IFELSE does.
cat > conftest.$ac_ext << EOF
-#line 16478 "configure"
+#line 16482 "configure"
#include "atomic_word.h"
int main()
{
@@ -16622,7 +16626,7 @@ $as_echo "mutex" >&6; }
# unnecessary for this test.
cat > conftest.$ac_ext << EOF
-#line 16625 "configure"
+#line 16629 "configure"
int main()
{
_Decimal32 d1;
@@ -16663,7 +16667,7 @@ ac_compiler_gnu=$ac_cv_cxx_compiler_gnu
# unnecessary for this test.
cat > conftest.$ac_ext << EOF
-#line 16666 "configure"
+#line 16670 "configure"
template<typename T1, typename T2>
struct same
{ typedef T2 type; };
@@ -53737,7 +53741,7 @@ $as_echo "$glibcxx_cv_libbacktrace_atomics" >&6; }
CXXFLAGS='-O0 -S'
cat > conftest.$ac_ext << EOF
-#line 53740 "configure"
+#line 53744 "configure"
#include <stddef.h>
int main()
{
@@ -54909,6 +54913,28 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
+# For atomic::compare_exchange
+
+ # Check whether --enable-atomic-cmpxchg-compat was given.
+if test "${enable_atomic_cmpxchg_compat+set}" = set; then :
+ enableval=$enable_atomic_cmpxchg_compat;
+ case "$enableval" in
+ yes|no) ;;
+ *) as_fn_error $? "Unknown argument to enable/disable
atomic-cmpxchg-compat" "$LINENO" 5 ;;
+ esac
+
+else
+ enable_atomic_cmpxchg_compat=no
+fi
+
+
+ if test $enable_atomic_cmpxchg_compat = yes; then
+
+$as_echo "#define _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT 1" >>confdefs.h
+
+ fi
+
+
# Define documentation rules conditionally.
# See if makeinfo has been installed and is modern enough
diff --git a/libstdc++-v3/configure.ac b/libstdc++-v3/configure.ac
index db08de7735e..af8efd22eef 100644
--- a/libstdc++-v3/configure.ac
+++ b/libstdc++-v3/configure.ac
@@ -666,6 +666,9 @@ GLIBCXX_CHECK_DEBUGGING
# For std::print
GLIBCXX_CHECK_STDIO_LOCKING
+# For atomic::compare_exchange
+GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT
+
# Define documentation rules conditionally.
# See if makeinfo has been installed and is modern enough
diff --git a/libstdc++-v3/doc/html/manual/using_macros.html
b/libstdc++-v3/doc/html/manual/using_macros.html
index 6cb467471c8..4cba4015305 100644
--- a/libstdc++-v3/doc/html/manual/using_macros.html
+++ b/libstdc++-v3/doc/html/manual/using_macros.html
@@ -177,4 +177,13 @@
<code class="code">result_type</code> values (instead of <code
class="code">double</code>).
This is required for standard conformance, but is incompatible with
the default implementation of those class templates.
+ </p></dd><dt><span class="term"><code
class="code">_GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT</code></span></dt><dd><p>
+ Undefined by default. Configurable via
+ <code class="code">--enable-atomic-cmpxchg-compat</code>. When
defined,
+ Forces atomic::compare_exchange_weak and
+ atomic::compare_exchange_strong to use slower implementation that
+ handles non-zero padding bits inside the stored object, that may be
+ produced by TUs compiled by GCC 12 or eariel.
+ Use for programs linked against binaries compiled with such GCC
+ version, that use atomic with types containing padding bits.
</p></dd></dl></div></div><div class="navfooter"><hr /><table
width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a
accesskey="p" href="using_headers.html">Prev</a> </td><td width="20%"
align="center"><a accesskey="u" href="using.html">Up</a></td><td width="40%"
align="right"> <a accesskey="n"
href="using_dual_abi.html">Next</a></td></tr><tr><td width="40%" align="left"
valign="top">Headers </td><td width="20%" align="center"><a accesskey="h"
href="../index.html">Home</a></td><td width="40%" align="right" valign="top">
Dual ABI</td></tr></table></div></body></html>
\ No newline at end of file
diff --git a/libstdc++-v3/doc/xml/manual/using.xml
b/libstdc++-v3/doc/xml/manual/using.xml
index da710be408f..5849f75205d 100644
--- a/libstdc++-v3/doc/xml/manual/using.xml
+++ b/libstdc++-v3/doc/xml/manual/using.xml
@@ -1412,6 +1412,19 @@ g++ -Winvalid-pch -I. -include stdc++.h -H -g -O2
hello.cc -o test.exe
the default implementation of those class templates.
</para>
</listitem></varlistentry>
+
<varlistentry><term><code>_GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT</code></term>
+ <listitem>
+ <para>
+ Undefined by default. Configurable via
+ <code>--enable-atomic-cmpxchg-compat</code>. When defined,
+ Forces atomic::compare_exchange_weak and
+ atomic::compare_exchange_strong to use slower implementation that
+ handles non-zero padding bits inside the stored object, that may be
+ produced by TUs compiled by GCC 12 or eariel.
+ Use for programs linked against binaries compiled with such GCC
+ version, that use atomic with types containing padding bits.
+ </para>
+ </listitem></varlistentry>
</variablelist>
</section>
diff --git a/libstdc++-v3/include/bits/atomic_base.h
b/libstdc++-v3/include/bits/atomic_base.h
index 3a8e9a7b4c1..9ac955be633 100644
--- a/libstdc++-v3/include/bits/atomic_base.h
+++ b/libstdc++-v3/include/bits/atomic_base.h
@@ -1055,7 +1055,11 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
return __ptr;
}
- template<bool _AtomicRef = false, typename _Tp>
+#ifdef _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT
+ template<bool _NonZeroPadding = true, typename _Tp>
+#else
+ template<bool _NonZeroPadding = false, typename _Tp>
+#endif
_GLIBCXX_ALWAYS_INLINE bool
__compare_exchange(_Tp& __val, _Val<_Tp>& __e, _Val<_Tp>& __i,
bool __is_weak,
@@ -1072,7 +1076,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
std::__addressof(__i), __is_weak,
int(__s), int(__f));
}
- else if constexpr (!_AtomicRef) // std::atomic<T>
+ else if constexpr (!_NonZeroPadding) // std::atomic<T> compiled with
GCC>=13
{
// Clear padding of the value we want to set:
_Vp* const __pi = __atomic_impl::__clear_padding(__i);
diff --git a/libstdc++-v3/testsuite/29_atomics/atomic/126230.cc
b/libstdc++-v3/testsuite/29_atomics/atomic/126230.cc
new file mode 100644
index 00000000000..b6b0d1e383d
--- /dev/null
+++ b/libstdc++-v3/testsuite/29_atomics/atomic/126230.cc
@@ -0,0 +1,36 @@
+// { dg-do run { target c++11 } }
+// { dg-require-atomic-builtins "" }
+
+#include <atomic>
+#include <cstring>
+#include <testsuite_hooks.h>
+
+// libstdc++/60658
+
+struct alignas(4) Foo {
+ char buf;
+};
+
+int
+main ()
+{
+ std::atomic<Foo> f(Foo{'z'});
+ // Simulate atomic that contains non-zero padding bits,
+ // that can be produced by TU compiled with GCC<=12.
+ std::memset(reinterpret_cast<char*>(&f) + 1, 'a', sizeof(Foo) - 1);
+ VERIFY(f.load().buf == 'z');
+
+ Foo expected{'z'};
+#ifdef _GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT
+ // compare_exchange handles non_zero padding bits, and if
+ // comparision of object representation fails, then clears
+ // padding bits for loaded value
+ VERIFY( f.compare_exchange_strong(expected, {}) );
+#else
+ // compare_exchange assumes that padding bits are cleared
+ // and unconditionally clears them inside expected before
+ // object representation, never finishes if they are empty
+ for (int i = 0; i < 100; ++i)
+ VERIFY( !f.compare_exchange_strong(expected, {}) );
+#endif
+}
diff --git a/libstdc++-v3/testsuite/29_atomics/atomic/126230_compat.cc
b/libstdc++-v3/testsuite/29_atomics/atomic/126230_compat.cc
new file mode 100644
index 00000000000..44a71fc7e9a
--- /dev/null
+++ b/libstdc++-v3/testsuite/29_atomics/atomic/126230_compat.cc
@@ -0,0 +1,5 @@
+// { dg-options "-D_GLIBCXX_FORCE_ATOMIC_CMPXCHG_COMPAT" }
+// { dg-do run { target c++11 } }
+// { dg-require-atomic-builtins "" }
+#include "126230.cc"
+
--
2.55.0
