[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

 is not compatible with C++: I disabled completly pyatomic.h on 
C++. pyatomic.h is only needed by Python core, not to compile Python 
extensions, so it's not an issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Leonardo Bianconi]

Changes by Leonardo Bianconi :


--
nosy: +lbianc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Roundup Robot]

Roundup Robot added the comment:

New changeset eb48295e1f8b by Victor Stinner in branch 'default':
Issue #23644, #22038: Move #include  inside the extern "C" { ... }
https://hg.python.org/cpython/rev/eb48295e1f8b

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

@haypo, I checked and your idea and implementation are very good, thank you 
very much.

Yes, there is a Clang-specific implementation of the stdatomic.h header [1].

The Musl libc for example created a stdatomic.h header with full compatibility 
[2].

[1] http://llvm.org/viewvc/llvm-project?view=revision&revision=218957

[2] http://www.openwall.com/lists/musl/2014/11/09/2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

At least, the latest change repaired FreeBSD 10 compilation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

I commited atomicv5.patch because it's simple, but I'm not sure that we are 
using stdatomic.h "correctly". The current code looks to be written for GCC, 
Clang fails to compile it (FreeBSD 10 now uses Clang instead of GCC).

Maybe the "_Atomic void*" type is wrong, and we should use the 
"atomic_uintptr_t" type instead and cast to the expected type? Attached 
atomic_pointer.patch implements this idea. It works on FreeBSD 10 with Clang 
and on Fedora 21 with GCC 4.9, both have stdatomic.h.

--
Added file: http://bugs.python.org/file37706/atomic_pointer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Roundup Robot]

Roundup Robot added the comment:

New changeset dacc944641b1 by Victor Stinner in branch 'default':
Issue #22038, configure: HAVE_STD_ATOMIC now also check that "atomic_int" and
https://hg.python.org/cpython/rev/dacc944641b1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

@haypo, @Arfrever, done: atomicv5.patch

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37704/atomicv5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Jeffrey Yasskin]

Changes by Jeffrey Yasskin :


--
nosy:  -jyasskin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[koobs]

koobs added the comment:

Oops, incomplete comment, apologies. Just noticed haypo has reported the issue 
here already

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[koobs]

koobs added the comment:

FreeBSD buildbots broken since fbe87fb071a67cb5e638b3496362b5aedc0fc9a7

--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

OK, I will do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

Can you please generate a patch for the default branch of Python?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Arfrever Frehtes Taifersar Arahesis]

Arfrever Frehtes Taifersar Arahesis added the comment:

Gustavo Temple: A patch against newest revision of default branch would be more 
useful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

@haypo, done: atomicv4.patch

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37694/atomicv4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Removed file: http://bugs.python.org/file37693/atomicv4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37693/atomicv4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

@haypo, OK, I will investigate the problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

I reopen the issue because Python cannot be compiled anymore on the Builder 
AMD64 FreeBSD 10.0 3.x:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.x/builds/2947/steps/configure/logs/stdio

checking for stdatomic.h... yes
checking for GCC >= 4.7 __atomic builtins... yes

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.x/builds/2947/steps/compile/logs/stdio

--- Parser/tokenizer_pgen.o ---
In file included from Parser/tokenizer_pgen.c:2:
In file included from Parser/tokenizer.c:4:
In file included from Include/Python.h:53:
Include/pyatomic.h:37:5: error: _Atomic cannot be applied to incomplete type 
'void' 
_Atomic void *_value;
^

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

Thank you, Victor!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

atomicv3.patch is wrong for GCC builtin atomic operations: the parenthesis is 
not closed. I fixed this typo in the commit.

Vitor & Gustavo: Thanks for the patch, it's now applied to Python 3.5.

I tested it on Fedora 21 (x86_64). I disabled manually HAVE_STD_ATOMIC in 
pyconfig.h to test the two new options (stdatomic header & GCC builtins).

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Roundup Robot]

Roundup Robot added the comment:

New changeset fbe87fb071a6 by Victor Stinner in branch 'default':
Issue #22038: pyatomic.h now uses stdatomic.h or GCC built-in functions for
https://hg.python.org/cpython/rev/fbe87fb071a6

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Removed file: http://bugs.python.org/file37499/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37500/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37499/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Removed file: http://bugs.python.org/file37498/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37498/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Removed file: http://bugs.python.org/file37497/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Gustavo Temple added the comment:

@haypo, done: atomicv3.patch

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Temple]

Changes by Gustavo Temple :


Added file: http://bugs.python.org/file37497/atomicv3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

atomicv2.patch:
> _Atomic int _value;

Why not using the atomic_int type from stdatomic.h here?

> https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html

"__atomic_store_n(): The valid memory model variants are __ATOMIC_RELAXED, 
__ATOMIC_SEQ_CST, and __ATOMIC_RELEASE."

I understand that _Py_atomic_store_explicit() only accept some values for 
order. An assertion should be added here, maybe for any implementation. 
Something like:

#define _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, ORDER) \
(assert((ORDER) == __ATOMIC_RELAXED   \
|| (ORDER) == __ATOMIC_SEQ_CST\
|| (ORDER) == __ATOMIC_RELEASE),  \
 __atomic_store_n(&(ATOMIC_VAL)->_value, NEW_VAL, ORDER))

Same remark for _Py_atomic_load_explicit():

"__atomic_load_n(): The valid memory model variants are __ATOMIC_RELAXED, 
__ATOMIC_SEQ_CST, __ATOMIC_ACQUIRE, and __ATOMIC_CONSUME."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

Fedora 20 provides GCC 4.7 but no stdatomic.c: HAVE_BUILTIN_ATOMIC set 
(HAVE_STD_ATOMIC unset).

Fedora 21 provides GCC 4.9 with stdatomic.c: HAVE_BUILTIN_ATOMIC and 
HAVE_STD_ATOMIC are set.

I tested atomicv2.patch on Fedora 20 (x86_64) and Fedora 21 (x86_64): the whole 
test suite pass.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Gustavo Frederico Temple Pedrosa]

Gustavo Frederico Temple Pedrosa added the comment:

ping

--
nosy: +gustavotemple

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[John Malmberg]

Changes by John Malmberg :


--
nosy: +John.Malmberg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Vitor de Lima]

Vitor de Lima added the comment:

Implemented a new version of the patch using either gcc builtins or the 
stdatomic.h header (this is detected by the configure script).

--
Added file: http://bugs.python.org/file36151/atomicv2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[STINNER Victor]

STINNER Victor added the comment:

+ #define __ATOMIC_RELAXED 0

You should use the "_Py_" prefix for these constants, to avoid conflicts in 
applications.

(You may also replace tabs with spaces, the PEP 7 says "Use 4-space indents and 
no tabs at all." but I also prefer to avoid tabs in other places.)

I tested your patch on Fedora 20 (Linux kernel 3.14.8, GCC 4.8.2, glibc 2.18) 
on x86_64 ("Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz") and the whole Python test 
suite pass.

GCC 4.9 (released a few month ago) provides the  header:
https://gcc.gnu.org/gcc-4.9/changes.html

pyatomic.h contains this comment:

/* XXX: When compilers start offering a stdatomic.h with lock-free
   atomic_int and atomic_address types, include that here and rewrite
   the atomic operations in terms of it. */

But using  header can be done in a separated issue.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +jyasskin, neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22038] Implement atomic operations on non-x86 platforms

[Vitor de Lima]

New submission from Vitor de Lima:

The atomic operations listed in the pyatomic.h header file were implemented 
only for the x86 architecture, this patch uses the atomic bultins available in 
GCC >= 4.7 to implement such operations, allowing it to work properly in other 
platforms.

--
components: Interpreter Core
files: atomic.patch
keywords: patch
messages: 223676
nosy: Vitor.de.Lima
priority: normal
severity: normal
status: open
title: Implement atomic operations on non-x86 platforms
type: enhancement
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file36035/atomic.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com