[issue33174] error building the _sha3 module with Intel 2018 compilers

[Kenneth Hoste]

Kenneth Hoste  added the comment:

After a bit of back and forth, we have figured out the underlying cause for the 
problem with the Intel compiler: if the stack limit is not set to a specific 
value (see output of "ulimit -s"), the Intel compilers fail with a segfault or 
Interal Compiler Error (ICE).

But if the stack limit is set to 'unlimited' (e.g. via "ulimit -s unlimited"), 
then the compilation works fine...

See also https://github.com/easybuilders/easybuild-easyconfigs/issues/6484 and 
our (optional) workaround in 
https://github.com/easybuilders/easybuild-easyblocks/pull/1441

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[Andrew Edmondson]

Andrew Edmondson  added the comment:

See https://software.intel.com/en-us/forums/intel-c-compiler/topic/780574

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[Andrew Edmondson]

Andrew Edmondson  added the comment:

I made the patch. I'm not sure it's suitable to commit back as it is, as it 
unrolls a macro but there seems to be a choice of macro in the code (depending 
on certain conditions).

The problem is that icc can't handle a line that is so long.

I didn't log the bug with Intel - but I agree that it's a compiler bug and not 
a Python bug. I'll log it now...

-Ed

--
nosy: +edmondac

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[Christian Heimes]

Christian Heimes  added the comment:

I read the patch. The manual loop unrolling is ugly but mostly ok. But 
I'm -1 on the "-O0" option and won't accept the patch unless it works with 
standard optimization. sha3 is already slow. Without optimization, the 
algorithm grinds to a halt.

Did anybody contact Intel and reported the bug? It's clearly a bug in ICC. GCC, 
clang and MSVC compile the code just fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[Kenneth Hoste]

Kenneth Hoste  added the comment:

@William: someone in the EasyBuild community seems to have figured out the 
culprit, and came up with a patch to work around the problem, see 
https://github.com/easybuilders/easybuild-easyconfigs/pull/6447/files#diff-bdbfca2206414b3b37794b77c0abb8e8
 .

Any feedback on that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[William Scullin]

William Scullin  added the comment:

Hi Kenneth:

I can recreate this issue on Haswell, Skylake-X, and KNL running SLES 12, Clear 
Linux, and Centos 7.5. I've tried 18.1 and 18.2. It's still present across the 
board with Intel C/C++ 18.0.2.199. On the Centos 7.5.1804 image, the build will 
seemingly complete, but since _sha3 didn't actually compile, the regression 
tests will fail with errors of the form:

$ /dev/shm/testinstall/icc/python/3.6.5/bin/python3 -m test
ERROR:root:code for hash sha3_224 was not found.
Traceback (most recent call last):
  File "/dev/shm/testinstall/icc/python/3.6.5/lib/python3.6/hashlib.py", line 
121, in __get_openssl_constructor
f = getattr(_hashlib, 'openssl_' + name)
AttributeError: module '_hashlib' has no attribute 'openssl_sha3_224'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/dev/shm/testinstall/icc/python/3.6.5/lib/python3.6/hashlib.py", line 
243, in 
globals()[__func_name] = __get_hash(__func_name)
  File "/dev/shm/testinstall/icc/python/3.6.5/lib/python3.6/hashlib.py", line 
128, in __get_openssl_constructor
return __get_builtin_constructor(name)
  File "/dev/shm/testinstall/icc/python/3.6.5/lib/python3.6/hashlib.py", line 
113, in __get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha3_224
ERROR:root:code for hash sha3_256 was not found.
Traceback (most recent call last):
  File "/dev/shm/testinstall/icc/python/3.6.5/lib/python3.6/hashlib.py", line 
121, in __get_openssl_constructor
f = getattr(_hashlib, 'openssl_' + name)
AttributeError: module '_hashlib' has no attribute 'openssl_sha3_256'

...

Searching about, it looks like you may have seen something similar in EasyBuild:

https://github.com/easybuilders/easybuild-easyconfigs/pull/5086

but it doesn't look like the root cause was sussed out.

- William

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[Kenneth Hoste]

Kenneth Hoste  added the comment:

Willian: on which OS and (Intel?) processor generation are you seeing this?

We're not seeing this on CentOS 7.4.1708 with Python 3.6.4 and icc 18.0.1.163 
on Intel Haswell or Intel Skylake.
We have seen this with other software on Intel Skylake though...

Have you tried with icc 18.0.2 (which was released March 21st 2018)?

--
nosy: +boegel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[Christian Heimes]

Change by Christian Heimes :


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33174] error building the _sha3 module with Intel 2018 compilers

[William Scullin]

New submission from William Scullin :

When building Python 3.6.X and later with icc (18.0.0.128 or
18.0.1.163), there's an error building the _sha3 module with any
optimization level other than -O0:

building '_sha3' extension
icc -pthread -fPIC -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3
-Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-parameter
-Wno-missing-field-initializers -fp-model strict -I./Include -I.
-I/usr/local/include
-I/derp/Python-3.6.4/Include
-I/derp/Python-3.6.4 -c
/derp/Python-3.6.4/Modules/_sha3/
sha3module.c -o
build/temp.linux-x86_64-3.6/derp/Python-3.6.4/Modules/_sha3/sha3module.o
": internal error: ** The compiler has encountered an unexpected problem.
** Segmentation violation signal raised. **
Access violation or stack overflow. Please contact Intel Support for assistance.

compilation aborted for
/derp/Python-3.6.4/Modules/_sha3/sha3module.c
(code 4)
...
[ jlselogin2: Python-3.6.4 ]$

if I drop to -O0, compilation works every time. I haven't found
disabling any particular set of optimizations to be useful in
obtaining a successful build.
...
[ jlselogin2: Python-3.6.4 ]$

dropping to -O0, compilation works every time. I haven't found
disabling any particular set of optimizations to be useful in
obtaining a successful build with icc.

 Intel has been notified and a bug filed as this is really a compiler bug. On 
the Python side, it does not appear possible to use Modules/Setup to drop the 
optimization level for just _sha3 and I'm hunting for a workaround.

--
components: Installation
messages: 314619
nosy: wscullin
priority: normal
severity: normal
status: open
title: error building the _sha3 module with Intel 2018 compilers
type: compile error
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com