[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[ https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719457#comment-16719457 ] Roger Leigh commented on XERCESC-2162: -- I'm unsure why volatile would be a needed qualifier on these specific variables. They shouldn't be optimised out, irrespective of whether volatile is added or not. fHeartBeat is intrinsically racy from what I can see; it's modified by both the main thread and the corresponding worker, and read in the main thread, but it's only used for some cosmetic reporting symbols as far as I can see. Moreover, we always run the thread tests with -quiet, so "if (gRunInfo.quiet == false..." on 1306 should always be false. fHeartBeat should be written once by the worker for each pass and never read. fInProgress is written in the worker and checked in the main thread at termination; should be safe given how it's used with or without volatile. I can see this deadlocking if a worker terminates early; it will never be reset back to false. fParses isn't actually used except for some trivial reporting; shouldn't be optimised out though fThreadNum is set once in the main thread and read by the worker, should be safe Are all four of these structure members definitely optimised out? Can you put a breakpoint in one of the workers and check each member at e.g. thInfo at line 1077 and 1150 for a few passes. I'm suspicious that the volatile qualifiers are hiding some deeper problem. It shouldn't be required for multi-threaded code, and neither GCC, Clang nor MSVC are optimising anything away. The compiler doesn't have sufficient information to elide these members as far as I can see. They are being passed by pointer to the thread main for each worker, and that should surely be a barrier to optimisation; the compiler should not be able to determine that it's not modified at this point, and that should kill any elision. At least, that's my take on it, but I could certainly be wrong. The 21 second timeout is because the test is run with "-time 20" and there's a final 1 second delay while the threads terminate. This is adjustable in the test configuration. > ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163 > --- > > Key: XERCESC-2162 > URL: https://issues.apache.org/jira/browse/XERCESC-2162 > Project: Xerces-C++ > Issue Type: Bug > Components: Samples/Tests >Affects Versions: 3.2.2 > Environment: cat /etc/redhat-release > Red Hat Enterprise Linux Server release 7.6 (Maipo) > uname -a > Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux > Two versions of the Intel compiler suite were tried: > icpc --version > icpc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icpc --version > icpc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. >Reporter: Sam Trahan >Priority: Major > > The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel > compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in > gdb reveals that all ten threads exit, and main() is stuck in a wait loop > calling sleep() forever. > export CXX=icpc > export CFLAGS='-fp-model precise' > export CXXFLAGS='-fp-model precise' > export CC=icc > export CPP="icc -E" > export CXXCPP="icpc -E" > ./configure --prefix=/some/path > make > make check > Changing the CXXFLAGS to this does not help: > export CXXFLAGS='-fp-model precise -std=c++11' > The last bit of output from "make check:" > make[3]: Entering directory `/a-very-long-path/.../tests' > PASS: scripts/DOMTest > PASS: scripts/DOMMemTest > PASS: scripts/RangeTest > PASS: scripts/DOMTraversalTest > XFAIL: scripts/XSerializerTest > PASS: scripts/XSerializerTest1 > PASS: scripts/XSerializerTest2 > PASS: scripts/XSerializerTest3 > PASS: scripts/XSerializerTest4 > PASS: scripts/XSerializerTest5 > PASS: scripts/XSValueTest > XFAIL: scripts/InitTermTest > PASS: scripts/InitTermTest1 > PASS: scripts/InitTermTest2 > PASS: scripts/InitTermTest3 > XFAIL: scripts/ThreadTest > The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is > running: > /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet > -threads 10 -time 20 personal.xml -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org For additional commands, e-mail: c-dev-h...@xerces.apache.org
[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[
https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719074#comment-16719074
]
Sam Trahan commented on XERCESC-2162:
-
I tried two tests:
1. no optimization - works
2. full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
where fHeartBeat is never true because it is optimized away:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
> ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
> ---
>
> Key: XERCESC-2162
> URL: https://issues.apache.org/jira/browse/XERCESC-2162
> Project: Xerces-C++
> Issue Type: Bug
> Components: Samples/Tests
>Affects Versions: 3.2.2
> Environment: cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.6 (Maipo)
> uname -a
> Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64
> x86_64 x86_64 GNU/Linux
> Two versions of the Intel compiler suite were tried:
> icpc --version
> icpc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icpc --version
> icpc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
>Reporter: Sam Trahan
>Priority: Major
>
> The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel
> compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in
> gdb reveals that all ten threads exit, and main() is stuck in a wait loop
> calling sleep() forever.
> export CXX=icpc
> export CFLAGS='-fp-model precise'
> export CXXFLAGS='-fp-model precise'
> export CC=icc
> export CPP="icc -E"
> export CXXCPP="icpc -E"
> ./configure --prefix=/some/path
> make
> make check
> Changing the CXXFLAGS to this does not help:
> export CXXFLAGS='-fp-model precise -std=c++11'
> The last bit of output from "make check:"
> make[3]: Entering directory `/a-very-long-path/.../tests'
> PASS: scripts/DOMTest
> PASS: scripts/DOMMemTest
> PASS: scripts/RangeTest
> PASS: scripts/DOMTraversalTest
> XFAIL: scripts/XSerializerTest
> PASS: scripts/XSerializerTest1
> PASS: scripts/XSerializerTest2
> PASS: scripts/XSerializerTest3
> PASS: scripts/XSerializerTest4
> PASS: scripts/XSerializerTest5
> PASS: scripts/XSValueTest
> XFAIL: scripts/InitTermTest
> PASS: scripts/InitTermTest1
> PASS: scripts/InitTermTest2
> PASS: scripts/InitTermTest3
> XFAIL: scripts/ThreadTest
> The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is
> running:
> /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet
> -threads 10 -time 20 personal.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org
[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[ https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718063#comment-16718063 ] Roger Leigh commented on XERCESC-2162: -- If you compile with the icc equivalent of "-g3", could you post the full stacktrace when it freezes so that we can see exactly where it's stuck? I wouldn't expect any significant difference between "standard" and "posix" because it's all ultimately pthreads underneath. But Xerces is using a somewhat indirect abstraction via the MutexManager, rather than the RAII-style you get with C++11 direct mutex locking. (Were we to require C++11, we could drop the MutexManager entirely and use C++11 threads and mutexes directly without this abstraction,) > ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163 > --- > > Key: XERCESC-2162 > URL: https://issues.apache.org/jira/browse/XERCESC-2162 > Project: Xerces-C++ > Issue Type: Bug > Components: Samples/Tests >Affects Versions: 3.2.2 > Environment: cat /etc/redhat-release > Red Hat Enterprise Linux Server release 7.6 (Maipo) > uname -a > Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux > Two versions of the Intel compiler suite were tried: > icpc --version > icpc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icpc --version > icpc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. >Reporter: Sam Trahan >Priority: Major > > The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel > compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in > gdb reveals that all ten threads exit, and main() is stuck in a wait loop > calling sleep() forever. > export CXX=icpc > export CFLAGS='-fp-model precise' > export CXXFLAGS='-fp-model precise' > export CC=icc > export CPP="icc -E" > export CXXCPP="icpc -E" > ./configure --prefix=/some/path > make > make check > Changing the CXXFLAGS to this does not help: > export CXXFLAGS='-fp-model precise -std=c++11' > The last bit of output from "make check:" > make[3]: Entering directory `/a-very-long-path/.../tests' > PASS: scripts/DOMTest > PASS: scripts/DOMMemTest > PASS: scripts/RangeTest > PASS: scripts/DOMTraversalTest > XFAIL: scripts/XSerializerTest > PASS: scripts/XSerializerTest1 > PASS: scripts/XSerializerTest2 > PASS: scripts/XSerializerTest3 > PASS: scripts/XSerializerTest4 > PASS: scripts/XSerializerTest5 > PASS: scripts/XSValueTest > XFAIL: scripts/InitTermTest > PASS: scripts/InitTermTest1 > PASS: scripts/InitTermTest2 > PASS: scripts/InitTermTest3 > XFAIL: scripts/ThreadTest > The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is > running: > /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet > -threads 10 -time 20 personal.xml -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org For additional commands, e-mail: c-dev-h...@xerces.apache.org
[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[ https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717650#comment-16717650 ] Sam Trahan commented on XERCESC-2162: - I tried both "standard" and "posix." Both resulted in the same hang. I tried using /tmp (tmpfs) and a much shorter path, instead of the Panasas filesystem. No luck there. Disabling threading (nothreads) succeeded. Of course, that leaves me with no threading support. Any other ideas? #!/bin/sh set -xue export CXX=icpc export CFLAGS='-fp-model precise' export CXXFLAGS='-fp-model precise' export CC=icc export CPP="icc -E" export CXXCPP="icpc -E" prefix=/a-very-long-path/... mkdir build cd build cmake -Dmutex-manager=posix \ -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \ -DCMAKE_INSTALL_PREFIX=$prefix .. cmake --build . make make test make install > ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163 > --- > > Key: XERCESC-2162 > URL: https://issues.apache.org/jira/browse/XERCESC-2162 > Project: Xerces-C++ > Issue Type: Bug > Components: Samples/Tests >Affects Versions: 3.2.2 > Environment: cat /etc/redhat-release > Red Hat Enterprise Linux Server release 7.6 (Maipo) > uname -a > Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux > Two versions of the Intel compiler suite were tried: > icpc --version > icpc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icpc --version > icpc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. >Reporter: Sam Trahan >Priority: Major > > The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel > compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in > gdb reveals that all ten threads exit, and main() is stuck in a wait loop > calling sleep() forever. > export CXX=icpc > export CFLAGS='-fp-model precise' > export CXXFLAGS='-fp-model precise' > export CC=icc > export CPP="icc -E" > export CXXCPP="icpc -E" > ./configure --prefix=/some/path > make > make check > Changing the CXXFLAGS to this does not help: > export CXXFLAGS='-fp-model precise -std=c++11' > The last bit of output from "make check:" > make[3]: Entering directory `/a-very-long-path/.../tests' > PASS: scripts/DOMTest > PASS: scripts/DOMMemTest > PASS: scripts/RangeTest > PASS: scripts/DOMTraversalTest > XFAIL: scripts/XSerializerTest > PASS: scripts/XSerializerTest1 > PASS: scripts/XSerializerTest2 > PASS: scripts/XSerializerTest3 > PASS: scripts/XSerializerTest4 > PASS: scripts/XSerializerTest5 > PASS: scripts/XSValueTest > XFAIL: scripts/InitTermTest > PASS: scripts/InitTermTest1 > PASS: scripts/InitTermTest2 > PASS: scripts/InitTermTest3 > XFAIL: scripts/ThreadTest > The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is > running: > /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet > -threads 10 -time 20 personal.xml -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org For additional commands, e-mail: c-dev-h...@xerces.apache.org
[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[ https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717529#comment-16717529 ] Roger Leigh commented on XERCESC-2162: -- Please could you try building with CMake with and without C++11 standard threads, rather than with the autoconf build. Use -Dmutex-manager=standard|posix as documented in the build instructions. This might do a better job of passing the appropriate options when compiling and linking, or doing icc-specific configuration. I haven't tried icc myself though, so I can't guarantee this will work. Thanks, Roger > ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163 > --- > > Key: XERCESC-2162 > URL: https://issues.apache.org/jira/browse/XERCESC-2162 > Project: Xerces-C++ > Issue Type: Bug > Components: Samples/Tests >Affects Versions: 3.2.2 > Environment: cat /etc/redhat-release > Red Hat Enterprise Linux Server release 7.6 (Maipo) > uname -a > Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux > Two versions of the Intel compiler suite were tried: > icpc --version > icpc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icpc --version > icpc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. >Reporter: Sam Trahan >Priority: Major > > The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel > compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in > gdb reveals that all ten threads exit, and main() is stuck in a wait loop > calling sleep() forever. > export CXX=icpc > export CFLAGS='-fp-model precise' > export CXXFLAGS='-fp-model precise' > export CC=icc > export CPP="icc -E" > export CXXCPP="icpc -E" > ./configure --prefix=/some/path > make > make check > Changing the CXXFLAGS to this does not help: > export CXXFLAGS='-fp-model precise -std=c++11' > The last bit of output from "make check:" > make[3]: Entering directory `/a-very-long-path/.../tests' > PASS: scripts/DOMTest > PASS: scripts/DOMMemTest > PASS: scripts/RangeTest > PASS: scripts/DOMTraversalTest > XFAIL: scripts/XSerializerTest > PASS: scripts/XSerializerTest1 > PASS: scripts/XSerializerTest2 > PASS: scripts/XSerializerTest3 > PASS: scripts/XSerializerTest4 > PASS: scripts/XSerializerTest5 > PASS: scripts/XSValueTest > XFAIL: scripts/InitTermTest > PASS: scripts/InitTermTest1 > PASS: scripts/InitTermTest2 > PASS: scripts/InitTermTest3 > XFAIL: scripts/ThreadTest > The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is > running: > /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet > -threads 10 -time 20 personal.xml -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org For additional commands, e-mail: c-dev-h...@xerces.apache.org
