Re: atomic tests timing out on Windows
Here are a couple of articles from the Intel Knowledge Base that don't appear to come with a restrictive license or even a copyright. The code is Windows-specific but it should be possible to translate it to something understandable to gcc and other compilers (Sun C++ on x86). How to Determine Whether a Processor Supports Hyper-Threading Technology: http://www3.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/knowledgebase/43843.htm How to Determine the Number of Logical Processors per Physical Processor: http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/knowledgebase/43842.htm Martin Martin Sebor wrote: Farid Zaripov wrote: [...] If my analysis is correct, we should avoid scheduling the threads on multiple logical CPUs on HT systems with a single physical CPU. Do you agree? I agree. Now we need to know how to determine if the CPU has more than 1 logical CPU. It's pretty easy to figure out on modern Linux kernels from /proc/cpuinfo. I don't know about older kernels or Windows. And I think it is possible to make machine with 2 HT-supporting CPU on board? Here (http://www.intel.com/cd/ids/developer/asmo-na/eng/275339.htm) I found the program with sources, that will help us. Good stuff! We can read it but we can't copy any of the code from the sample program because of the Intel license. Martin
Re: atomic tests timing out on Windows
Martin Sebor wrote: Here are a couple of articles from the Intel Knowledge Base that don't appear to come with a restrictive license or even a copyright. The code is Windows-specific but it should be possible to translate it to something understandable to gcc and other compilers (Sun C++ on x86). How to Determine Whether a Processor Supports Hyper-Threading Technology: http://www3.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/knowledgebase/43843.htm How to Determine the Number of Logical Processors per Physical Processor: http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/knowledgebase/43842.htm If you are looking for BSD-license compatible logic for this, I'm just guessing that the BSD OS sources themselves might be a great resource. Bill
Re: atomic tests timing out on Windows
William A. Rowe, Jr. wrote: Martin Sebor wrote: Here are a couple of articles from the Intel Knowledge Base that don't appear to come with a restrictive license or even a copyright. The code is Windows-specific but it should be possible to translate it to something understandable to gcc and other compilers (Sun C++ on x86). How to Determine Whether a Processor Supports Hyper-Threading Technology: http://www3.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/knowledgebase/43843.htm How to Determine the Number of Logical Processors per Physical Processor: http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/knowledgebase/43842.htm If you are looking for BSD-license compatible logic for this, I'm just guessing that the BSD OS sources themselves might be a great resource. Good suggestion! Thanks! Martin
RE: atomic tests timing out on Windows
-Original Message- From: Martin Sebor [mailto:[EMAIL PROTECTED] On Behalf Of Martin Sebor Sent: Thursday, July 26, 2007 7:50 PM To: stdcxx-dev@incubator.apache.org Subject: Re: atomic tests timing out on Windows As I understand the technology, the benefit of hyperthreading is in the processor's ability to make use of its idle circuits while other circuits are busy doing things. But because a HT processor does not actually duplicate most of ordinary processor's circuits (I've seen 5% being tossed around as the increase in the number of transistors between an ordinary CPU and one with HT) it means that while one thread that does FP processing can execute in parallel with another that does integer arithmetic, two threads that are doing the same thing cannot actually run simultaneously. In our case, since both threads do exactly the same thing, trying to run them on their own virtual processors (emulated by HT) must make them run essentially serially just as they would on an ordinary uniprocessor, and the scheduling overhead involved in the OS and CPU switching between the two threads must actually account for most of time spent by the process. If my analysis is correct, we should avoid scheduling the threads on multiple logical CPUs on HT systems with a single physical CPU. Do you agree? I agree. Now we need to know how to determine if the CPU has more than 1 logical CPU. And I think it is possible to make machine with 2 HT-supporting CPU on board? Here (http://www.intel.com/cd/ids/developer/asmo-na/eng/275339.htm) I found the program with sources, that will help us. Farid.
Re: atomic tests timing out on Windows
Farid Zaripov wrote: -Original Message- From: Martin Sebor [mailto:[EMAIL PROTECTED] On Behalf Of Martin Sebor Sent: Thursday, July 26, 2007 6:17 AM To: stdcxx-dev@incubator.apache.org Subject: Re: atomic tests timing out on Windows That tests works fine, but slow. This is because of using critical section for synchronization for all types except signed/unsigned int and long for which are used InterlockedXXX functions. That doesn't explain why the test runs so much faster in other builds on the same architecture (x86). Even with gcc on CygWin it runs to completion, as well as with MSVC on Windows 2003. All of these complete in under 30 seconds. The test on gcc uses pthreads, which could be implemented without using system critical sections. For example on my computer the one execution of the run_test takes about 25 seconds when used critical section and only 3.5 seconds when used InterlockedXXX functions. The full test takes 445 seconds. And a big strange is that CPU load only ~40% during the test. Is it a 2 CPU or dual core machine? If so, that might explain (some of) it. The CPU must wait for for the other one updates the variable. It's a 1.5 CPU machine :) (Pentium4 with HT). And seems that the timing out problem in HT enabled. I have played with atomic_xchg test: Test1 Test2 Test3 HT disabled:9735 9765 9765 HT enabled, process affinity mask = 3 (default): 202250 - - HT enabled, process affinity mask = 1: 10625 10750 10782 HT enabled, process affinity mask = 2: 10062 10047 10047 Interesting. As I understand the technology, the benefit of hyperthreading is in the processor's ability to make use of its idle circuits while other circuits are busy doing things. But because a HT processor does not actually duplicate most of ordinary processor's circuits (I've seen 5% being tossed around as the increase in the number of transistors between an ordinary CPU and one with HT) it means that while one thread that does FP processing can execute in parallel with another that does integer arithmetic, two threads that are doing the same thing cannot actually run simultaneously. In our case, since both threads do exactly the same thing, trying to run them on their own virtual processors (emulated by HT) must make them run essentially serially just as they would on an ordinary uniprocessor, and the scheduling overhead involved in the OS and CPU switching between the two threads must actually account for most of time spent by the process. If my analysis is correct, we should avoid scheduling the threads on multiple logical CPUs on HT systems with a single physical CPU. Do you agree? Martin The numbers is the time in milliseconds. And if you look into the night tests results: the all platforms where atomic_xxx tests are timed out has Pentium4 with HT processor. Farid.
Re: atomic tests timing out on Windows
Looks like the two atomic tests, atomic_add.exe and atomic_xchg.exe, are timing out in some (optimized) MSVC builds on Windows. First, the timeout seems to be set to 180 seconds on Windows but 300 seconds on all other platforms. I think we should increase it on Windows to make it consistent. How do we go about adjusting the timeout? Second, I'm concerned that the atomic tests are timing out at all because when they run to completion they only take about 15 to 20 seconds (depending on the hardware). Farid, have you seen this in your manual builds? (The test are important because they exercise functionality that string depends on for thread safety). That tests works fine, but slow. This is because of using critical section for synchronization for all types except signed/unsigned int and long for which are used InterlockedXXX functions. For example on my computer the one execution of the run_test takes about 25 seconds when used critical section and only 3.5 seconds when used InterlockedXXX functions. The full test takes 445 seconds. And a big strange is that CPU load only ~40% during the test. Maybe would be useful use own critical section like this: http://www.codeproject.com/useritems/CritSectEx.asp Farid.
Re: atomic tests timing out on Windows
Farid Zaripov wrote: Looks like the two atomic tests, atomic_add.exe and atomic_xchg.exe, are timing out in some (optimized) MSVC builds on Windows. First, the timeout seems to be set to 180 seconds on Windows but 300 seconds on all other platforms. I think we should increase it on Windows to make it consistent. How do we go about adjusting the timeout? Second, I'm concerned that the atomic tests are timing out at all because when they run to completion they only take about 15 to 20 seconds (depending on the hardware). Farid, have you seen this in your manual builds? (The test are important because they exercise functionality that string depends on for thread safety). That tests works fine, but slow. This is because of using critical section for synchronization for all types except signed/unsigned int and long for which are used InterlockedXXX functions. That doesn't explain why the test runs so much faster in other builds on the same architecture (x86). Even with gcc on CygWin it runs to completion, as well as with MSVC on Windows 2003. All of these complete in under 30 seconds. There must be something wrong on this platform (Windows XP). For example on my computer the one execution of the run_test takes about 25 seconds when used critical section and only 3.5 seconds when used InterlockedXXX functions. The full test takes 445 seconds. And a big strange is that CPU load only ~40% during the test. Is it a 2 CPU or dual core machine? If so, that might explain (some of) it. The CPU must wait for for the other one updates the variable. Maybe would be useful use own critical section like this: http://www.codeproject.com/useritems/CritSectEx.asp Maybe, but I don't think that's the answer to the problem we're seeing. Martin
atomic tests timing out on Windows
Looks like the two atomic tests, atomic_add.exe and atomic_xchg.exe, are timing out in some (optimized) MSVC builds on Windows. First, the timeout seems to be set to 180 seconds on Windows but 300 seconds on all other platforms. I think we should increase it on Windows to make it consistent. How do we go about adjusting the timeout? Second, I'm concerned that the atomic tests are timing out at all because when they run to completion they only take about 15 to 20 seconds (depending on the hardware). Farid, have you seen this in your manual builds? (The test are important because they exercise functionality that string depends on for thread safety). Here are a couple of builds with the timed-out tests: http://people.apache.org/~sebor/stdcxx/results/win_xp-2-x86-msvc-32b-8.0-12d-win32-558871-log.gz.txt http://people.apache.org/~sebor/stdcxx/results/win_xp-2-x86-msvc-7.1-12d-win32-558871-log.gz.txt Martin
