Re: [Mingw-w64-public] How to detect which threading model is used with the preprocessor?
On Fri, 27 Jun 2014 14:00:06, K. Frank wrote: Well, it's not as easy... The code currently use a portable lib (Qt) for the threads but the question is about splitting a part of the app code for making a lib, intended to be used in a plugin sdk. And asking the customer to depend on external libraries is a sensitive topic... It's not entirely clear to me what your problem is. (In your original question you asked whether you could detect pthreads vs. win32 threads with the preprocessor at compile time.) Why not use g++ / pthreads on linux, and mingw-w64's g++ / winpthreads on windows and just use pthreads for your threading code? I would expect (expect, but don't know for certain) that the Qt's QThreads end up getting mapped to native pthreads on linux and to win32 threads on windows. But unless you try to synchronize your own user pthreads with QThreads, I think you should be able to use both pthreads and QThreads in the same application both on linux and windows. Could you sketch out the precise use case that you're concerned about? I will probably do that (or even use std::thread). The concern I had were about these posts articles stating that mingw-w64 posix threads have a significant lower performance than win32 threads. As I'm writing a monte-carlo integration, which is running on clusters, and for which 5% perf drop is a money loss, I was thinking of writing a win32-thread specific code and use the win32 only mingw-w64 build for that. Hence the preprocessor question. Etienne -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
Re: [Mingw-w64-public] How to detect which threading model is used with the preprocessor?
Hi Etienne! On Mon, Jun 30, 2014 at 4:09 AM, Etienne Sandré-Chardonnal etienne.san...@m4x.org wrote: On Fri, 27 Jun 2014 14:00:06, K. Frank wrote: ... It's not entirely clear to me what your problem is. (In your original question you asked whether you could detect pthreads vs. win32 threads with the preprocessor at compile time.) Why not use g++ / pthreads on linux, and mingw-w64's g++ / winpthreads on windows and just use pthreads for your threading code? I would expect (expect, but don't know for certain) that the Qt's QThreads end up getting mapped to native pthreads on linux and to win32 threads on windows. But unless you try to synchronize your own user pthreads with QThreads, I think you should be able to use both pthreads and QThreads in the same application both on linux and windows. Could you sketch out the precise use case that you're concerned about? I will probably do that (or even use std::thread). The concern I had were about these posts articles stating that mingw-w64 posix threads have a significant lower performance than win32 threads. As I'm writing a monte-carlo integration, which is running on clusters, and for which 5% perf drop is a money loss, I was thinking of writing a win32-thread specific code and use the win32 only mingw-w64 build for that. Hence the preprocessor question. I would recommend that you use std::thread (assuming that you can use c++11). I have used std::thread in a Qt application without any problem. (I should note that it was a small, experimental application, so it was never really stress-tested, and it did not use QThreads.) In terms of performance, I would make the following comments: I would imagine that your Monte Carlo integration parallelizes quite nicely. Once you get your calculation threads spawned and running, there should not be any performance difference between pthreads and win32 threads (or std::threads and QThreads for that matter). You would only expect to see possible performance differences when spawning the threads or synchronizing them (e.g., waiting on a mutex or condition variable). Given that I believe that all the different brands of threads end up being implemented in terms of the native OS threads (pthreads on linux and win32 threads on windows), I would not even expect to see any performance difference when context switching the threads. In your use case I imagine that you spawn a handful of calculation threads (one for each core in the cluster?), let them calculate for a long time, and then synchronize the threads so that you can collect the partial results together. So in my imagination, the calculation dominates, and the cost of spawning and synchronizing the threads doesn't really matter. So the relative performance of spawning and synchronizing the threads also doesn't matter. Second, I played around with implementing std::thread for mingw-w64 in terms of both win32 threads and pthreads (as implemented in mingw-w64's winpthreads library). For some things the win32 threads were faster (I think spawning), and for some things the pthreads were faster (I think, if I remember correctly, mutex contention). So even if the performance of spawning / synchronization does matter to you, I don't think that it goes without saying that win32 threads would be faster for your use case. If you really care, you would want to benchmark. (One last comment: If you do use win32 threads, you should not use win32 mutexes. These are heavy, expensive, interprocess synchronization structures. You should use instead win32 critical sections which are much lighter weight -- intraprocess only -- unless, of course, you need to synchronize processes, not just multiple threads within a single process.) Etienne Good luck. K. Frank -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
Re: [Mingw-w64-public] How to detect which threading model is used with the preprocessor?
Hello Etienne! On Fri, Jun 27, 2014 at 12:26 PM, Etienne Sandré-Chardonnal etienne.san...@m4x.org wrote: On Thu, 26 Jun 2014 22:32:14, Ruben Van Boxem wrote: ... You can use win32 threads!! Just the C++11 thread, mutex and future stuff is build on top of the winpthreads library. Nothing is preventing you from using only Win32 Threads. So you confirm that the stackoverflow post (http://stackoverflow.com/questions/17242516/mingw-w64-threads-posix-vs-win32) is wrong about the sentence: So if you're using the the version of the runtime that was built with posix threads but decide to create threads in your own code with the win32 apis, you're likely to have problems at some point. ? I can't give you a decisive confirmation, but I think that this is unlikely to be true. (Ruben and Kai should be able to offer greater certainty.) It is an implementation detail, so the (posix threads) implementation could be doing something that would break win32 threads, but I would doubt it. I would expect that you should be able to spawn and use pthreads, spawn and use win32 threads (and spawn and use std::thread threads), and everything should work fine. The only place where I might expect trouble would be if you tried to synchronize different brands of threads with one another. (Actually, I believe that all brands of threads are ultimately implemented as win32 threads, so I bet you could synchronize all brands of threads on win32 synchronization structures.) Also, the stackoverflow comment refers to the mingw wiki, even thought the stackoverflow question asks about mingw-w64. (mingw and mingw-w64 are different projects.) I doubt that the stackoverflow comment is true for either mingw or mingw-w64, but it could, in principle, be true for mingw, but not for mingw-w64. ... Well, it's not as easy... The code currently use a portable lib (Qt) for the threads but the question is about splitting a part of the app code for making a lib, intended to be used in a plugin sdk. And asking the customer to depend on external libraries is a sensitive topic... It's not entirely clear to me what your problem is. (In your original question you asked whether you could detect pthreads vs. win32 threads with the preprocessor at compile time.) Why not use g++ / pthreads on linux, and mingw-w64's g++ / winpthreads on windows and just use pthreads for your threading code? I would expect (expect, but don't know for certain) that the Qt's QThreads end up getting mapped to native pthreads on linux and to win32 threads on windows. But unless you try to synchronize your own user pthreads with QThreads, I think you should be able to use both pthreads and QThreads in the same application both on linux and windows. Could you sketch out the precise use case that you're concerned about? Thanks, Etienne Happy Mixed-Thread Multithreaded Hacking! K. Frank -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
Re: [Mingw-w64-public] How to detect which threading model is used with the preprocessor?
2014-06-26 9:12 GMT+02:00 Etienne Sandré-Chardonnal etienne.san...@m4x.org : Hi, Is it possible to detect the threading model (win32 / pthreads) with the preprocessor? At least I would like to make the code compile in these two cases - mingw-w64 with native win32 thread support : use win32 api - mingw-w64 with pthreads or linux gcc : use c++11 Will _GLIBCXX_HAS_GTHREADS work in these cases or is this mingw-only? Thanks! The posix threading model enabled toolchains do not interfere or prevent the use of Win32 threads. In fact, C++ std::thread is a wrapper of a wrapper (GCC's internal gthread) of a wrapper (winpthreads) of Win32 native threading primitives. The best way is actually a configure-time check for std::thread functionality (e.g. test a simple join() call). I know of no preprocessor macro. Ruben Etienne -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
Re: [Mingw-w64-public] How to detect which threading model is used with the preprocessor?
On Thu, 26 Jun 2014 09:12:30 +0200, Etienne Sandré-Chardonnal wrote: Hi, Is it possible to detect the threading model (win32 / pthreads) with the preprocessor? At least I would like to make the code compile in these two cases - mingw-w64 with native win32 thread support : use win32 api - mingw-w64 with pthreads or linux gcc : use c++11 Will _GLIBCXX_HAS_GTHREADS work in these cases or is this mingw-only? Thanks! The macro you reference made me curious. Gthreads (part of glib) written in C, is a wrapper around pthreads or win32 depending on how it was built. This allows for a mix of generic code that uses the wrappers and back-end specific code wrapped in #ifdef for portability. While I am aware that C++ 11 introduced a threading abstraction in- language, I am not aware of the details. Does it not internally wrap OS thread APIs? Are you trying to also mix generic C++11 threading with os specific threading sections, or just use the native os APIs? Pthreads on windows is yet another variation on a theme since it would have to wrap the win32 threading api. Even with that I believe there is still a way to mix in some straight up win32 api code if so needed. I would be interested in hearing the details form those who know. -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
Re: [Mingw-w64-public] How to detect which threading model is used with the preprocessor?
Hi Thomas! On Thu, Jun 26, 2014 at 11:55 AM, C. Thomas Stover c...@thomasstover.com wrote: On Thu, 26 Jun 2014 09:12:30 +0200, Etienne Sandré-Chardonnal wrote: ... Will _GLIBCXX_HAS_GTHREADS work in these cases or is this mingw-only? ... The macro you reference made me curious. Gthreads (part of glib) written in C, is a wrapper around pthreads or win32 depending on how it was built. This allows for a mix of generic code that uses the wrappers and back-end specific code wrapped in #ifdef for portability. While I am aware that C++ 11 introduced a threading abstraction in- language, I am not aware of the details. Does it not internally wrap OS thread APIs? Whether c++11's std::thread wraps the OS threading api internally is an implementation detail. (One would expect that it wraps at least some of the lowest-level native threading and synchronization primitives.) g++'s std::thread, both on linux and windows (i.e., mingw-w64) is implemented in terms of gthreads, which is a light-weight pthreads wrapper (that is very pthreads-centric). On linux, it uses native pthreads; in the case of mingw-w64, it uses winpthreads, the mingw-w64 port of pthreads to windows. Kai can tell you more, but I believe that winpthreads uses native windows threads for its underlying threads, but implements its synchronization structures (mutexes, condition variables, c.) in terms of low-level windows atomics, rather than in terms of the analogous windows synchronization structures. Are you trying to also mix generic C++11 threading with os specific threading sections, or just use the native os APIs? Pthreads on windows is yet another variation on a theme since it would have to wrap the win32 threading api. Again, mingw-w64 implements std::thread in terms of (the mingw-w64 port of) pthreads. This wraps (as I believe) native windows threads, but reimplements most thread synchronization in terms of lower-level windows atomics. That is, it partially wraps the windows threading api, and partially reimplements equivalent functionality. Even with that I believe there is still a way to mix in some straight up win32 api code if so needed. I would be interested in hearing the details form those who know. In general I wouldn't try this (except for fun), as you would be depending on implementation details, but I do believe that you could spawn a (mingw-w64) std::thread or pthread, and then synchronize it on, for example, a windows critical section (a type of windows mutex). This should work because (as I believe), the underlying thread is, in fact, a windows thread. But you would not, for example, be able to use a windows condition variable with a (mingw-w64) std::mutex or pthreads mutex, because (as I believe) winpthreads does not use the windows synchronization structures under the hood. Happy Hacking! K. Frank -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Mingw-w64-public mailing list Mingw-w64-public@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
