Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jacek Caban]

On 11/15/22 15:17, LIU Hao wrote:
I think that when there is a real bug, we should fix it. As for 
standard themselves, I think that C/C++ standards are important. 
Praising POSIX-alike standards for mingw-w64, who's CRT is a wrapper 
around Windows system CRT, seems a bit misplaced for me. Using it 
makes sense sometimes, but I don't think it should be a goal on its 
own. For full POSIX compatibility, one needs something like Cygwin, 
MSYS or WSL2 anyway.




Do we conclude that we shouldn't introduce the extra complexity, and 
should just maintain the same behavior as MSVC, although it's 
non-standard and causes deadlocks or crashes occasionally? This 
decision is reasonable on itself; and if that is the decision, then 
fair enough. 



As I mentioned, I think that real bugs should be fixed. But we need 
proper understanding of the problem and find root of it. For example you 
said that:



> 1. In a DLL, destructors of static objects and callbacks that are 
registered
> with `atexit()`, are executed by `LdrShutdownProcess()`, after all 
the other
> thread have been terminated `ZwTerminateProcessO(NULL, status)`. This 
means
> that, if another thread has been terminated while holding a mutex, 
the mutex

> can never get unlocked. If a destructor attempts to lock the same mutex,
> deadlocks will occur. Destructors of executables do not suffer from this
> issue, because they are executed before `RtlExitUserProcess()`.


This is simply not true when msvcrt is properly used. atexit callbacks 
are executed by msvcrt before calling ExitProcess(), see Wine 
implementation (callbacks are invoked by _cexit()):


https://gitlab.winehq.org/jacek/wine/-/blob/master/dlls/msvcrt/exit.c#L364

It's possible that mingw-w64 does not use it properly, but in that case 
we should identify the exact problem and simply use it properly.



Jacek



___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[LIU Hao]

在 2022-11-15 04:50, Jacek Caban 写道:
I was short on spare time lately, my apologies. I was planning to learn your code to have more 
insightful comments, but it requires non-trivial amount of time that I couldn't allocate. Anyway, I 
don't think there is any procrastination, your threading model is merged after all.




Well, I was wondering about the mingw-w64 part. (more follow)


I think that when there is a real bug, we should fix it. As for standard themselves, I think that 
C/C++ standards are important. Praising POSIX-alike standards for mingw-w64, who's CRT is a wrapper 
around Windows system CRT, seems a bit misplaced for me. Using it makes sense sometimes, but I don't 
think it should be a goal on its own. For full POSIX compatibility, one needs something like Cygwin, 
MSYS or WSL2 anyway.




Do we conclude that we shouldn't introduce the extra complexity, and should just maintain the same 
behavior as MSVC, although it's non-standard and causes deadlocks or crashes occasionally? This 
decision is reasonable on itself; and if that is the decision, then fair enough.



Unless it's not clear, I appreciate your efforts to fix things. I just don't agree that some 
(admittedly important) implementation details are the best way to fix them.




Thanks for your understanding.


--
Best regards,
LIU Hao



OpenPGP_signature
Description: OpenPGP digital signature
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[sotrdg sotrdg]

I do work on windows 95 with mingw-w64. C++17 filesystem breaks dynamic linking 
of Libstdc++ for me which sucks.
So stop representing other people and just admit C++ standard did a horrible 
job and that is why Linus Torvalds hates it.

https://www.youtube.com/watch?v=C9rHAt1KMq0

If you do believe C++ standard is “so important.” Then why array, 
std::addressof are not freestanding?
https://www.youtube.com/watch?v=DorYSHu4Sjk

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows

From: Jacek Caban<mailto:ja...@codeweavers.com>
Sent: Monday, November 14, 2022 15:52
To: LIU Hao<mailto:lh_mo...@126.com>; 
mingw-w64-public@lists.sourceforge.net<mailto:mingw-w64-public@lists.sourceforge.net>;
 JonY<mailto:jo...@users.sourceforge.net>
Subject: Re: [Mingw-w64-public] Adding a new thread model to GCC

On 11/14/22 09:57, LIU Hao wrote:
> 在 2022/10/25 04:50, Jacek Caban 写道:
>>
>> This also may be supported in existing threading models. Overflow is
>> trivial to fix by waiting in a loop. (There are other reasons why OS
>> support for absolute timeout is slightly better, but the price of
>> this design decision makes it questionable. I plan to elaborate more
>> on that on mingw ML, but I need to find time to do a bit of research
>> first).
>>
>>
>
> What's the status of your 'a bit of research'? This looks like
> procrastination to me.


I was short on spare time lately, my apologies. I was planning to learn
your code to have more insightful comments, but it requires non-trivial
amount of time that I couldn't allocate. Anyway, I don't think there is
any procrastination, your threading model is merged after all.


Meantime, Eric updated his patches on gcc-patches. I wonder what's the
next step for them? Jon?


> Do you (and some others) consider standard conformance not-a-thing; or
> should we maintain such bug-to-bug compatibility, letting users to
> solve such issues themselves?


I think that when there is a real bug, we should fix it. As for standard
themselves, I think that C/C++ standards are important. Praising
POSIX-alike standards for mingw-w64, who's CRT is a wrapper around
Windows system CRT, seems a bit misplaced for me. Using it makes sense
sometimes, but I don't think it should be a goal on its own. For full
POSIX compatibility, one needs something like Cygwin, MSYS or WSL2 anyway.


Unless it's not clear, I appreciate your efforts to fix things. I just
don't agree that some (admittedly important) implementation details are
the best way to fix them.


Thanks,

Jacek



___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public


___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[sotrdg sotrdg]

LOL. “important” when C++ standard did horrible mistakes like C++17 filesystem, 
charconv, C++20 format etc.

They are not. They made bad standard that adds implementation, binary size and 
performance burden to everyone. They do not even consider the implementation 
difficulties in the first place and that leads to the mess you’ve seen today.

Mingw-w64 is technically x86_64-windows-gnu (which is to support GCC with 
microsoft’s own C runtime.) If you want something that is “POSIX-compliant” to 
work on windows, feel free to go cygwin.

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows

From: Jacek Caban<mailto:ja...@codeweavers.com>
Sent: Monday, November 14, 2022 15:52
To: LIU Hao<mailto:lh_mo...@126.com>; 
mingw-w64-public@lists.sourceforge.net<mailto:mingw-w64-public@lists.sourceforge.net>;
 JonY<mailto:jo...@users.sourceforge.net>
Subject: Re: [Mingw-w64-public] Adding a new thread model to GCC

On 11/14/22 09:57, LIU Hao wrote:
> 在 2022/10/25 04:50, Jacek Caban 写道:
>>
>> This also may be supported in existing threading models. Overflow is
>> trivial to fix by waiting in a loop. (There are other reasons why OS
>> support for absolute timeout is slightly better, but the price of
>> this design decision makes it questionable. I plan to elaborate more
>> on that on mingw ML, but I need to find time to do a bit of research
>> first).
>>
>>
>
> What's the status of your 'a bit of research'? This looks like
> procrastination to me.


I was short on spare time lately, my apologies. I was planning to learn
your code to have more insightful comments, but it requires non-trivial
amount of time that I couldn't allocate. Anyway, I don't think there is
any procrastination, your threading model is merged after all.


Meantime, Eric updated his patches on gcc-patches. I wonder what's the
next step for them? Jon?


> Do you (and some others) consider standard conformance not-a-thing; or
> should we maintain such bug-to-bug compatibility, letting users to
> solve such issues themselves?


I think that when there is a real bug, we should fix it. As for standard
themselves, I think that C/C++ standards are important. Praising
POSIX-alike standards for mingw-w64, who's CRT is a wrapper around
Windows system CRT, seems a bit misplaced for me. Using it makes sense
sometimes, but I don't think it should be a goal on its own. For full
POSIX compatibility, one needs something like Cygwin, MSYS or WSL2 anyway.


Unless it's not clear, I appreciate your efforts to fix things. I just
don't agree that some (admittedly important) implementation details are
the best way to fix them.


Thanks,

Jacek



___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public


___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jacek Caban]

On 11/14/22 09:57, LIU Hao wrote:

在 2022/10/25 04:50, Jacek Caban 写道:


This also may be supported in existing threading models. Overflow is 
trivial to fix by waiting in a loop. (There are other reasons why OS 
support for absolute timeout is slightly better, but the price of 
this design decision makes it questionable. I plan to elaborate more 
on that on mingw ML, but I need to find time to do a bit of research 
first).





What's the status of your 'a bit of research'? This looks like 
procrastination to me.



I was short on spare time lately, my apologies. I was planning to learn 
your code to have more insightful comments, but it requires non-trivial 
amount of time that I couldn't allocate. Anyway, I don't think there is 
any procrastination, your threading model is merged after all.



Meantime, Eric updated his patches on gcc-patches. I wonder what's the 
next step for them? Jon?



Do you (and some others) consider standard conformance not-a-thing; or 
should we maintain such bug-to-bug compatibility, letting users to 
solve such issues themselves?



I think that when there is a real bug, we should fix it. As for standard 
themselves, I think that C/C++ standards are important. Praising 
POSIX-alike standards for mingw-w64, who's CRT is a wrapper around 
Windows system CRT, seems a bit misplaced for me. Using it makes sense 
sometimes, but I don't think it should be a goal on its own. For full 
POSIX compatibility, one needs something like Cygwin, MSYS or WSL2 anyway.



Unless it's not clear, I appreciate your efforts to fix things. I just 
don't agree that some (admittedly important) implementation details are 
the best way to fix them.



Thanks,

Jacek



___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[LIU Hao]

在 2022/10/25 04:50, Jacek Caban 写道:


This also may be supported in existing threading models. Overflow is trivial to fix by waiting in a 
loop. (There are other reasons why OS support for absolute timeout is slightly better, but the price 
of this design decision makes it questionable. I plan to elaborate more on that on mingw ML, but I 
need to find time to do a bit of research first).





What's the status of your 'a bit of research'? This looks like procrastination to me. Do you (and 
some others) consider standard conformance not-a-thing; or should we maintain such bug-to-bug 
compatibility, letting users to solve such issues themselves?



--
Best regards,
LIU Hao



OpenPGP_signature
Description: OpenPGP digital signature
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[LIU Hao]

在 2022/10/25 04:50, Jacek Caban 写道:


Those 4 points describes problems that you solve in the new threading model, but there is no reason 
they can't be fixed for existing threading models. In fact, ideally they would be fixed for all 
threading models. Except now we need to worry about one more threading model, meaning that future 
bugs will be even harder to fix.




Since below is no longer related to GCC itself and can be narrowed down to mingw-w64, I am stopping 
CC'ing others.



All these four issues boil down to the fact that TLS callbacks / DLL entrypoints are invoked at a 
wrong moment - after exit callbacks have been invoked, and all the other threads have been killed. 
There are certainly people who just make calls to `_exit()` and don't care about destructors, but 
eventually get deadlocks instead, such LLVM.


That being said, if standard conformance is desired, it is necessary to re-implement `atexit()`, 
`exit()`, `_exit()`, `quick_exit()`, `at_quick_exit()`, etc.



Then there is another fun fact: We already know that, in C++, destructors of static objects are 
executed in the reverse order of completion of their construction. But, for thread-local objects - 
if we have a static object `SA`, whose constructor contructs another static object `SB`, followed by 
a thread-local object `TL`, and returns normally, then, the order of completion initialization is 
`SB` -> `TL` -> `SA`; however, according to the C++ standard ([basic.start.term]/2, ISO/IEC 14882) 
the destruction of thread-local objects shall happen before the destruction of static objects, so 
the order of destruction is actually `TL` -> `SA` -> `SB`.


So, for `__cxa_thread_atexit()` or emutls, it's wrong to register cleanup of thread-local data with 
`atexit()`; it will lead to a wrong order of destruction.


This illustrates why a new thread model is a must-have, and why mcfgthread provides 
`__cxa_atexit()`, although it may seem strange, duplicate, or redundant. Such order of destruction 
is deeply bundled into the `__cxa_finalize()` implementation, and if the Microsoft implementation 
doesn't work the standard way, it is completely impossible to work around that.





This also may be supported in existing threading models. Overflow is trivial to fix by waiting in a 
loop. (There are other reasons why OS support for absolute timeout is slightly better, but the price 
of this design decision makes it questionable. I plan to elaborate more on that on mingw ML, but I 
need to find time to do a bit of research first).




Well, if I use your own words, it would also be to 'make things more complicated than they are'. 
`ZwWaitForSingleObject()` is a public driver API, ripe, well documented [1], and guaranteed. So why 
not use that? Why stick to `WaitForSingleObject()`, which is nothing but a wrapper for the 
aforementioned syscall? Why do we have to maintain something that might continue to work on 9x or 
CE, but in reality, nobody ever cares about?



[1] 
https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-zwwaitforsingleobject


--
Best regards,
LIU Hao


OpenPGP_signature
Description: OpenPGP digital signature
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jacek Caban]

On 10/24/22 05:40, LIU Hao via Gcc-patches wrote:

在 2022/10/21 20:34, i.nix...@autistici.org 写道:


got it...
anyway it seems logical to me the way I proposed :)




Below is a message forwarded from mingw-w64-public, elaborating the 
necessity of a new thread model.


As there are objections from other mingw-w64 developers, I am putting 
those patches against mingw-w64-crt on hold for now. Despite that, all 
threading facilities - mutexes, condition variables, once flags, etc. 
- are still fully functional within the mcf thread model.


In addition, I will keep maintaining my personal builds (from GCC 12 
release branch) with these patches at https://gcc-mcf.lhmouse.com/.



 Forwarded Message 
在 2022/10/23 18:06, Jacek Caban 写道:
>
> Please, let's not do that. It's possible to fix existing 
implementations, we don't need to make

> things more complicated than they are.
>

Okay okay, I think I have to compose a thorough list of problems that 
we are facing at the moment, and had better have a permalink to the 
mailing list archive that I can reference elsewhere. I have been tired 
of repeating the same grounds of arguments again and again:



1. In a DLL, destructors of static objects and callbacks that are 
registered
    with `atexit()`, are executed by `LdrShutdownProcess()`, after all 
the other
    thread have been terminated `ZwTerminateProcessO(NULL, status)`. 
This means
    that, if another thread has been terminated while holding a mutex, 
the mutex
    can never get unlocked. If a destructor attempts to lock the same 
mutex,
    deadlocks will occur. Destructors of executables do not suffer 
from this

    issue, because they are executed before `RtlExitUserProcess()`.

    Standard behavior: Static destructors and exit callbacks should be 
executed
    while other threads are running. If another thread attempts to 
access a
    destroyed object, the behavior is undefined; the user is 
responsible to

    prevent this from happening, by joining or suspending it.


2. Following 1, in a DLL, static destructors and exit callbacks are still
    invoked when `_Exit()` or `quick_exit()` is called.

    Standard behavior: `_Exit()` should not perform any cleanup; not 
even open
    files are flushed. `quick_exit()` shall invoke all quick-exit 
callbacks in

    reverse order, then call `_Exit()`.


3. There is a use-after-free bug [1] about thread-local destructors. I 
suspect

    this is caused by emutls, because GCC uses `__cxa_thread_atexit()` to
    register thread-local destructors, which could interleave with
    `emutls_destroy()`.

    Standard behavior: This is not allowed to happen. mcfgthread 
solves this

    issue by running thread-local destructors and thread-specific key
    destructors as two separate passes [3].

    [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80816
    [2] 
https://github.com/gcc-mirror/gcc/blob/f84e4fb44aa26b71fbc64e0532fd24d96e5caa3f/libgcc/emutls.c#L96
    [3] 
https://github.com/lhmouse/mcfgthread/blob/63e034d375caf585e2921cd3455f1048feb2172d/src/xglobals.c#L249



4. In the win32 thread model, thread-specific key destructors are 
called at

    process exit [4], after static destructors.

    Standard behavior: They shall be called only when a thread exits, 
and the
    associated thread-specific values are not a null pointer. They 
shall not be

    called when a program terminates; instead, users are responsible for
    deallocating such resources before calling `exit()`. This 
requirement is
    missing in POSIX, but formally specified by ISO/IEC 9899:2017, as 
the 4th

    paragraph in '7.26.6.1 The tss_create function'.

    [4] 
https://github.com/mingw-w64/mingw-w64/blob/d0a034a04d312434b842c4869a8a900568d8db98/mingw-w64-crt/crt/tlsthrd.c#L134



Those 4 points describes problems that you solve in the new threading 
model, but there is no reason they can't be fixed for existing threading 
models. In fact, ideally they would be fixed for all threading models. 
Except now we need to worry about one more threading model, meaning that 
future bugs will be even harder to fix.





5. Wait operations, of timed mutexes and condition variables, should take
    absolute time points as `struct timespec`.

    Standard behavior: Both POSIX and ISO C specifies them as such, 
while all
    Windows APIs take relative durations as a 32-bit integer of 
milliseconds,

    which can also easily get overflown.



This also may be supported in existing threading models. Overflow is 
trivial to fix by waiting in a loop. (There are other reasons why OS 
support for absolute timeout is slightly better, but the price of this 
design decision makes it questionable. I plan to elaborate more on that 
on mingw ML, but I need to find time to do a bit of research first).



Jacek



___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

One more question about block-scoped static objects:

>From compiled assembly I have learned that GCC uses a 64-bit integer guard to 
>ensure once-initialization of static objects with block scopes.
Code in gcc/libstdc++-v3/libsupc++/guard.cc uses a global mutex to protect 
multiple threads from racing, which, as described in that file, could 
potentially cause a deadlock that can be avoided if a condition variable is 
used.

This, however, is unnecessary with mcfgthread. The mcfgthread library uses the 
once_flag itself as both the mutex and condition variable, eliminating the need 
of a global mutex and condition variable.
Code using mcfgthread might look like this:
[code]

class foo { ... };

static ::_MCFCRT_OnceFlag flag;
static alignas(foo) char obj[sizeof(foo)];

foo *get_obj(){
const auto construct_obj = []{ ::new(static_cast(obj)) foo(); };
const auto destruct_obj  = []{ reinterpret_cast(obj)->~foo(); };

// Lock the once flag as if it were a mutex.
const auto result = ::_MCFCRT_WaitForOnceFlagForever(); 
// This never times out, so result can be either _MCFCRT_kOnceResultInitial 
or _MCFCRT_kOnceResultFinished.
if(result == ::_MCFCRT_kOnceResultInitial){
try {
// Construct the object in-place.
construct_obj();
if(std::atexit(_obj) != 0){
// Assume we have run out of memory.
destruct_obj();
throw std::bad_alloc();
}
} catch(...){
// Unlock the once flag, allowing other threads to retry the 
initialization.
// This works like pthread_cond_signal().
::_MCFCRT_SignalOnceFlagAsAborted();
throw;
}
// We are done here. Unlock the once flag and other threads will not be 
blocked any more.
// This works like pthread_cond_broadcast().
_MCFCRT_SignalOnceFlagAsFinished();
}
return reinterpret_cast(obj);
}
[/code]

Now here is my question:
That '__cxxabiv1::__guard' thing seems built-in of GCC. Does it have anything 
to do with code generation? And where is it involved?

--   
Best regards,
lh_mouse
2016-04-20


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Thanks to ktietz, Elieux, mingwandroid (via IRC) and jwakely (via mail), 
the integration of mcfgthread and gcc has been accomplished.

This 9000- patch applies to the gcc-5-branch after all other patches from
https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gcc-git are 
applied.
After applying the patches please run 'configure' with --enable-threads=mcf.
The newly compiled i686 gcc has passed 3-stages bootstrap and seems working.
I am going to start the x64 build very soon.

Source code and pre-built binaries of the 'mcfgthread' library can be found 
here:
https://github.com/lhmouse/mcfgthread/tree/master/release
Problem reports and suggestions are welcome. :>

Thanks for your help and interest!


--   
Best regards,
lh_mouse
2016-04-19
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

>>> If the current code assumes a struct and the Windows API calls need an
>>> integer then either the existing code needs to be made more flexible,
>>> or you need to define it as a struct and then convert to an integer
>>> inside your new gthread wrapper functions.
>>
>> The Windows APIs involved use LARGE_INTEGER - a union of a 64-bit integer 
>> and an array of two 32-bit integers - because some languages might have no 
>> native 64-bit integer types.
>> Actually in C99 and C++11 we just assume there is long long and int64_t 
>> despite the fact that ISO C marks int64_t as optional, so it can be regarded 
>> as a union whose only element is an int64_t.
>> 
> What are the units of the argument? Milliseconds?

>From Windows SDK:
  File Times
  A file time is a 64-bit value that represents the number of 100-nanosecond 
intervals that have elapsed since 12:00 A.M. January 1, 1601 Coordinated 
Universal Time (UTC).

This decision makes it much easier to count leap years, no?

> You have two choices, either modify all the places in libstdc++ that
> use __gthread_time_t, changing the code to use some new function
> template that populates a __gthread_time_t from a
> std::chrono::time_point or std::chrono::duration, or define
> a similar struct in your gthreads header and convert it to int64_t
> inside your functions.  For functions taking a relative time the
> conversion from a { seconds, nanoseconds } struct to milliseconds is
> trivial, for functions taking an absolute time you need to know the
> epoch of the clock that was used when populating the struct.

Will do that next week. Still need to make gcc work with my library.

--   
Best regards,
lh_mouse
2016-04-18



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jonathan Wakely]

On 18 April 2016 at 10:57, Jonathan Wakely wrote:
> On 18 April 2016 at 10:18, lh_mouse wrote:
>>> I don't see why it has to be a struct, it just has to be suitable as
>>> an argument to the relevant __gthread functions.
>>
>> The type __gthread_time_t is referenced in 
>> gcc/libstdc++-v3/include/std/mutex:157
>>   __gthread_time_t __ts = {
>> static_cast(__s.time_since_epoch().count()),
>> static_cast(__ns.count())
>>   };
>> This definition uses a braced-init-list that has two elements and is 
>> unsuitable for scalar types.
>
> Yes I know.
>
> What I meant is that there's no fundamental reason it needs to be a
> struct. That code could be changed.

However, that code in  is only needed for timed mutexes, and I
thought you said that wouldn't be supported.

There is code using __gthread_time_t in  that
needs to work even if timed mutexes are not supported.

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Oh I missed the build-in specs in gcc/config/i386/mingw32.h and it was lack of 
-lmcfgthread in it that caused the failure. Stage 1 seemed ok.
Already hacked that. Rebuilding.

Apologize for that.

--   
Best regards,
lh_mouse
2016-04-18

-
发件人：Jonathan Wakely 
发送日期：2016-04-18 16:59
收件人：lh_mouse
抄送：gcc,mingw-w64-public
主题：Re: Re: Adding a new thread model to GCC

On 18 April 2016 at 08:39, lh_mouse wrote:
> I have added a thread model and added its corresponding header files. But it 
> failed the linker.
>
> The file 'gcc/libgcc/config/i386/t-mingw-pthread' which contained two lines:
>   SHLIB_PTHREAD_CFLAG = -pthread
>   SHLIB_PTHREAD_LDFLAG = -Wl,-lpthread
>
> I copied the file to 'gcc/libgcc/config/i386/t-mingw-mcfgthread' and modified 
> the two lines to:
>   SHLIB_PTHREAD_CFLAG = -lmcfgthread
>   SHLIB_PTHREAD_LDFLAG = -Wl,-lmcfgthread
>
> It didn't work and I got a number of undefined references. The command line 
> that invoked the linker didn't include either option.
>
> How to solve this problem?

What are the linker errors? When do they happen?



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jonathan Wakely]

On 18 April 2016 at 08:39, lh_mouse wrote:
> I have added a thread model and added its corresponding header files. But it 
> failed the linker.
>
> The file 'gcc/libgcc/config/i386/t-mingw-pthread' which contained two lines:
>   SHLIB_PTHREAD_CFLAG = -pthread
>   SHLIB_PTHREAD_LDFLAG = -Wl,-lpthread
>
> I copied the file to 'gcc/libgcc/config/i386/t-mingw-mcfgthread' and modified 
> the two lines to:
>   SHLIB_PTHREAD_CFLAG = -lmcfgthread
>   SHLIB_PTHREAD_LDFLAG = -Wl,-lmcfgthread
>
> It didn't work and I got a number of undefined references. The command line 
> that invoked the linker didn't include either option.
>
> How to solve this problem?

What are the linker errors? When do they happen?

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jonathan Wakely]

On 17 April 2016 at 17:56, lh_mouse  wrote:
> A glance over gthr.h reminds me __gthread_time_t. There seem few requirements 
> documented in gthr.h.
> I discussed this with Adrien Nader on mingw-w64's mailing list a few days ago.
>
> Specifically, here are the two questions:
> 0) Should __gthread_time_t be a struct or a plain integral type?
> The 'struct timespec' used by pthread is a struct introduced in POSIX.
> However my implementation uses a plain uint64_t.

I don't see why it has to be a struct, it just has to be suitable as
an argument to the relevant __gthread functions.

If the current code assumes a struct and the Windows API calls need an
integer then either the existing code needs to be made more flexible,
or you need to define it as a struct and then convert to an integer
inside your new gthread wrapper functions.


> 1) How to obtain a __gthread_time_t representing the current time?
> According to Linux man pages, the timeout parameter of 
> pthread_cond_timedwait() is the same as gettimeofday() - that is, it uses the 
> wall clock.
> My implementation uses GetTickCount64() - that is, my implementation uses a 
> monotonic clock.

std::condition_variable::__clock_t must be a typedef for the clock
used by the underlying implementation, so it sounds like you should
use std::chrono::steady_clock for your thread model.


> Quoting from ISO/IEC WG21 Draft N4582 (C++1z):
> [quote]
> 30.4.1.3.1 Class timed_mutex [thread.timedmutex.class]
> ...
> template 
> bool try_lock_for(const chrono::duration& rel_time);
> template 
> bool try_lock_until(const chrono::time_point& abs_time);
> ...
> [/quote]
> the std::timed_mutex::try_lock_for() function template shall accept any clock 
> type, hence we have to do timestamp translation. It is also important to know 
> how libstdc++ handles this.

All conversions are done using the std::chrono facilities, before any
conversion to __gthread_time_t. That means the conversions are
portable.

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

I have added a thread model and added its corresponding header files. But it 
failed the linker.

The file 'gcc/libgcc/config/i386/t-mingw-pthread' which contained two lines:
  SHLIB_PTHREAD_CFLAG = -pthread
  SHLIB_PTHREAD_LDFLAG = -Wl,-lpthread

I copied the file to 'gcc/libgcc/config/i386/t-mingw-mcfgthread' and modified 
the two lines to:
  SHLIB_PTHREAD_CFLAG = -lmcfgthread
  SHLIB_PTHREAD_LDFLAG = -Wl,-lmcfgthread

It didn't work and I got a number of undefined references. The command line 
that invoked the linker didn't include either option.

How to solve this problem?

--   
Best regards,
lh_mouse
2016-04-18
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

A glance over gthr.h reminds me __gthread_time_t. There seem few requirements 
documented in gthr.h.
I discussed this with Adrien Nader on mingw-w64's mailing list a few days ago.

Specifically, here are the two questions:
0) Should __gthread_time_t be a struct or a plain integral type?
The 'struct timespec' used by pthread is a struct introduced in POSIX.
However my implementation uses a plain uint64_t.
1) How to obtain a __gthread_time_t representing the current time?
According to Linux man pages, the timeout parameter of pthread_cond_timedwait() 
is the same as gettimeofday() - that is, it uses the wall clock.
My implementation uses GetTickCount64() - that is, my implementation uses a 
monotonic clock.

Quoting from ISO/IEC WG21 Draft N4582 (C++1z):
[quote]
30.4.1.3.1 Class timed_mutex [thread.timedmutex.class]
...
template 
bool try_lock_for(const chrono::duration& rel_time);
template 
bool try_lock_until(const chrono::time_point& abs_time);
...
[/quote]
the std::timed_mutex::try_lock_for() function template shall accept any clock 
type, hence we have to do timestamp translation. It is also important to know 
how libstdc++ handles this.

--   
Best regards,
lh_mouse
2016-04-18


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

I appreciate your interest in mcfgthread! :>

I am still struggling to compiling gcc and have no time to make a patch.

But if you have successfully compiled gcc you might want to make one yourself:

Currently mcfgthread can only be linked as a dynamic library, so (I imagine) 
the following steps would do the trick:

0) Clone the repository:
   $ git clone https://github.com/lhmouse/mcfgthread.git;
1) Generate configure script, configure, make, make install:
   $ autoconf -i && ./configure --enable-shared --prefix='' && 
make -j4 && make install
2) In step 1) the headers should be in /include and libmcfgthread.dll.a 
and *-mcfgthread-*.dll should be in /lib.
   Since the newly built gcc is going to need these files, make sure they are 
placed in the correct location such that they can be found by gcc later.
3) Modify gcc's configure.ac to create a thread model or just replace the 
existent win32 one, as Jonathan Wakely described in a mail two days ago.
   We should hardcode an '-lmcfgthread' in both LDFLAGS and the builtin specs. 
I have no idea how to do that by now.
4) Run the 3-step bootstrap of GCC and test whether it is working.

--   
Best regards,
lh_mouse
2016-04-15

-
发件人：Yuta Nakai <nak5...@live.jp>
发送日期：2016-04-15 22:09
收件人：mingw-w64-public@lists.sourceforge.net
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

What I mean is that i want the patch "to use your new thread model".
How can I build gcc with mcfgthread?


From: lh_mouse <lh_mo...@126.com>
Sent: 15 April 2016 23:01
To: mingw-w64-public@lists.sourceforge.net
Subject: Re: [Mingw-w64-public] Adding a new thread model to GCC

According to Alexpux's reply patches could be found on GitHub:

$ git clone https://github.com/Alexpux/MINGW-packages.git

--
Best regards,
lh_mouse
2016-04-15

-
发件人：Nakai Yuta <nak5...@live.jp>
发送日期：2016-04-15 19:59
收件人：mingw-w64-public@lists.sourceforge.net
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

Hi lh_mouse,

Where can I get a patch(es) for gcc in order to use your new thread model?


From: lh_mouse <lh_mo...@126.com>
Sent: Wednesday, April 13, 2016 18:17
To: gcc; mingw-w64-public
Subject: [Mingw-w64-public] Adding a new thread model to GCC

Hi all,

The 'win32' thread model of gcc has been there since long long ago, being 
compatible with very old Windows versions, also having a number of drawbacks:
  0) its implementation is very inefficient, and
  1) its mutexes and condition variables require dynamic initialization and are 
unusable in C++11 as std::mutex requires a `constexpr` constructor, and
  2) allocating a number of rarely used mutexes or condition variables would 
eventually make the system run out of kernel objects.

As a solution for 1) and 2), Microsoft introduced keyed events, details of 
which can be found here:
http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/

In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
since Vista and are working well without any dynamic initialization, just like 
FUTEX.
But there are still some major drawbacks:
  0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
equivalent function to pthread_mutex_timedwait, and
  1) the CONDITION_VARIABLE can only be used with SRWLOCK and CRITICAL_SECTION 
and no user-defined mutexes can be supportd.

As a solution, I come up with my own library: 
https://github.com/lhmouse/mcfgthread
With 4 threads contending my mutex implementation turns to be 10 times faster 
than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
I have also done some research on gthr.h and successfully created a wrapper for 
__gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

I am currently looking for ways to integrate this library into gcc as a new 
thread model, let's say 'nt6' for example, so GCC can be configured with 
--enable-threads=nt6.
But there is so little documentation about that. If someone know how to add a 
new thread model in GCC, I would appreciate that.

Also, this library has not been fully tested. If someone has interest in 
exploring how it works or testing it, I might be able to help.

--
Best regards,
lh_mouse
2016-04-13


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Yuta Nakai]

What I mean is that i want the patch "to use your new thread model".
How can I build gcc with mcfgthread?


From: lh_mouse <lh_mo...@126.com>
Sent: 15 April 2016 23:01
To: mingw-w64-public@lists.sourceforge.net
Subject: Re: [Mingw-w64-public] Adding a new thread model to GCC

According to Alexpux's reply patches could be found on GitHub:

$ git clone https://github.com/Alexpux/MINGW-packages.git

--
Best regards,
lh_mouse
2016-04-15

-
发件人：Nakai Yuta <nak5...@live.jp>
发送日期：2016-04-15 19:59
收件人：mingw-w64-public@lists.sourceforge.net
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

Hi lh_mouse,

Where can I get a patch(es) for gcc in order to use your new thread model?


From: lh_mouse <lh_mo...@126.com>
Sent: Wednesday, April 13, 2016 18:17
To: gcc; mingw-w64-public
Subject: [Mingw-w64-public] Adding a new thread model to GCC

Hi all,

The 'win32' thread model of gcc has been there since long long ago, being 
compatible with very old Windows versions, also having a number of drawbacks:
  0) its implementation is very inefficient, and
  1) its mutexes and condition variables require dynamic initialization and are 
unusable in C++11 as std::mutex requires a `constexpr` constructor, and
  2) allocating a number of rarely used mutexes or condition variables would 
eventually make the system run out of kernel objects.

As a solution for 1) and 2), Microsoft introduced keyed events, details of 
which can be found here:
http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/

In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
since Vista and are working well without any dynamic initialization, just like 
FUTEX.
But there are still some major drawbacks:
  0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
equivalent function to pthread_mutex_timedwait, and
  1) the CONDITION_VARIABLE can only be used with SRWLOCK and CRITICAL_SECTION 
and no user-defined mutexes can be supportd.

As a solution, I come up with my own library: 
https://github.com/lhmouse/mcfgthread
With 4 threads contending my mutex implementation turns to be 10 times faster 
than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
I have also done some research on gthr.h and successfully created a wrapper for 
__gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

I am currently looking for ways to integrate this library into gcc as a new 
thread model, let's say 'nt6' for example, so GCC can be configured with 
--enable-threads=nt6.
But there is so little documentation about that. If someone know how to add a 
new thread model in GCC, I would appreciate that.

Also, this library has not been fully tested. If someone has interest in 
exploring how it works or testing it, I might be able to help.

--
Best regards,
lh_mouse
2016-04-13


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

According to Alexpux's reply patches could be found on GitHub:

$ git clone https://github.com/Alexpux/MINGW-packages.git

--   
Best regards,
lh_mouse
2016-04-15

-
发件人：Nakai Yuta <nak5...@live.jp>
发送日期：2016-04-15 19:59
收件人：mingw-w64-public@lists.sourceforge.net
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

Hi lh_mouse,

Where can I get a patch(es) for gcc in order to use your new thread model?


From: lh_mouse <lh_mo...@126.com>
Sent: Wednesday, April 13, 2016 18:17
To: gcc; mingw-w64-public
Subject: [Mingw-w64-public] Adding a new thread model to GCC

Hi all,

The 'win32' thread model of gcc has been there since long long ago, being 
compatible with very old Windows versions, also having a number of drawbacks:
  0) its implementation is very inefficient, and
  1) its mutexes and condition variables require dynamic initialization and are 
unusable in C++11 as std::mutex requires a `constexpr` constructor, and
  2) allocating a number of rarely used mutexes or condition variables would 
eventually make the system run out of kernel objects.

As a solution for 1) and 2), Microsoft introduced keyed events, details of 
which can be found here:
http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/

In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
since Vista and are working well without any dynamic initialization, just like 
FUTEX.
But there are still some major drawbacks:
  0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
equivalent function to pthread_mutex_timedwait, and
  1) the CONDITION_VARIABLE can only be used with SRWLOCK and CRITICAL_SECTION 
and no user-defined mutexes can be supportd.

As a solution, I come up with my own library: 
https://github.com/lhmouse/mcfgthread
With 4 threads contending my mutex implementation turns to be 10 times faster 
than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
I have also done some research on gthr.h and successfully created a wrapper for 
__gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

I am currently looking for ways to integrate this library into gcc as a new 
thread model, let's say 'nt6' for example, so GCC can be configured with 
--enable-threads=nt6.
But there is so little documentation about that. If someone know how to add a 
new thread model in GCC, I would appreciate that.

Also, this library has not been fully tested. If someone has interest in 
exploring how it works or testing it, I might be able to help.

--
Best regards,
lh_mouse
2016-04-13


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Nakai Yuta]

Hi lh_mouse,

Where can I get a patch(es) for gcc in order to use your new thread model?


From: lh_mouse <lh_mo...@126.com>
Sent: Wednesday, April 13, 2016 18:17
To: gcc; mingw-w64-public
Subject: [Mingw-w64-public] Adding a new thread model to GCC

Hi all,

The 'win32' thread model of gcc has been there since long long ago, being 
compatible with very old Windows versions, also having a number of drawbacks:
  0) its implementation is very inefficient, and
  1) its mutexes and condition variables require dynamic initialization and are 
unusable in C++11 as std::mutex requires a `constexpr` constructor, and
  2) allocating a number of rarely used mutexes or condition variables would 
eventually make the system run out of kernel objects.

As a solution for 1) and 2), Microsoft introduced keyed events, details of 
which can be found here:
http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/

In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
since Vista and are working well without any dynamic initialization, just like 
FUTEX.
But there are still some major drawbacks:
  0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
equivalent function to pthread_mutex_timedwait, and
  1) the CONDITION_VARIABLE can only be used with SRWLOCK and CRITICAL_SECTION 
and no user-defined mutexes can be supportd.

As a solution, I come up with my own library: 
https://github.com/lhmouse/mcfgthread
With 4 threads contending my mutex implementation turns to be 10 times faster 
than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
I have also done some research on gthr.h and successfully created a wrapper for 
__gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

I am currently looking for ways to integrate this library into gcc as a new 
thread model, let's say 'nt6' for example, so GCC can be configured with 
--enable-threads=nt6.
But there is so little documentation about that. If someone know how to add a 
new thread model in GCC, I would appreciate that.

Also, this library has not been fully tested. If someone has interest in 
exploring how it works or testing it, I might be able to help.

--
Best regards,
lh_mouse
2016-04-13


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Talk specifically about C++ std::timed_mutex:

From ISO/IEC WG21 Draft N4582 (C++1z)
[quote]
30.4.1.3.1 Class timed_mutex [thread.timedmutex.class]
...
template 
bool try_lock_for(const chrono::duration<Rep, Period>& rel_time);
template 
bool try_lock_until(const chrono::time_point<Clock, Duration>& abs_time);
...
[/quote]

Merely, I haven't used timed_mutex.
According to this specification, std::timed_mutex shall work with not only a 
specific, but actually ANY clock type.
That is to say, we are always going to perform timestamp translation no matter 
what clock type we use. And that is also the reason why I have decided to use a 
'sane', monotic clock type.
The fact that gthread isn't pthread thus isn't subject to convention of 'struct 
timespec' also gives us the right to do so.

It is still how to obtain a __gthread_time_t that represents the current system 
time that the problem is.

--   
Best regards,
lh_mouse
2016-04-15

-
发件人：Adrien Nader <adr...@notk.org>
发送日期：2016-04-14 19:42
收件人：mingw-w64-public
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

On Thu, Apr 14, 2016, lh_mouse wrote:
> C++0x threading APIs are currently unimplemented.
> 
> It is only because gthr.h is so obscure about __gthred_time_t.
> It doesn't say anything about whether __gthread_time_t should be a struct 
> (like struct timespec) or a plain integer or even a floating point type, 
> neither does it say where and how to obtain a __gthread_time_t that 
> represents the current system time.
> Nonetheless, all *timedwait() interfaces, for mutex, condition variable and 
> (as extensions) even thread and once-init, are naturally supported by 
> mcfgthread.
> 
> I really hate the 'struct timespec' thing, as well as using a UNIX timestamp 
> in *_timedwait() functions because the wall clock is subject to DST and NTP 
> and can also be manually set by the user.
> Rather, (currently) all timepoints in mcfgthread are obtained using the 
> _MCFCRT_GetFastMonoClock() function which is simply a wrapper for 
> GetTickCount64() that is only available since Vista.
> This guarantees that the clock is monotonic and can't be set by the user, as 
> well as a good resolution of 1ms.
> 
> However, should readjusting the time origin be essential, it would be easy to 
> use an alternative clock, code about which could be found here:
> https://github.com/lhmouse/mcfgthread/blob/master/src/env/_nt_timeout.h?ts=4#L15

The use of unix timestamps and the clock choice are completely
orthogonal issues.
CLOCK_MONOTONIC can be happily used with unix timestamps. If it
couldn't, then I wonder why the REALTIME, REALTIME_COARSE, MONOTONIC,
MONOTONIC_COARSE, RAW, BOOTTIME, PROCESS_CPUTIME_ID, THREAD_CPUTIME_ID
clocks would have been added to glibc (well, except thread cputime :) ).

If you want to use a specific clock with says pthread_cond_timedwait,
you should set the corresponding attribute on the condition variable
with pthread_condattr_setclock().

I'm not sure you (or someone else) would start implementing C++11
threading on top of that but as far as I know, C++, at least C++11,
gives absolutely no guarantee on the clock being used. It's not even
that you can't rely on a monotonic one being available, it's that you
can't rely on the dumbest one being available and you cannot chose nor
check anything about clocks.
What you are doing here is using your own preference. I agree it makes
much more sense but you shouldn't chose a behaviour based on that,
especially when the most-widespread behaviour does the opposite.

-- 
Adrien Nader



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Torvald Riegel]

On Thu, 2016-04-14 at 10:24 +0800, lh_mouse wrote:
> Yes I learnt those *ByAddress* functions a few months ago and was shocked 
> that Microsoft had plagiarized futex from Linux.
> As you have pointed out already, those APIs are only available on  Windows 8 
> and later.

Yes.  On Windows 8, given that we have futexes, I'd just use those.  An
exception might be made if Windows offers native support for certain
datastructures such as rwlock or condvars.

The question that remains is what we then do for Vista and Win 7.  What
you initially proposed is one option.

> If we need fuex semantics on Windows 7 and earlier, we must simulate it.
> However, since futexes and keyed events work differently, simulation of 
> futexes using keyed events would:
>   0) require a lot of work, and

Are you sure about this?  Unlike POSIX, we have no process-shared
concurrent data structures, so all you'd need for a futex implementation
would be to build wait queues.  You need that anyway unless you don't
want to support std::synchronic (which is likely to become part of a TS
in the near future).

>   1) suffer from performance penalty just like if we simulate keyed events on 
> Windows 2000.

Why would that be the case? Especially for WakeByAddressSingle, there's
not much you need to do compare to what you'd do for a mutex.  Blocking
(ie, where you'd use a futex) is on the slow path anyway.

> The major difference is that, the FUTEX_WAIT syscall is an atomic 
> compare-and-sleep operation, while the NtWaitForKeyedEvent syscall provides 
> no comparison.
> Instead, the NtReleasedKeyedEvent syscall blocks the current thread until one 
> thread is woken up, while FUTEX_WAKE does nothing and returns immediately.

Yeah, the emulation would need an explicit handshake.

> So here is my condition: in no event shall portability harm either efficiency 
> or maintainability.

Maintainability is what the motivation behind what I proposed.

> Linux and Windows work differently. I consider it 'harmful' to make one look 
> like the other.

Hey, at the latest once we end up in C++, they do look the same!  So all
we're discussing here is at which level we're unifying.  I'm not asking
you to emulate POSIX or syscalls.


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Adrien Nader]

On Thu, Apr 14, 2016, lh_mouse wrote:
> C++0x threading APIs are currently unimplemented.
> 
> It is only because gthr.h is so obscure about __gthred_time_t.
> It doesn't say anything about whether __gthread_time_t should be a struct 
> (like struct timespec) or a plain integer or even a floating point type, 
> neither does it say where and how to obtain a __gthread_time_t that 
> represents the current system time.
> Nonetheless, all *timedwait() interfaces, for mutex, condition variable and 
> (as extensions) even thread and once-init, are naturally supported by 
> mcfgthread.
> 
> I really hate the 'struct timespec' thing, as well as using a UNIX timestamp 
> in *_timedwait() functions because the wall clock is subject to DST and NTP 
> and can also be manually set by the user.
> Rather, (currently) all timepoints in mcfgthread are obtained using the 
> _MCFCRT_GetFastMonoClock() function which is simply a wrapper for 
> GetTickCount64() that is only available since Vista.
> This guarantees that the clock is monotonic and can't be set by the user, as 
> well as a good resolution of 1ms.
> 
> However, should readjusting the time origin be essential, it would be easy to 
> use an alternative clock, code about which could be found here:
> https://github.com/lhmouse/mcfgthread/blob/master/src/env/_nt_timeout.h?ts=4#L15

The use of unix timestamps and the clock choice are completely
orthogonal issues.
CLOCK_MONOTONIC can be happily used with unix timestamps. If it
couldn't, then I wonder why the REALTIME, REALTIME_COARSE, MONOTONIC,
MONOTONIC_COARSE, RAW, BOOTTIME, PROCESS_CPUTIME_ID, THREAD_CPUTIME_ID
clocks would have been added to glibc (well, except thread cputime :) ).

If you want to use a specific clock with says pthread_cond_timedwait,
you should set the corresponding attribute on the condition variable
with pthread_condattr_setclock().

I'm not sure you (or someone else) would start implementing C++11
threading on top of that but as far as I know, C++, at least C++11,
gives absolutely no guarantee on the clock being used. It's not even
that you can't rely on a monotonic one being available, it's that you
can't rely on the dumbest one being available and you cannot chose nor
check anything about clocks.
What you are doing here is using your own preference. I agree it makes
much more sense but you shouldn't chose a behaviour based on that,
especially when the most-widespread behaviour does the opposite.

-- 
Adrien Nader

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Something about replacing winpthread:

If mcfgthread is loaded as a dynamic library, everything is fine. Its DllMain() 
function gets a DLL_THREAD_DETACH notification upon thread exit, enabling 
cleanup of thread local storage.

However if you wish to link mcfgthread statically, there will be trouble:
The linker must be aware that there is a TLS directory to enable the essential 
TLS callback since there is no DllMain() in an executable.
TheTLS directory can be placed in standard startup files (crt\d.o) or an 
individual library but also must be refered in startup files, otherwise the 
linker would consider it unused and throw it away.
__attribute__((__used__)) woundn't help, because it is the object file (.o) in 
the static library that will be ignored.

This would require hacking the startup files too and might break the ABI.

--   
Best regards,
lh_mouse
2016-04-14

-
发件人：Ruben Van Boxem <vanboxem.ru...@gmail.com>
发送日期：2016-04-14 18:17
收件人：mingw-w64-public@lists.sourceforge.net
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

I'd really really really suggest just using --enable-threads=windows or
something, and implement them using somthing compatible to the windows
runtime if at all possible. This is available on Windows 7+ and seems as
reasonable as anything.

If there are better routines in e.g. Windows 8 or 10, it would be better to
make the decision at runtime if at all possible IMHO, so that newer GCC
versions can also be extended without yet another ABI breaking change like
this.

But hey, I'm all for a new threading model. This would be a great
opportunity to remove the winpthreads dependeny of C++11's std::thread
implementation in libstdc++.

Ruben

2016-04-14 12:12 GMT+02:00 Dongsheng Song <dongsheng.s...@gmail.com>:

> If we use  --enable-threads=win32-win7 , the result gcc running on and
> target to win7 or later looks reasonable.
>
> On Thu, Apr 14, 2016 at 2:34 PM, Martin Mitáš <m...@morous.org> wrote:
> >
> >
> > Dne 14. 4. 2016 v 5:02 Dongsheng Song napsal(a):
> >> Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map
> >>
> >> --enable-threads=win32-vista
> >> --enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
> >> --enable-threads=win32-win8   // Windows 8 and Windows Server 2012
> >> --enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
> >> --enable-threads=win32-win10  // Windows 10 and Windows Server 2016
> >>
> >> to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?
> >
> > Please don't.
> >
> > Consider that something like -D_WIN32_WINNT=xy -DWINVER=_WIN32_WINNT
> > is often used by projects to see newer Win32API structures and #defines,
> > but the project often still supports older windows in run time. A lot
> > of code out there plays with GetProcAddress() but doesn't want to
> redefine
> > all the new types and constants.
> >
> > Best regards,
> > Martin
> >
> >
> >



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

C++0x threading APIs are currently unimplemented.

It is only because gthr.h is so obscure about __gthred_time_t.
It doesn't say anything about whether __gthread_time_t should be a struct (like 
struct timespec) or a plain integer or even a floating point type, neither does 
it say where and how to obtain a __gthread_time_t that represents the current 
system time.
Nonetheless, all *timedwait() interfaces, for mutex, condition variable and (as 
extensions) even thread and once-init, are naturally supported by mcfgthread.

I really hate the 'struct timespec' thing, as well as using a UNIX timestamp in 
*_timedwait() functions because the wall clock is subject to DST and NTP and 
can also be manually set by the user.
Rather, (currently) all timepoints in mcfgthread are obtained using the 
_MCFCRT_GetFastMonoClock() function which is simply a wrapper for 
GetTickCount64() that is only available since Vista.
This guarantees that the clock is monotonic and can't be set by the user, as 
well as a good resolution of 1ms.

However, should readjusting the time origin be essential, it would be easy to 
use an alternative clock, code about which could be found here:
https://github.com/lhmouse/mcfgthread/blob/master/src/env/_nt_timeout.h?ts=4#L15

--   
Best regards,
lh_mouse
2016-04-14

-
发件人：Ruben Van Boxem <vanboxem.ru...@gmail.com>
发送日期：2016-04-14 18:17
收件人：mingw-w64-public@lists.sourceforge.net
抄送：
主题：Re: [Mingw-w64-public] Adding a new thread model to GCC

I'd really really really suggest just using --enable-threads=windows or
something, and implement them using somthing compatible to the windows
runtime if at all possible. This is available on Windows 7+ and seems as
reasonable as anything.

If there are better routines in e.g. Windows 8 or 10, it would be better to
make the decision at runtime if at all possible IMHO, so that newer GCC
versions can also be extended without yet another ABI breaking change like
this.

But hey, I'm all for a new threading model. This would be a great
opportunity to remove the winpthreads dependeny of C++11's std::thread
implementation in libstdc++.

Ruben

2016-04-14 12:12 GMT+02:00 Dongsheng Song <dongsheng.s...@gmail.com>:

> If we use  --enable-threads=win32-win7 , the result gcc running on and
> target to win7 or later looks reasonable.
>
> On Thu, Apr 14, 2016 at 2:34 PM, Martin Mitáš <m...@morous.org> wrote:
> >
> >
> > Dne 14. 4. 2016 v 5:02 Dongsheng Song napsal(a):
> >> Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map
> >>
> >> --enable-threads=win32-vista
> >> --enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
> >> --enable-threads=win32-win8   // Windows 8 and Windows Server 2012
> >> --enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
> >> --enable-threads=win32-win10  // Windows 10 and Windows Server 2016
> >>
> >> to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?
> >
> > Please don't.
> >
> > Consider that something like -D_WIN32_WINNT=xy -DWINVER=_WIN32_WINNT
> > is often used by projects to see newer Win32API structures and #defines,
> > but the project often still supports older windows in run time. A lot
> > of code out there plays with GetProcAddress() but doesn't want to
> redefine
> > all the new types and constants.
> >
> > Best regards,
> > Martin
> >
> >
> >

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[mity]

Maybe we misunderstand each other?

I understood your question as a proposal to make -D_WIN32_WINNT=xy imply
also implicitly some --enable-threads=xy. That would be a problem, because
semantics of -D_WIN32_WINNT=xy should imho have no implication on chosen
run time, but just on what's made available in Win32 header files.

Martin

> If we use  --enable-threads=win32-win7 , the result gcc running on and
> target to win7 or later looks reasonable.
>
> On Thu, Apr 14, 2016 at 2:34 PM, Martin Mitáš  wrote:
>>
>>
>> Dne 14. 4. 2016 v 5:02 Dongsheng Song napsal(a):
>>> Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map
>>>
>>> --enable-threads=win32-vista
>>> --enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
>>> --enable-threads=win32-win8   // Windows 8 and Windows Server 2012
>>> --enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
>>> --enable-threads=win32-win10  // Windows 10 and Windows Server 2016
>>>
>>> to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?
>>
>> Please don't.
>>
>> Consider that something like -D_WIN32_WINNT=xy -DWINVER=_WIN32_WINNT
>> is often used by projects to see newer Win32API structures and #defines,
>> but the project often still supports older windows in run time. A lot
>> of code out there plays with GetProcAddress() but doesn't want to
>> redefine
>> all the new types and constants.
>>
>> Best regards,
>> Martin
>>
>>
>> --
>> Find and fix application performance issues faster with Applications
>> Manager
>> Applications Manager provides deep performance insights into multiple
>> tiers of
>> your business applications. It resolves application problems quickly and
>> reduces your MTTR. Get your free trial!
>> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
>> ___
>> Mingw-w64-public mailing list
>> Mingw-w64-public@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>
> --
> Find and fix application performance issues faster with Applications
> Manager
> Applications Manager provides deep performance insights into multiple
> tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> ___
> Mingw-w64-public mailing list
> Mingw-w64-public@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Ruben Van Boxem]

I'd really really really suggest just using --enable-threads=windows or
something, and implement them using somthing compatible to the windows
runtime if at all possible. This is available on Windows 7+ and seems as
reasonable as anything.

If there are better routines in e.g. Windows 8 or 10, it would be better to
make the decision at runtime if at all possible IMHO, so that newer GCC
versions can also be extended without yet another ABI breaking change like
this.

But hey, I'm all for a new threading model. This would be a great
opportunity to remove the winpthreads dependeny of C++11's std::thread
implementation in libstdc++.

Ruben

2016-04-14 12:12 GMT+02:00 Dongsheng Song :

> If we use  --enable-threads=win32-win7 , the result gcc running on and
> target to win7 or later looks reasonable.
>
> On Thu, Apr 14, 2016 at 2:34 PM, Martin Mitáš  wrote:
> >
> >
> > Dne 14. 4. 2016 v 5:02 Dongsheng Song napsal(a):
> >> Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map
> >>
> >> --enable-threads=win32-vista
> >> --enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
> >> --enable-threads=win32-win8   // Windows 8 and Windows Server 2012
> >> --enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
> >> --enable-threads=win32-win10  // Windows 10 and Windows Server 2016
> >>
> >> to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?
> >
> > Please don't.
> >
> > Consider that something like -D_WIN32_WINNT=xy -DWINVER=_WIN32_WINNT
> > is often used by projects to see newer Win32API structures and #defines,
> > but the project often still supports older windows in run time. A lot
> > of code out there plays with GetProcAddress() but doesn't want to
> redefine
> > all the new types and constants.
> >
> > Best regards,
> > Martin
> >
> >
> >
> --
> > Find and fix application performance issues faster with Applications
> Manager
> > Applications Manager provides deep performance insights into multiple
> tiers of
> > your business applications. It resolves application problems quickly and
> > reduces your MTTR. Get your free trial!
> > https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> > ___
> > Mingw-w64-public mailing list
> > Mingw-w64-public@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>
>
> --
> Find and fix application performance issues faster with Applications
> Manager
> Applications Manager provides deep performance insights into multiple
> tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> ___
> Mingw-w64-public mailing list
> Mingw-w64-public@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Dongsheng Song]

If we use  --enable-threads=win32-win7 , the result gcc running on and
target to win7 or later looks reasonable.

On Thu, Apr 14, 2016 at 2:34 PM, Martin Mitáš  wrote:
>
>
> Dne 14. 4. 2016 v 5:02 Dongsheng Song napsal(a):
>> Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map
>>
>> --enable-threads=win32-vista
>> --enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
>> --enable-threads=win32-win8   // Windows 8 and Windows Server 2012
>> --enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
>> --enable-threads=win32-win10  // Windows 10 and Windows Server 2016
>>
>> to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?
>
> Please don't.
>
> Consider that something like -D_WIN32_WINNT=xy -DWINVER=_WIN32_WINNT
> is often used by projects to see newer Win32API structures and #defines,
> but the project often still supports older windows in run time. A lot
> of code out there plays with GetProcAddress() but doesn't want to redefine
> all the new types and constants.
>
> Best regards,
> Martin
>
>
> --
> Find and fix application performance issues faster with Applications Manager
> Applications Manager provides deep performance insights into multiple tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> ___
> Mingw-w64-public mailing list
> Mingw-w64-public@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Martin Mitáš]

Dne 14. 4. 2016 v 5:02 Dongsheng Song napsal(a):
> Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map
> 
> --enable-threads=win32-vista
> --enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
> --enable-threads=win32-win8   // Windows 8 and Windows Server 2012
> --enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
> --enable-threads=win32-win10  // Windows 10 and Windows Server 2016
> 
> to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?

Please don't. 

Consider that something like -D_WIN32_WINNT=xy -DWINVER=_WIN32_WINNT
is often used by projects to see newer Win32API structures and #defines,
but the project often still supports older windows in run time. A lot
of code out there plays with GetProcAddress() but doesn't want to redefine
all the new types and constants.

Best regards,
Martin


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

And don't forget that --enable-threads=win32 also enables emulated TLS, which 
is completely unnecessary if we have some kind of TLS support.

Because pthread_key_create() requires a destructor, TLS must still be emulated 
on x86 because there is no native support. The fiber local storage (FLS) APIs 
require the god damned __stdcall calling convention and are impossible to be 
made any use of.
On x64 __stdcall has no effect and FLS APIs can be used to implement TLS 
functions.

I tend to make a copy of the POSIX thread model and make some modification of 
it.

--   
Best regards,
lh_mouse
2016-04-14

-
发件人：Dongsheng Song 
发送日期：2016-04-14 11:02
收件人：lh_mouse
抄送：Torvald Riegel,gcc,mingw-w64-public
主题：Re: Re: Adding a new thread model to GCC

Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map

--enable-threads=win32-vista
--enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
--enable-threads=win32-win8   // Windows 8 and Windows Server 2012
--enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
--enable-threads=win32-win10  // Windows 10 and Windows Server 2016

to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?

Regards,
Dongsheng

On Thu, Apr 14, 2016 at 10:24 AM, lh_mouse  wrote:
> Yes I learnt those *ByAddress* functions a few months ago and was shocked 
> that Microsoft had plagiarized futex from Linux.
> As you have pointed out already, those APIs are only available on  Windows 8 
> and later.
>
> If we need fuex semantics on Windows 7 and earlier, we must simulate it.
> However, since futexes and keyed events work differently, simulation of 
> futexes using keyed events would:
>   0) require a lot of work, and
>   1) suffer from performance penalty just like if we simulate keyed events on 
> Windows 2000.
>
> The major difference is that, the FUTEX_WAIT syscall is an atomic 
> compare-and-sleep operation, while the NtWaitForKeyedEvent syscall provides 
> no comparison.
> Instead, the NtReleasedKeyedEvent syscall blocks the current thread until one 
> thread is woken up, while FUTEX_WAKE does nothing and returns immediately.
>
> So here is my condition: in no event shall portability harm either efficiency 
> or maintainability.
> Linux and Windows work differently. I consider it 'harmful' to make one look 
> like the other.
>
>
> --
> Best regards,
> lh_mouse
> 2016-04-14
>
> -
> 发件人：Torvald Riegel 
> 发送日期：2016-04-14 01:40
> 收件人：lh_mouse
> 抄送：gcc,mingw-w64-public
> 主题：Re: Adding a new thread model to GCC
>
> On Wed, 2016-04-13 at 17:17 +0800, lh_mouse wrote:
>> Hi all,
>>
>> The 'win32' thread model of gcc has been there since long long ago, being 
>> compatible with very old Windows versions, also having a number of drawbacks:
>>   0) its implementation is very inefficient, and
>>   1) its mutexes and condition variables require dynamic initialization and 
>> are unusable in C++11 as std::mutex requires a `constexpr` constructor, and
>>   2) allocating a number of rarely used mutexes or condition variables would 
>> eventually make the system run out of kernel objects.
>>
>> As a solution for 1) and 2), Microsoft introduced keyed events, details of 
>> which can be found here:
>> http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/
>
> Have you looked at WaitOnAddress and WakeByAddressSingle /
> WakeByAddressAll too?  AFAIK this is new in Windows 8, and seems similar
> to futexes.
>
> I think it might be better to get a std::synchronic (or similar)
> implementation into GCC, and then use these to implement at least the
> mutexes:
> https://github.com/ogiroux/synchronic/blob/master/include/synchronic
>
> One benefit would be that we then have one place where we have optimized
> spinning/blocking in libstdc++ on Windows, and less platform-specific
> code.
>
>
>



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Dongsheng Song]

Currently, --enable-threads=win32 map to gthr-win32.{h|c}, could we map

--enable-threads=win32-vista
--enable-threads=win32-win7   // Windows 7 and Windows Server 2008 R2
--enable-threads=win32-win8   // Windows 8 and Windows Server 2012
--enable-threads=win32-win8.1 // Windows 8.1 and Windows Server 2012 R2
--enable-threads=win32-win10  // Windows 10 and Windows Server 2016

to gthr-windows.{h|c} with the corresponding _WIN32_WINNT macros ?

Regards,
Dongsheng

On Thu, Apr 14, 2016 at 10:24 AM, lh_mouse  wrote:
> Yes I learnt those *ByAddress* functions a few months ago and was shocked 
> that Microsoft had plagiarized futex from Linux.
> As you have pointed out already, those APIs are only available on  Windows 8 
> and later.
>
> If we need fuex semantics on Windows 7 and earlier, we must simulate it.
> However, since futexes and keyed events work differently, simulation of 
> futexes using keyed events would:
>   0) require a lot of work, and
>   1) suffer from performance penalty just like if we simulate keyed events on 
> Windows 2000.
>
> The major difference is that, the FUTEX_WAIT syscall is an atomic 
> compare-and-sleep operation, while the NtWaitForKeyedEvent syscall provides 
> no comparison.
> Instead, the NtReleasedKeyedEvent syscall blocks the current thread until one 
> thread is woken up, while FUTEX_WAKE does nothing and returns immediately.
>
> So here is my condition: in no event shall portability harm either efficiency 
> or maintainability.
> Linux and Windows work differently. I consider it 'harmful' to make one look 
> like the other.
>
>
> --
> Best regards,
> lh_mouse
> 2016-04-14
>
> -
> 发件人：Torvald Riegel 
> 发送日期：2016-04-14 01:40
> 收件人：lh_mouse
> 抄送：gcc,mingw-w64-public
> 主题：Re: Adding a new thread model to GCC
>
> On Wed, 2016-04-13 at 17:17 +0800, lh_mouse wrote:
>> Hi all,
>>
>> The 'win32' thread model of gcc has been there since long long ago, being 
>> compatible with very old Windows versions, also having a number of drawbacks:
>>   0) its implementation is very inefficient, and
>>   1) its mutexes and condition variables require dynamic initialization and 
>> are unusable in C++11 as std::mutex requires a `constexpr` constructor, and
>>   2) allocating a number of rarely used mutexes or condition variables would 
>> eventually make the system run out of kernel objects.
>>
>> As a solution for 1) and 2), Microsoft introduced keyed events, details of 
>> which can be found here:
>> http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/
>
> Have you looked at WaitOnAddress and WakeByAddressSingle /
> WakeByAddressAll too?  AFAIK this is new in Windows 8, and seems similar
> to futexes.
>
> I think it might be better to get a std::synchronic (or similar)
> implementation into GCC, and then use these to implement at least the
> mutexes:
> https://github.com/ogiroux/synchronic/blob/master/include/synchronic
>
> One benefit would be that we then have one place where we have optimized
> spinning/blocking in libstdc++ on Windows, and less platform-specific
> code.
>
>
>

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Yes I learnt those *ByAddress* functions a few months ago and was shocked that 
Microsoft had plagiarized futex from Linux.
As you have pointed out already, those APIs are only available on  Windows 8 
and later.

If we need fuex semantics on Windows 7 and earlier, we must simulate it.
However, since futexes and keyed events work differently, simulation of futexes 
using keyed events would:
  0) require a lot of work, and
  1) suffer from performance penalty just like if we simulate keyed events on 
Windows 2000.

The major difference is that, the FUTEX_WAIT syscall is an atomic 
compare-and-sleep operation, while the NtWaitForKeyedEvent syscall provides no 
comparison.
Instead, the NtReleasedKeyedEvent syscall blocks the current thread until one 
thread is woken up, while FUTEX_WAKE does nothing and returns immediately.

So here is my condition: in no event shall portability harm either efficiency 
or maintainability.
Linux and Windows work differently. I consider it 'harmful' to make one look 
like the other.


--   
Best regards,
lh_mouse
2016-04-14

-
发件人：Torvald Riegel 
发送日期：2016-04-14 01:40
收件人：lh_mouse
抄送：gcc,mingw-w64-public
主题：Re: Adding a new thread model to GCC

On Wed, 2016-04-13 at 17:17 +0800, lh_mouse wrote:
> Hi all,
> 
> The 'win32' thread model of gcc has been there since long long ago, being 
> compatible with very old Windows versions, also having a number of drawbacks:
>   0) its implementation is very inefficient, and
>   1) its mutexes and condition variables require dynamic initialization and 
> are unusable in C++11 as std::mutex requires a `constexpr` constructor, and
>   2) allocating a number of rarely used mutexes or condition variables would 
> eventually make the system run out of kernel objects.
> 
> As a solution for 1) and 2), Microsoft introduced keyed events, details of 
> which can be found here:
> http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/

Have you looked at WaitOnAddress and WakeByAddressSingle /
WakeByAddressAll too?  AFAIK this is new in Windows 8, and seems similar
to futexes.

I think it might be better to get a std::synchronic (or similar)
implementation into GCC, and then use these to implement at least the
mutexes:
https://github.com/ogiroux/synchronic/blob/master/include/synchronic

One benefit would be that we then have one place where we have optimized
spinning/blocking in libstdc++ on Windows, and less platform-specific
code.




--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Thanks for your reply. :>

However I am still having problems building GCC. (Yesterday stage 1 was ok, but 
during stage 2 autoconf just failed to find any standard headers. I am still 
struggling with it.)
Once I build gcc successfully I will have a look at that configure.ac. Wish me 
luck. (btw this could be one or two days later. :<)

--   
Best regards,
lh_mouse
2016-04-13

-
发件人：Jonathan Wakely 
发送日期：2016-04-13 18:30
收件人：lh_mouse
抄送：gcc,mingw-w64-public
主题：Re: Adding a new thread model to GCC

On 13 April 2016 at 10:17, lh_mouse wrote:
> Hi all,
>
> The 'win32' thread model of gcc has been there since long long ago, being 
> compatible with very old Windows versions, also having a number of drawbacks:
>   0) its implementation is very inefficient, and
>   1) its mutexes and condition variables require dynamic initialization and 
> are unusable in C++11 as std::mutex requires a `constexpr` constructor, and
>   2) allocating a number of rarely used mutexes or condition variables would 
> eventually make the system run out of kernel objects.
>
> As a solution for 1) and 2), Microsoft introduced keyed events, details of 
> which can be found here:
> http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/
>
> In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
> since Vista and are working well without any dynamic initialization, just 
> like FUTEX.
> But there are still some major drawbacks:
>   0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
> equivalent function to pthread_mutex_timedwait, and

Some of our supported targets don't support pthread_mutex_timedwait
either (at least Mac OS X and HP-UX) so we have code in libstdc++ to
support mutexes but not timed mutexes. That might also work for your
new model.

>   1) the CONDITION_VARIABLE can only be used with SRWLOCK and 
> CRITICAL_SECTION and no user-defined mutexes can be supportd.

That's OK, std::condition_variable only needs to work with std::mutex,
and then std::condition_variable_any (which works with user-defined
mutexes) only requires std::mutex and std::condition_variable. The
code to use user-defined mutexes is portable C++11, so doesn't need
any platform-specific code.

> As a solution, I come up with my own library: 
> https://github.com/lhmouse/mcfgthread
> With 4 threads contending my mutex implementation turns to be 10 times faster 
> than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
> I have also done some research on gthr.h and successfully created a wrapper 
> for __gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

Great! I haven't looked at the code, but I've been hoping someone
would do something like this to improve the std::mutex and
std::condition_variable implementations for modern Windows systems.


> I am currently looking for ways to integrate this library into gcc as a new 
> thread model, let's say 'nt6' for example, so GCC can be configured with 
> --enable-threads=nt6.
> But there is so little documentation about that. If someone know how to add a 
> new thread model in GCC, I would appreciate that.

See line 1589 in gcc/configure.ac, where the allowed thread models are
defined. Your gthreads wrapper header should be added as
libgcc/config/i386/gthr-nt6.h and you might also need some changes to
libgcc/configure.ac (see how it handles the existing gthr-win32.h
model).

You will need to use autoconf to regenerate the configure files after
editing the configure.ac files (or for now you can just edit the
configure files by hand, but eventually changes must go into the
configure.ac files). I can probably help with that, although I can't
build or test anything on Windows.

Once you've added the new thread model as a gthr-xxx.h file in libgcc
there will also be some work needed in libstdc++ to detect and
correctly use the new model (e.g. to detect that the timedwait
functions are not supported, so that timed mutexes will be disabled).

We might want to discuss that latter part on the libstdc++ list.



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

Re: [Mingw-w64-public] Adding a new thread model to GCC

[Jonathan Wakely]

On 13 April 2016 at 10:17, lh_mouse wrote:
> Hi all,
>
> The 'win32' thread model of gcc has been there since long long ago, being 
> compatible with very old Windows versions, also having a number of drawbacks:
>   0) its implementation is very inefficient, and
>   1) its mutexes and condition variables require dynamic initialization and 
> are unusable in C++11 as std::mutex requires a `constexpr` constructor, and
>   2) allocating a number of rarely used mutexes or condition variables would 
> eventually make the system run out of kernel objects.
>
> As a solution for 1) and 2), Microsoft introduced keyed events, details of 
> which can be found here:
> http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/
>
> In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
> since Vista and are working well without any dynamic initialization, just 
> like FUTEX.
> But there are still some major drawbacks:
>   0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
> equivalent function to pthread_mutex_timedwait, and

Some of our supported targets don't support pthread_mutex_timedwait
either (at least Mac OS X and HP-UX) so we have code in libstdc++ to
support mutexes but not timed mutexes. That might also work for your
new model.

>   1) the CONDITION_VARIABLE can only be used with SRWLOCK and 
> CRITICAL_SECTION and no user-defined mutexes can be supportd.

That's OK, std::condition_variable only needs to work with std::mutex,
and then std::condition_variable_any (which works with user-defined
mutexes) only requires std::mutex and std::condition_variable. The
code to use user-defined mutexes is portable C++11, so doesn't need
any platform-specific code.

> As a solution, I come up with my own library: 
> https://github.com/lhmouse/mcfgthread
> With 4 threads contending my mutex implementation turns to be 10 times faster 
> than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
> I have also done some research on gthr.h and successfully created a wrapper 
> for __gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

Great! I haven't looked at the code, but I've been hoping someone
would do something like this to improve the std::mutex and
std::condition_variable implementations for modern Windows systems.


> I am currently looking for ways to integrate this library into gcc as a new 
> thread model, let's say 'nt6' for example, so GCC can be configured with 
> --enable-threads=nt6.
> But there is so little documentation about that. If someone know how to add a 
> new thread model in GCC, I would appreciate that.

See line 1589 in gcc/configure.ac, where the allowed thread models are
defined. Your gthreads wrapper header should be added as
libgcc/config/i386/gthr-nt6.h and you might also need some changes to
libgcc/configure.ac (see how it handles the existing gthr-win32.h
model).

You will need to use autoconf to regenerate the configure files after
editing the configure.ac files (or for now you can just edit the
configure files by hand, but eventually changes must go into the
configure.ac files). I can probably help with that, although I can't
build or test anything on Windows.

Once you've added the new thread model as a gthr-xxx.h file in libgcc
there will also be some work needed in libstdc++ to detect and
correctly use the new model (e.g. to detect that the timedwait
functions are not supported, so that timed mutexes will be disabled).

We might want to discuss that latter part on the libstdc++ list.

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

[Mingw-w64-public] Adding a new thread model to GCC

[lh_mouse]

Hi all,

The 'win32' thread model of gcc has been there since long long ago, being 
compatible with very old Windows versions, also having a number of drawbacks:
  0) its implementation is very inefficient, and
  1) its mutexes and condition variables require dynamic initialization and are 
unusable in C++11 as std::mutex requires a `constexpr` constructor, and
  2) allocating a number of rarely used mutexes or condition variables would 
eventually make the system run out of kernel objects.

As a solution for 1) and 2), Microsoft introduced keyed events, details of 
which can be found here:
http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/

In one word: the Windows SRWLOCK and CONDITION_VARIABLE have been introduced 
since Vista and are working well without any dynamic initialization, just like 
FUTEX.
But there are still some major drawbacks:
  0) the SRWLOCK can be used as an exclusive mutex but it does not have an 
equivalent function to pthread_mutex_timedwait, and 
  1) the CONDITION_VARIABLE can only be used with SRWLOCK and CRITICAL_SECTION 
and no user-defined mutexes can be supportd.

As a solution, I come up with my own library: 
https://github.com/lhmouse/mcfgthread
With 4 threads contending my mutex implementation turns to be 10 times faster 
than the gthr-win32 implementation on my Xeon E3-1230 v3 processor.
I have also done some research on gthr.h and successfully created a wrapper for 
__gthread_* functhins, which can be found in mcfgthread/src/env/gthread.h.

I am currently looking for ways to integrate this library into gcc as a new 
thread model, let's say 'nt6' for example, so GCC can be configured with 
--enable-threads=nt6.
But there is so little documentation about that. If someone know how to add a 
new thread model in GCC, I would appreciate that.

Also, this library has not been fully tested. If someone has interest in 
exploring how it works or testing it, I might be able to help.

--
Best regards,
lh_mouse
2016-04-13


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public