On 09/13/2018 09:49 PM, Jonathan M Davis wrote:
Have you read the concurrency chapter in The D Programming Language by
Andrei? It sounds like you're trying to describe something vere similar to
the synchronized classes from TDPL (which have never been fully implemented
in the language). They would make it so that you had a class with shared
members but where the outer layer of shared was stripped away inside member
functions, because the compiler is able to guarantee that they don't escape
(though it can only guarantee that for the outer layer). Every member
function is synchronized and no direct access to the member variables
outside of the class (even in the same module) is allowed. It would make
shared easier to use in those cases where it makes sense to wrapped
everything protected by a mutex in a class (though since it can only safely
strip away the outer layer of shared, it's more limited than would be nice,
and there are plenty of cases where it doesn't make sense to stuff something
in a class just to use it as shared).
I hadn't read the book, but that's indeed the gist of what I'm
proposing. I think it could be enough to restrict it to value types,
where it's easier to assume (and even check) that there are no external
references.
[snip]
If we're going to find ways to make shared require less manual work, it
means finding a way to protect a shared object (or group of shared objects)
with a mutex in a way that is able to guarantee that when you operate on the
data, it's protected by that mutex and that no reference to that data has
escaped. TDPL's synchronized classes are one attempt to do that, but the
requirement that no references escape (so that shared can safely be cast
away) makes it so that only the outer layer of shared can be cast away, and
it's extremely difficult to do better than that with having holes such that
it isn't actually guaranteed to be thread-safe when shared is cast away.
Maybe someone will come up with something that will work, but I wouldn't bet
on it. Either way, I don't see how any solution is going to be acceptable
which does not actually guarantee thread-safety, because it would be
violating the guarantees of shared otherwise. A programmer can choose to
cast away shared in an unsafe manner (or use __gshared) and rely on their
ability to ensure that the code is thread-safe rather than letting shared do
its job, but that's not the sort of thing that we're going to do with a
language construct, and given that the compiler assumes that anything that
isn't shared or immutable is thread-local, it's very much a risky thing to
do.
I completely agree with this argument, however please note that there
must be a sensible way to work with shared, otherwise we enter in the
"the perfect is the enemy of the good" area.
For reference types it's somehow workable, because you can just cast
away and store it in a new variable:
```
class A {
this() { }
}
shared synchronized class B {
this(A a) {
a_ = cast (shared) new A; // no shared this()
}
void foo() {
A a = cast () a_;
// Work with it
}
private:
A a_;
}
```
It's still somewhat cumbersome, specially if you have many such members,
but still doable.
However, this is not possible for value types, and it makes it nigh on
impossible to work with them in a sensible way. You have either to use
pointers, or cast away every type you want to use it. None of them are
what I would call "practical".
While not the biggest problem (see the later point), I still think that
synchronized classes are a good compromise, specially with the
restriction of only applying to full value types (no internal references
allowed). Of course it is still perhaps possible to bypass that
mechanism, but so is the case with many other ones (assumeUnique?).
If it's hard enough to do by mistake, it can be assumed that the people
messing with it should know what they are doing.
Finally, you suggest using __gshared, and I'm not sure you're not having
the same misunderstanding I had: __gshared implies "static", so it's not
a valid solution for class fields in most cases.
As for __gshared, it's intended specifically for C globals, and using it for
anything else is just begging for bugs. Because the compiler assumes that
anything which is not marked as shared or immutable is thread-local, having
such an object actually be able to be mutated by another thread risks subtle
bugs of the sort that shared was supposed to prevent in the first place.
Unfortunately, due to some of the difficulties in using shared and some of
the misunderstandings about it, a number of folks have just used __gshared
instead of shared, but once you do that, you're risking subtle bugs, because
that's not at all what __gshared is intended for. If you're using __gshared
for anything other than a C global, it's arguably a bug. Certainly, it's a
risky proposition.
As I said, the current semantics of __gshared doesn't allow it to be a
"drop-in" replacement of "shared". I also agree that it's not what it
was meant for, and that changing that right now would risk breaking a
lot of code.
However, I think that there should be *some* way in the language itself
to express that without having to cast all over the place: access this
member of the shared class as if it were local, I'll take care of
controlling the access to it.
You can use a wrapper type, and that's what I'm trying to do right now,
but I'm pretty sure there will be a ton of corner cases and interactions
that will make it really hard to work reliably in a generic way.
Then, implementing a kind of "synchronized classes" that would automate
this when possible would of course be a further and welcome improvement.
A.
