Re: Sending an immutable object to a thread
On Friday, 24 July 2015 at 18:02:58 UTC, Ali Çehreli wrote:
Although the example casts to void*, ubyte* and others are
possible as well, and casting back to the correct class type
seems to work:
Thanks, Ali.
I just tried a few things, and apparently, you don't need to go
to a different type or void. You can make it a pointer the the
actual type, which might be good for self-documentation or
pattern matching. But what I find somewhat odd and fascinating is
what you show, that for the reference, r:
cast(D*)r != r
So this code all works out:
import std.stdio;
class B
{
int i;
this(int i) { this.i = i; }
}
void main()
{
auto r1 = new B(42),
r2 = r1;
writefln(Address of reference r1: %s, r1);
writefln(Address of reference r2: %s, r2);
writefln(Address of object r1 : %s, cast(B*)r1);
writefln(Address of object r2 : %s, cast(B*)r2);
assert(cast(B*)r1 == cast(B*)r2);
assert(cast(B*)r1 != r1);
assert(cast(B*)r2 != r2);
}
and prints:
Address of reference r1: 7FFF1D4E4C70
Address of reference r2: 7FFF1D4E4C78
Address of object r1 : 7F01CD506000
Address of object r2 : 7F01CD506000
So then, of course, I hope/wonder/assume that the pointer to the
heap is sufficient to keep the heap memory alive, and that this
would be OK from the GC perspective to do something like this:
B* make_b_thing(int i) { cast(B*) new B(i); }
That seems to work, but I guess I should try to force the garbage
collector to run to see if I can crash the program.
***BUT***: The really, really weird thing is that even though you
*think* that you have a pointer to a B object, you don't really.
Dereferencing is accepted by the compiler, but it plays a nasty
trick on you:
B* p = make_b_thing(42);
writefln(Address of pointer: %s, p);
writefln(Value of i: %s, p.i);
writefln(Value of i: %s, (*p).i);
writefln(Value of i: %s, (cast(B)p).i);
This compiles and runs fine, but produces:
Address of pointer: 7F7EE77CF020
Value of i: 4445040
Value of i: 4445040
Value of i: 42
Maybe it's my C++ background talking, but that seems a bit
counter-intuitive.
Re: Sending an immutable object to a thread
On Friday, 24 July 2015 at 19:28:35 UTC, anonymous wrote: I haven't followed the discussion, so I may be missing the point here. I started by asking how to send a reference to an immutable class object from one thread to another if the reference is one of several parameters being sent. The concurrency library can't make a tuple for send/receive if the reference is immutable. The short answer is (probably) to use a rebindable reference, but a suggestion arose about the possible use of a pointer instead. So we've devolved into a discussion of how pointers to class objects work, and how to keep the heap memory alive as the pointer is sent to the second thread as the original reference goes out of scope in the first. A B* is not a pointer to the memory of the object. It's a pointer to a class reference. The class reference itself, B, is a pointer to the memory of the object, under the hood. Hahaha. My forehead is red from the number of times I've thought I've gotten it in this discussion. So I think I understand that when you start to peek under the hood, the language treats the reference and the heap memory as distinct entities, and that they work differently for struct's and classes. So then: is there a pointer notation to which you can cast the B reference, which thus points to the heap, but retains type identity of the heap object? And the reason I ask if I wanted to declare a type which is a mutable pointer to an immutable object that is on the GC heap, keeping type info? Or is the answer, no, just use Rebindable when necessary? At this point, I'm not asking what should I do, but what could I do.
Re: Sending an immutable object to a thread
On Thursday, 23 July 2015 at 09:05:12 UTC, Marc Schütz wrote:
It is not safe, but for a different reason: `mt` is already a
_reference_ to the actual object (that's how classes behave in
D). This reference is located in a register or on the stack,
and `mt` is therefore a pointer into the stack.
It's illegal to return that pointer from the function, because
it will become invalid once the function is left. Fortunately,
the compiler can detect simple cases like this one, and will
refuse to compile it:
Object* foo() {
Object o;
return o;
}
xx.d(3): Error: escaping reference to local o
Very interesting. You see, I am trying to resolve the distinction
be a value type and a reference type in D.
If Object were declared as a struct, this would make sense to
me. The object would be created on the stack as a temporary, and
it would disappear when the function exited. So returning a
pointer to it would be a very, very bad thing.
But a class object is allocated in the GC heap. I would have
guessed that, since a class reference is just essentially a
hidden pointer, that the address-of operator '' for a class
object would return the address into the heap... not the address
of the reference itself! Just a little syntactic sugar.
But that's not the case. I thought this was true:
class MyThing { ... };
MyThing a = new MyThing,
b = a;
assert(a == b); // Fails
In a weird way, that makes total sense to me, and no sense at all.
So, passing a pointer to a stack-based reference from one thread
is another is not necessarily a good thing to do, as the original
reference might disappear while the thread is using it.
Is there a way to get the address of the actual heap object from
a class reference? Or am I drifting to far into the realm of
unsafe.
This all goes back to my original question of passing an
immutable object from one thread to another. It is simple with
arrays, since there is a clear distinction between the array
reference and its contents. You can easily create a mutable
reference to immutable contents with an array.
But it seems rather convoluted with class objects.
So, in the end, it seems best to send a rebindable reference to
the other thread, and perhaps hide the mild ugliness behind a
library API that takes an immutable object and then sends the
rebindable version, like:
void send_message(Tid tid, immutable(Message) msg) {
send(tid, thisTid(), rebindable(msg));
}
That seems easy enough.
Thanks much for all the help.
Re: Sending an immutable object to a thread
On Wednesday, 22 July 2015 at 09:04:49 UTC, Marc Schütz wrote:
But as long as the original pointer is still on the stack, that
one _will_ keep the object alive. It is only a problem if all
pointers to a GC managed object are stored in places the GC
isn't informed about.
Sorry, I have gotten confused. In Ali's example, the pointer to a
class object (via the address-of '' operator) actually points
into the GC heap. It is *not* a pointer to a pointer, right?
My reading of the Garbage web doc page is that this pointer to
memory in the GC heap is sufficient (by some magic) to keep the
memory alive, in and of itself.
So the pointer, passed to the other thread is sufficient to keep
the memory alive, even if the original reference disappears.
Or, to put it another way, getting threads out of the equation,
is this safe?
class MyThing { ... }
MyThing* create_a_thing() {
MyThing mt = new MyThing();
do_something_with(mt);
return mt;
}
void main() {
MyThing* pmt = create_a_thing();
// ...
}
The thing will remain alive for the duration of main() ??
Thanks
Re: Sending an immutable object to a thread
On Sunday, 19 July 2015 at 17:12:07 UTC, rsw0x wrote:
a pointer to a pointer(or in this case, a reference) does not
keep it alive.
Interesting. If you de-reference the pointer and assign it back,
do you get back the keep-alive? Like, in the receiving thread:
void threadFunc()
{
receive((Tid cli, immutable(Message) *m) {
immutable(Message) msg = *m; // ---
int retCode = do_something_with(msg);
send(cli, retCode);
});
}
I assume that even if so, there is a race condition there. You
would need to keep the original reference alive until at least
the msg = *m assignment happened, right?
Or... could you tell the GC to leave the memory alone until the
thread gets it? Like in the sending thread:
Tid tid = spawn(threadFunc);
auto p = cast(void*) msg;
GC.addRoot(p);
GC.setAttr(p, GC.BlkAttr.NO_MOVE);
send(tid, thisTid(), msg);
//...
Is that possible? Is it efficient enough to do if you're sending
lots and lots of messages?
Thanks.
Re: Sending an immutable object to a thread
It looks like passing a pointer to an immutable(Message) works
as well:
Oh, yes, pointer. Ha! I didn't even think of that. Thanks.
I'm not familiar with how garbage collection works in D. If the
initial reference goes out of scope, and you just have a pointer
- in another thread, no less - then are you still guaranteed that
the object will not disappear while the pointer exists?
Like if I did something akin to:
// ...like before...
void send_msg(Tid tid, int n)
{
auto msg = new immutable(Message)(n);
send(tid, thisTid(), msg);
}
void main()
{
Tid tid = spawn(threadFunc);
send_msg(tid, 100);
receiveOnly!int();
}
Do I know that the message object won't be garbage collected
before the thread finishes with it? (I realize this is a very
artificial example, but something like this could happen in a
bigger library).
Frank
Sending an immutable object to a thread
Hey All,
I'm trying to send immutable class objects to a thread, and am
having trouble if the object is one of several variables sent to
the thread. For example, I have a Message class:
class Message { ... }
and I create an immutable object from it, and send it to another
thread:
auto msg = immutable Message(...);
Tid tid = spawn(threadFunc);
send(tid, thisTid(), msg);
I then attempt to receive it in the threadFunc like:
receive(
(Tid cli, immutable Message msg) {
int retCode = do_something_with(msg);
send(cli, retCode);
}
);
I get compilation errors about the inability of building the
tuple, like:
/usr/include/dmd/phobos/std/variant.d(346): Error: cannot modify
struct *zat Tuple!(Tid, immutable(Message)) with immutable members
/usr/include/dmd/phobos/std/variant.d(657): Error: template
instance std.variant.VariantN!32LU.VariantN.handler!(Tuple!(Tid,
immutable(Message))) error instantiating
/usr/include/dmd/phobos/std/variant.d(580):instantiated
from here: opAssign!(Tuple!(Tid, immutable(Message)))
/usr/include/dmd/phobos/std/concurrency.d(124):
instantiated from here: __ctor!(Tuple!(Tid, immutable(Message)))
/usr/include/dmd/phobos/std/concurrency.d(628):
instantiated from here: __ctor!(Tid, immutable(Message))
/usr/include/dmd/phobos/std/concurrency.d(618):... (1
instantiations, -v to show) ...
/usr/include/dmd/phobos/std/concurrency.d(594):
instantiated from here: _send!(Tid, immutable(Message))
MsgTest.d(92):instantiated from here: send!(Tid,
immutable(Message))
I tried various combinations of using Rebindable, but couldn't
get anything to compile.
Thanks.
Re: Sending an immutable object to a thread
OK, I found a couple of solutions, though if anyone can tell me
something better, I would love to hear it.
By making an alias to a rebindable reference, the receive() was
able to create the tuple. So I renamed the class MessageType:
class MessageType { ... };
and then made a Message an immutable one of these:
alias immutable(MessageType) Message;
and finally made a VarMessage as a rebindable Message (thus, a
mutable reference to an immutable object):
alias Rebindable!(Message) VarMessage;
[I will likely rethink these names, but anyway... ]
Now I can send a reference to an immutable object across threads.
The receiver wants the VarMessage:
receive(
(Tid cli, VarMessage msg) {
int retVal = do_something_with(msg);
send(cli, retVal);
}
);
and a few different things work to send the object:
auto msg = new Message(...);
send(tid, thisTid(), VarMessage(msg));
or:
send(tid, thisTid(), rebindable(msg));
or:
VarMessage vmsg = new Message(...);
send(tid, thisTid(), vmsg);
A second way that seems plausible is to just make the message a
var type using a struct and then send a copy to the thread. This
seems viable since the vast bulk of the message is a string
payload, and thus the size of the struct is pretty small.
Re: Looking for MQTT client library
I got the OK to submit the D library to Eclipse Paho. So, hopefully within the next few weeks there will be a Paho incubator project for the D language client.
