Re: Building application with LDC and -flto=thin fails in link stage
On Monday, 26 March 2018 at 22:07:49 UTC, Nordlöw wrote: When I try build my application using LDC and -flto=thin it fails in the final linking as According to LDC's Release info: Known issues: ThinLTO may not work well with the ld.bfd linker, use ld.gold instead (-linker=gold). Maybe this is the problem. Arek
Re: Terminating multiple processes
On Thursday, 1 February 2018 at 12:30:24 UTC, Russel Winder wrote:
On Thu, 2018-02-01 at 12:15 +, Arek via Digitalmars-d-learn
wrote:
[…]
Try to use inotify in non-blocking mode (an example here:
https://gist.github.com/pkrnjevic/6016356) with select or
epoll and timeouts.
Isn't there a C++ binding for the C API?
I am using DInotify which is a D binding. I will be checking
soon but I am assuming there is a timeout version so I can loop
to check the application state.
DInitify doesn't cover full capabilities of the inotify API.
Especially it doesn't utilize newer inotify_init1 syscall and
doesn't expose 'select' interface.
This C++ wrapper may be interesting for you:
https://github.com/erikzenker/inotify-cpp
But I haven't used it.
I would use shared memory here (eg. atomic bool) because any
communication channel introduces possibility of further
blocking problems.
A priori I am not convinced. I have used a state variable in
C++ and Python code where there is no channel system, but in
Go, Groovy/GPars, using channels is always preferable. Given
the channel has a "read if there is something to read" there
can't be a blocking problem – if the channel system is a good
one.
You may be right. But interrupting the thread is very low level
mechanism.
Eg in Java, it is incorporated into Thread class: thread has
method 'interrupt()' which
sets the flag, and there is property 'isInterrupted()'. Usually
your thread code have something like
while(!Thread.currentThread().isInterrupted()) { /* do
something*/ }
I would use similar pattern.
Arek
Re: Terminating multiple processes
On Thursday, 1 February 2018 at 11:42:32 UTC, Russel Winder wrote: On Wed, 2018-01-31 at 22:15 +, Arek via Digitalmars-d-learn wrote: […] The problem is actually a thread blocked in an inotify blocking read. As both Steven and yourself have pointed out I am going to have to use a timeout to check the state of the application. Try to use inotify in non-blocking mode (an example here: https://gist.github.com/pkrnjevic/6016356) with select or epoll and timeouts. I guess there is a choice here between shared memory to set the termination flag, or using an input channel and sending the termination message. I think the latter may be preferable, and certainly more consistent with how the other threads terminate. I would use shared memory here (eg. atomic bool) because any communication channel introduces possibility of further blocking problems. Arek
Re: Terminating multiple processes
On Wednesday, 31 January 2018 at 17:44:37 UTC, Russel Winder wrote: So, I have an application which has a sort of nano-services architecture, basically it is a set of communicating processes. Terminating those processes blocked on an input channel is quite easy, send a terminate message on the input channel. But what about a process that has no input channel, one that is blocked on OS events? Is there a way of forcibly, but nicely, terminating a spawned process that never executes `receive()`? Assuming your're talking about threads: there's no secure method of forcing the thread to stop. Threads share the state (eg. can hold the locks) and killing them is always risky. If your threads are blocked reading the socket, you probably can close these sockets and exit after the read error. Another way is to use atomic flag indicating that thread needs to be interrupted. After any blocking operation, the thread have to check this flag and finish the job. It's good to use timeouts (eg socket timeout) in such scenario (if possible). Arek
Re: Dub and libraries
On Thursday, 4 January 2018 at 19:05:59 UTC, Russel Winder wrote: Is it the case that, for library things on the Dub repository, Dub will only create library archives, .a, that it is unable to create shared objects and DLLs? If they have "targetType" set to "dynamicLibrary" dub creates shared libraries. https://code.dlang.org/package-format?lang=sdl#target-types
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Tuesday, 15 August 2017 at 21:54:23 UTC, Steven Schveighoffer
wrote:
On 8/15/17 5:27 PM, Arek wrote:
On Tuesday, 15 August 2017 at 10:37:08 UTC, Kagamin wrote:
Well, no wrapper is actually needed here:
[...]
The issue is that send cannot handle shared value types due to
a bug in the implementation. In essence, there is no reason to
send a shared A -- it's an unrelated copy and not actually
shared.
You can send a shared reference to an A just fine:
static shared A a;
send(tid, ); // works
You *should* be able to send a shared(A). There is no reason to
disallow it. But again, it's not super useful.
Yes, I absolutely agree. The object may be shared due to other
requirements and there is no reason to reject 'share' qualifier
what I want to send/receive it.
This very simple code also doesn't compile:
shared struct S
{
int i;
~this()
{
}
}
void main()
{
shared s = shared S();
}
In general, shared structs with postblit and destructor make
problems.
postblit should work.
It does look like destructors are a problem. It appears the
compiler attempts to call the destructor while unshared.
-Steve
But I have no idea how and when the object loses its shared
nature. Yes, it looks like a bug in dmd. ldc 1.3 doesn't complain
in this case.
Arek
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Tuesday, 15 August 2017 at 10:37:08 UTC, Kagamin wrote:
Well, no wrapper is actually needed here:
class A
{
int method() shared;
}
void consumer()
{
shared a = receiveOnly!(shared A)();
}
void producer()
{
auto cons = spawn();
send(cons, new shared A());
}
Yes, but this doesn't compile:
import std.stdio;
import std.concurrency;
struct A
{
int t;
int r;
int method() shared
{
return 0;
}
}
void consumer()
{
shared a = receiveOnly!(shared A)();
}
void main()
{
auto cons = spawn();
send(cons, shared A());
}
This very simple code also doesn't compile:
shared struct S
{
int i;
~this()
{
}
}
void main()
{
shared s = shared S();
}
In general, shared structs with postblit and destructor make
problems.
Arek
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Monday, 14 August 2017 at 21:27:48 UTC, Jonathan M Davis wrote:
On Monday, August 14, 2017 15:22:23 Steven Schveighoffer via
Digitalmars-d- learn wrote:
On 8/13/17 11:40 PM, Jonathan M Davis via Digitalmars-d-learn
wrote:
> On Saturday, August 12, 2017 18:57:44 Arek via
> Digitalmars-d-learn
wrote:
>> I have the folowing problem:
>> I like to envelope the class object in struct to control the
>> destruction moment and then send this object to another
>> thread/fiber (or task, cause I use vibe-d).
>>
>> I can't find any method to make it working. Any ideas?
>
> Unfortunately, send and receive do not currently work with
> shared because of issues with Variant, which they use
> internally.
This can't be a correct statement. This is the whole point of
shared.
What's incorrect about it? It's a longstanding issue that
because Variant can't contain shared data, send and receive do
not work with shared. You can send and receive mutable data
with no indirections, and AFAIK, you can send immutable data
(though the OP is apparently having problems with that, so I
guess that that doesn't work completely, though I definitely
recall doing so previously), but you get compilation errors if
you try to send shared data. For instance, this code
import std.concurrency;
void main()
{
static void func(Tid parent)
{
auto received = receiveOnly!(shared int[string]);
}
shared int[string] aa;
auto tid = spawn(, thisTid);
send(tid, aa);
}
I've found some simple workaround for this problem:
import std.stdio;
import std.concurrency;
struct Envelope(T) if (is(T == class)) // for simplicity of this
example, only classes
{
shared(T)[] obj;
this(shared T o)
{
this.obj = [o];
}
T get() @property nothrow @nogc
{
return cast() obj[0];
}
}
class A
{
}
void consumer()
{
auto r = receiveOnly!(Envelope!(A))();
writeln("Got: ", typeof(r).stringof);
}
void main()
{
auto cons = spawn();
auto o = Envelope!A(new A());
send(cons, o);
}
Shared object can be encapsulated in the array. In case of other
(non-class) types the pointer can be used, and get() should
return ref to the pointed object (after stripping off the shared
qualifier).
send() could encapsulate itself shared objectes.
Arek
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Monday, 14 August 2017 at 19:22:23 UTC, Steven Schveighoffer wrote: On 8/13/17 11:40 PM, Jonathan M Davis via Digitalmars-d-learn wrote: On Saturday, August 12, 2017 18:57:44 Arek via Digitalmars-d-learn wrote: I have the folowing problem: I like to envelope the class object in struct to control the destruction moment and then send this object to another thread/fiber (or task, cause I use vibe-d). I can't find any method to make it working. Any ideas? Unfortunately, send and receive do not currently work with shared because of issues with Variant, which they use internally. This can't be a correct statement. This is the whole point of shared. -Steve First of all, I'm not native English speaker, so forgive me possible misunderstanding In my opinion the whole problem of 'shared' is that when I use send/receive operation (I believe it's should be something similar to go's channels or erlang messages) I do not want to share the memory so any "shared" concept is useless. What I really would like to get is possibility to make: // in first thread Unique!MyObject msg = new MyObject() send(msg.release); // in other thread: auto msg = receiveOnly!(Unique!MyObject)(); My object disappears in "send" operation and its new instance is constructed on receive. If I can ensure the uniqueness of the object, there is no need to "share" it or synchronize the access. But now such operation is impossible and the compiler forces me to cope with shared if MyObject has any references to other areas of memory. Anyway, thanks for all replies. Arek
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Monday, 14 August 2017 at 03:59:48 UTC, Jonathan M Davis wrote:
On Sunday, August 13, 2017 16:40:03 crimaniak via
Digitalmars-d-learn wrote:
More of this, I think, you can't avoid __gshared for any
complex work. Even mutexes from Phobos doesn't support shared,
so I had to 'cowboy with __gshared' when implementing my site
engine.
The way to handle shared is to protect the section of code
that's using the shared object with either a mutex or
synchronized block, and then you cast away shared from the
object within that section and operate on it as thread-local.
When you're done, you make sure that you don't have any
thread-local references to the data, and you release the mutex
or exit the synchronized block. e.g. something like
shared T sharedObj = getSharedObj();
synchronized(mutex)
{
T nonSharedObj = cast(T)sharedObject
// do stuff...
// make sure that no references to nonSharedObj have escaped
}
// now, there's just the shared version of the object
Yeah, and this is what i'm doing now (more or less).
To be more precise, I don't even want to synchronize access to
the shared resource between the threads. I just want to move the
object from one thread to another.
Of course I could copy the local object, but my obcjecs has
indirect references to others, forming kind of tree.
I like the idea of channels in Go. I've tried to get something
similiar with send/receive.
In this case, object could be also immutable, because once there
are created (in the deserialization process) they will no be
modified. I just have to emit them into another task (to be
honest, I use fiber, so it's not even another thread, and they
will not be accessed in parallel).
But all this language protections makes the issue unexpectedly
complicated.
And no, this isn't ideal, but the only semi-decent solution
that's been proposed that safely casts away shared for you is
synchronized classes, which Andrei describes in TDPL but have
never been implemented. And because they can only safely strip
off the outermost layer of shared, they're of questionable
usefulness anyway. Ultimately, even with synchronized classes,
in many situations, the programmer is going to have to
carefully cast away shared to operate on the object within a
protected context.
Now, the fact that the mutex objects don't handle shared
correctly is another issue entirely. Having to cast away shared
from mutexes is dumb, because you're obviously not going to be
protecting them with a mutex, and their operations have to be
atomic anyway for them to do what they do. So, that definitely
needs to be fixed. However, I believe that it _has_ been fixed
in master, and it might have made it into a release now, but
I'm not sure. So, core.sync.mutex.Mutex _should_ now be useable
as shared like it should be.
In general though, the idea is that you simply don't operate on
shared objects except via atomic operations. Otherwise, you
risk concurrency problems. And really, this is the same as what
you'd do in C/C++, except that in C/C++, it doesn't catch you
when you operate on an object that's shared across threads with
non-atomic operations (because the object isn't explicitly
typed as shared), and you don't have to cast away shared to do
non-atomic operations. So, having to cast away shared is the
price of getting the protection against accidentally using
non-atomic operations on a shared object as well as the price
we pay to be able to have the type system distinguish between
shared and thread-local objects so that it's able to optimize
based on the knowledge that an object is thread-local.
Ultimately though, you're doing the same thing that you'd do in
C++ if you're handling concurrency safely. You just have to
explicitly mark stuff as shared and carefully cast away shared
in certain, protected contexts.
Using __gshared in extern(D) code is just asking for it,
because then you have an object that the compiler thinks is
thread-local but isn't, and you risk subtle and nasty bugs as a
result. __gshared is only intended for binding to extern(C),
global variables. To an extent, you can get away with using it
with extern(D) variables, but that's not its intended purpose,
and you risk running afoul of the compiler and what it chooses
to do based on the assumption that the object is thread-local.
- Jonathan M Davis
Thanks for the explanation! It would be good to have a
comprehensive article on this subject.
Arek
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Monday, 14 August 2017 at 03:40:26 UTC, Jonathan M Davis wrote: On Saturday, August 12, 2017 18:57:44 Arek via Digitalmars-d-learn wrote: I have the folowing problem: I like to envelope the class object in struct to control the destruction moment and then send this object to another thread/fiber (or task, cause I use vibe-d). I can't find any method to make it working. Any ideas? Unfortunately, send and receive do not currently work with shared because of issues with Variant, which they use internally. - Jonathan M Davis That's what I suspected. Thanks for the confirmation. I was not sure if I was doing everything right. Arek
Re: Does anyone understand how to use "shared" types with concurrency send/receive functions?
On Sunday, 13 August 2017 at 02:50:13 UTC, crimaniak wrote: On Saturday, 12 August 2017 at 18:57:44 UTC, Arek wrote: I have the folowing problem: I like to envelope the class object in struct to control the destruction moment and then send this object to another thread/fiber (or task, cause I use vibe-d). I can't find any method to make it working. Any ideas? I tried it too some time ago. Then I read Alexandrescu book and realized that the authors of the language do not want anyone to do this. Long story short, just plan your application so that each complex object is monitored by only one thread/task, and pass not objects, but messages (immutable structs) about what to do with them. Yeah, I've read this. But conurrency.send cannot pass immutable object. The same story with Unique. What means, that if I have complex objects which I want to transfer between threads I must keep them somewhere in global memory and share among threads. I can use send/receive only to signal availability of their existance. What I need is kind of "move" operation beetwen threads. More over, "shared" looks rather like unfinished concept. It is really difficult to create proper struct (with postblit and destructor) working as shared object. I even have no clue what is exact semantics of "shared" types. Language specification is a little laconic. Anyway, _gshared looks very promising, so I will try to work out any approach. Thanks for answer. Arek
Does anyone understand how to use "shared" types with concurrency send/receive functions?
I have the folowing problem:
I like to envelope the class object in struct to control the
destruction moment and then send this object to another
thread/fiber (or task, cause I use vibe-d).
I can't find any method to make it working. Any ideas?
dmd (version 075) gives so stupid results, I belive it's broken
(I've created the issue
https://issues.dlang.org/show_bug.cgi?id=17749 )
ldc2 returns some errors:
.../import/std/variant.d(610,27): Error: function
core.stdc.string.memcpy (void* s1, const(void*) s2, ulong n) is
not callable using argument types (ubyte[32]*,
shared(Env!(shared(A)))*, ulong)
.../import/std/conv.d(4186,9): Error: static assert "Cannot
emplace a Env!(shared(A)) because Env!(shared(A)).this(this) is
annotated with @disable."
.../import/std/conv.d(4198,24):instantiated from here:
emplaceRef!(Env!(shared(A)), Env!(shared(A)), Env!(shared(A)))
.../import/std/variant.d(301,35):instantiated from here:
emplaceRef!(Env!(shared(A)), Env!(shared(A)))
.../import/std/variant.d(630,21):instantiated from here:
handler!(shared(Env!(shared(A
.../import/std/variant.d(544,17):... (5 instantiations,
-v to show) ...
../../../.dub/packages/vibe-core-1.1.1/vibe-core/source/vibe/core/concurrency.d(1223,64):
instantiated from here: send!(shared(Env!(shared(A
app.d(74,7):instantiated from here:
send!(shared(Env!(shared(A
The code may be complied like this: dub --single --compiler=ldc2
./app.d (assuming it's saved in app.d file).
/+
dub.sdl:
name "simple"
dependency "vibe-core" version="~>1.1.0"
+/
import std.stdio;
import std.format;
import vibe.core.core;
import vibe.core.task;
import vibe.core.concurrency;
// simple class with destructor
shared class A
{
int i;
this(int i)
{
this.i = i;
}
~this()
{
writeln("destruct ", i);
}
}
shared struct Env(T) if (is(T == class) && is(T == shared))
{
T obj;
this(T o)
{
obj = obj;
}
this(this)
{
// some magic here
}
auto opAssign(shared(Env!T) other)
{
obj = other.obj;
return this;
}
auto opAssign(ref shared(Env!T) other)
{
obj = other.obj;
return this;
}
~this()
{
if (obj !is null)
destroy(obj);
obj = null;
}
}
void main()
{
auto consumer = runTask({
auto got = receiveOnly!(shared Env!(shared A))();
writeln(typeof(got).stringof);
});
auto producer = runTask({
auto a = new shared A(1);
shared b = shared Env!(shared A)(a);
writeln(typeof(b).stringof);
send(consumer, b);
});
runApplication();
}
Re: Implementing interface in the class hierarchy
On Friday, 14 July 2017 at 12:31:49 UTC, Steven Schveighoffer wrote: Relevant enhancement request: https://issues.dlang.org/show_bug.cgi?id=2565 -Steve So it looks like there are no rational arguments for such a language specification, and this behavior is derived from some aspect of the compiler implementation. Thanks Arek
Implementing interface in the class hierarchy
According to language reference (part 'Interfaces') this code
will not compile:
interface D
{
int foo();
}
class A : D
{
int foo() { return 1; }
}
class B : A, D <- Error: class B interface function 'foo' is not
implemented
{
}
Because: 'A reimplemented interface must implement all the
interface functions, it does not inherit them from a super class'.
Why?
Each B object 'is an' A object (and each cat 'is an' animal) so
if A implements D, then B implements D too. Implementing D second
time doesn't change the nature of A and B.
More over, another example (more practical because here, the D
interface is going to be implemented only once):
interface D
{
int foo();
}
class A
{
int foo() { return 1; }
}
class B : A, D <- Error: class B interface function 'foo' is not
implemented
{
}
Class A doesn't implement D, but it has the method satisfied the
D interface.
Why I have to provide the explicit implementation of 'foo' in B
class?
I cannot logically explain this property of Dlang's OOP. Anyone
could?
Thanks in advance.
Arek
Re: Static array size?
On Thursday, 9 February 2017 at 11:22:28 UTC, Suliman wrote: Docs says that: "The total size of a static array cannot exceed 16Mb." But when I am creation array of: int [1000_000] x; // 1000_000 is equal ~ 0,95MB app crush on start. Should it's reserve this memory with guaranty? I mean that after app start it should take +0.95MB of RAM in task manager. First of all, as others said it is 4 bytes * 100, what gives about 4MB. Do you allocate this array on the stack? (local variables are created on the stack) If yes, maybe you've exceeded the size of the stack. Declare the array out of the scope of any function and check again. The size of the stack can be set in the linker (http://forum.dlang.org/post/mailman.766.1291253609.21107.digitalmar...@puremagic.com).
