Re: I'm porting some go code to D
On Aug 24, 2013, at 1:59 AM, David wrote: > Daemonic Threads often end with a segfault, so if your main thread > exists, the other threads will probably segfault. By default, sure. But with daemon threads you really want to have some kind of shutdown mechanism inside a static dtor somewhere. The goal is more to have threads that don't implicitly block app shutdown.
Re: I'm porting some go code to D
I've made the scheduler a bit more inteligent and the channel
implementation is now backed by a lock free queue.
https://github.com/rjmcguire/goport/blob/tip/wrapper2.d for a example using
libev
https://github.com/rjmcguire/goport/blob/tip/goroutine.d for the "go"
routines
https://github.com/rjmcguire/goport/blob/tip/channel.d channel
implementation, the channel still has a read lock because I didn't like
looping wasting cpu cycles/battery, I may change this to use the same loop
as the select(...) expression with a timeout that backs off.
https://github.com/rjmcguire/goport/blob/tip/concurrentlinkedqueue.d queue
implementation
there is a problem with the file EV handler in that it can get spurios READ
events and I can't seem to set it to non-blocking.
The select(...) implementation and usage is quite interesting because it
works very much like Go's select statement.
On Mon, Aug 26, 2013 at 1:28 AM, Rory McGuire wrote:
> Awesome thanks, thought what I did there was dodgy. It was really weird
> when this() started multiple schedulers at least now I see the obvious
> reason.
>
> BTW: gist is at: https://gist.github.com/rjmcguire/6336931
>
> Could someone point me at the correct linked list to use inside the
> channel. I'd prefer to use a range container of some kind if it exists in
> the std lib. I tried SList and had a bad experience hence the custom
> implementation.
>
>
>
> On Mon, Aug 26, 2013 at 1:21 AM, Timon Gehr wrote:
>
>> On 08/26/2013 12:55 AM, Rory McGuire wrote:
>>
>>>
>>> shared chan!Fiber scheduler; // channel contains Fibers waiting for
>>> their time slice
>>> static this () {
>>> if (scheduler is null) {
>>>
>>
>>
>> You want 'shared static this' instead to avoid a race condition.
>>
>
>
Re: I'm porting some go code to D
Awesome thanks, thought what I did there was dodgy. It was really weird
when this() started multiple schedulers at least now I see the obvious
reason.
BTW: gist is at: https://gist.github.com/rjmcguire/6336931
Could someone point me at the correct linked list to use inside the
channel. I'd prefer to use a range container of some kind if it exists in
the std lib. I tried SList and had a bad experience hence the custom
implementation.
On Mon, Aug 26, 2013 at 1:21 AM, Timon Gehr wrote:
> On 08/26/2013 12:55 AM, Rory McGuire wrote:
>
>>
>> shared chan!Fiber scheduler; // channel contains Fibers waiting for
>> their time slice
>> static this () {
>> if (scheduler is null) {
>>
>
>
> You want 'shared static this' instead to avoid a race condition.
>
Re: I'm porting some go code to D
On 08/26/2013 12:55 AM, Rory McGuire wrote:
shared chan!Fiber scheduler; // channel contains Fibers waiting for
their time slice
static this () {
if (scheduler is null) {
You want 'shared static this' instead to avoid a race condition.
Re: I'm porting some go code to D
On Sat, Aug 24, 2013 at 3:26 PM, bearophile wrote:
> David Nadlinger:
>
>
> It's a cute idea, but not really practical, I'm afraid – Goroutines are
>> managed by a scheduler in the Go runtime library, whereas D threads
>> directly map to OS threads.
>>
>
> Can't Rory McGuire add a scheduler to his code? How much code does it take?
>
> Bye,
> bearophile
>
Here is a basic co-operative scheduler based version of go!(F) that uses a
channel to keep things simple, its co-op so Fiber.yield has to be called at
some point. If the channel detects its in a Fiber it calls yield if there
is nothing available in it:
void go(alias F)() {
scheduler._ = new Fiber(F);
}
shared chan!Fiber scheduler; // channel contains Fibers waiting for their
time slice
static this () {
if (scheduler is null) {
scheduler = makeChan!Fiber(100);
// create the workers
auto goprocs = environment.get("GOPROCS");
int num_threads = 1;
if (goprocs != null) {
num_threads = to!int(goprocs);
}
foreach (i; 0..num_threads) {
// create threads that process the live fibers
auto t = new Thread(() {
for (;;) {
auto fiber = scheduler._;
fiber.call();
if (fiber.state != Fiber.State.TERM) {
scheduler._ (fiber);
}
}
});
t.start();
}
}
}
Just need to figure out a way to detect when main() has exited.
Re: I'm porting some go code to D
On 24 Aug 2013 23:55, "Paulo Pinto" wrote: > > Funny, I always thought otherwise, because Windows only has threads. > > Processes are just a means of grouping threads on Windows, as there > isn't the distinction between threads and processes that UNIX systems used to make. > > Then again, I lost track how the performance on Linux systems changed across the whole Processes -> LinuxThreads -> NPTL -> Posix Threads evolution. > > -- > Paulo Just did a quick search to see if I was right. I'm not sure if it's still the case but it appears that is actually the scheduler that causes windows to be much slower than Linux. Ideally one would want a hybrid of threads and fibres anyway. I wonder how much of the standard library would need to change.
Re: I'm porting some go code to D
On Saturday, 24 August 2013 at 20:03:58 UTC, Rory McGuire wrote: On Sat, Aug 24, 2013 at 3:26 PM, bearophile wrote: David Nadlinger: It's a cute idea, but not really practical, I'm afraid – Goroutines are managed by a scheduler in the Go runtime library, whereas D threads directly map to OS threads. Can't Rory McGuire add a scheduler to his code? How much code does it take? Bye, bearophile I imagine that it will be fine on Linux because threads truly are lightweight on Linux, but its not going to be great on windows. Funny, I always thought otherwise, because Windows only has threads. Processes are just a means of grouping threads on Windows, as there isn't the distinction between threads and processes that UNIX systems used to make. Then again, I lost track how the performance on Linux systems changed across the whole Processes -> LinuxThreads -> NPTL -> Posix Threads evolution. -- Paulo
Re: I'm porting some go code to D
On Sat, Aug 24, 2013 at 3:26 PM, bearophile wrote: > David Nadlinger: > > > It's a cute idea, but not really practical, I'm afraid – Goroutines are >> managed by a scheduler in the Go runtime library, whereas D threads >> directly map to OS threads. >> > > Can't Rory McGuire add a scheduler to his code? How much code does it take? > > Bye, > bearophile > I imagine that it will be fine on Linux because threads truly are lightweight on Linux, but its not going to be great on windows. Go's scheduler is basically the same as the way vibe.d works. Fibers. I haven't tried it but I could use fibers to do the same sort of thing, though it would be easiest to use in Vibe.d because even in go you have to do something that will wait/sleep in order to have your program advance unless you set GOPROCS to a value greater than 1. Go's scheduler is not preemptive so there are the equivalent of Fiber.yield() spread throughout the standard library, I think. I'm not sure how threads + fibers would have to interact in such a system.
Re: I'm porting some go code to D
On Saturday, 24 August 2013 at 13:26:32 UTC, bearophile wrote: David Nadlinger: It's a cute idea, but not really practical, I'm afraid – Goroutines are managed by a scheduler in the Go runtime library, whereas D threads directly map to OS threads. Can't Rory McGuire add a scheduler to his code? How much code does it take? It would be very nice to have a builtin scheduler in the runtime. It would make async non-threaded programming so much nicer in D. /Jonas
Re: I'm porting some go code to D
On 24 Aug 2013 11:25, "Moritz Maxeiner" wrote:
>
> On Friday, 23 August 2013 at 23:54:55 UTC, Rory McGuire wrote:
>>
>> So I'm porting so #golang code to #dlang and there is all these blasted
>> "go" statements.So I thought I'd give implmenting it in D a shot. What do
>> you guys think?
>> Fire away :).
>>
>> /**
>> * chan allows messaging between threads without having to deal with
locks,
>> similar to how chan works in golang
>> */
>> class chan_(T) {
>> shared Mutex lock;
>> struct Container(T) {
>> T value;
>> Container!T* next;
>
>
> I'm probably missunderstanding somehting about the TLS model, but from
what I know, for something like this, shouldn't you make the class instance
itself go into shared storage instead instead of all the members?
>
I have no idea if shared on the class makes all it's parts shared.
It was a struct but I had problems with passing it to the spawn func when I
was using std.concurrent .
I'm trying to port a cassandra cql library from go so it's really just to
help with that.
I'm having a hard time imagining how to implement select from go. Could
probably use std.concurrent.receive because that can handle multiple types
at once.
Sorry I'm rambling...
Re: I'm porting some go code to D
David Nadlinger: It's a cute idea, but not really practical, I'm afraid – Goroutines are managed by a scheduler in the Go runtime library, whereas D threads directly map to OS threads. Can't Rory McGuire add a scheduler to his code? How much code does it take? Bye, bearophile
Re: I'm porting some go code to D
On Friday, 23 August 2013 at 23:54:55 UTC, Rory McGuire wrote: So I'm porting so #golang code to #dlang and there is all these blasted "go" statements.So I thought I'd give implmenting it in D a shot. What do you guys think? It's a cute idea, but not really practical, I'm afraid – Goroutines are managed by a scheduler in the Go runtime library, whereas D threads directly map to OS threads. Thus, if the Go application you are porting uses many Goroutines (and the Go code I've seen usually does so very liberally), the performance of the D equivalent is going to be horrible. David
Re: I'm porting some go code to D
On Friday, 23 August 2013 at 23:54:55 UTC, Rory McGuire wrote:
So I'm porting so #golang code to #dlang and there is all these
blasted
"go" statements.So I thought I'd give implmenting it in D a
shot. What do
you guys think?
Fire away :).
/**
* chan allows messaging between threads without having to deal
with locks,
similar to how chan works in golang
*/
class chan_(T) {
shared Mutex lock;
struct Container(T) {
T value;
Container!T* next;
I'm probably missunderstanding somehting about the TLS model, but
from what I know, for something like this, shouldn't you make the
class instance itself go into shared storage instead instead of
all the members?
Re: I'm porting some go code to D
On 24 Aug 2013 11:00, "David" wrote: > > Daemonic Threads often end with a segfault, so if your main thread > exists, the other threads will probably segfault. Thanks, I wonder what they're accessing that they shouldn't.
Re: I'm porting some go code to D
Daemonic Threads often end with a segfault, so if your main thread exists, the other threads will probably segfault.
Re: I'm porting some go code to D
On 24 Aug 2013 03:15, "Jesse Phillips" wrote: > > On Friday, 23 August 2013 at 23:54:55 UTC, Rory McGuire wrote: >> >> So I'm porting so #golang code to #dlang and there is all these blasted >> "go" statements.So I thought I'd give implmenting it in D a shot. What do >> you guys think? >> Fire away :). > > > I'd suggest posting long snippets of code to https://gist.github.com/ or http://dpaste.dzfl.pl/ > > A couple lines is fine but whole implementations, definitely want a more color friendly solution. Good point.
Re: I'm porting some go code to D
On Friday, 23 August 2013 at 23:54:55 UTC, Rory McGuire wrote: So I'm porting so #golang code to #dlang and there is all these blasted "go" statements.So I thought I'd give implmenting it in D a shot. What do you guys think? Fire away :). I'd suggest posting long snippets of code to https://gist.github.com/ or http://dpaste.dzfl.pl/ A couple lines is fine but whole implementations, definitely want a more color friendly solution.
I'm porting some go code to D
So I'm porting so #golang code to #dlang and there is all these blasted
"go" statements.So I thought I'd give implmenting it in D a shot. What do
you guys think?
Fire away :).
import std.stdio;
import core.thread : Thread;
import core.sync.mutex : Mutex;
import core.time;
void main() {
auto ch = chan!int(1);
go!({
foreach (i; 22..44) {
ch._ = i;
} });
foreach (i; 0..10) {
writeln("pop: ", ch._);
}
}
// Implementation
void go(alias F)() {
auto t = new Thread(F);
t.isDaemon(true); // we don't care if this thread dies.
t.start();
}
/**
* chan allows messaging between threads without having to deal with locks,
similar to how chan works in golang
*/
class chan_(T) {
shared Mutex lock;
struct Container(T) {
T value;
Container!T* next;
}
shared Container!T* buf;
shared Container!T* last;
shared size_t length;
shared void insert(shared T v) {
shared Container!T* newItem = new shared Container!T();
newItem.value = v;
if (buf is null) {
buf = newItem;
last = newItem;
} else {
last.next = newItem;
last = newItem;
}
length++;
}
shared T popFront() {
T ret;
synchronized (lock) {
ret = buf.value;
buf = buf.next;
length--;
}
return ret;
}
size_t maxItems;
bool blockOnFull = false;
this(int maxItems = 1024, bool blockOnFull = true) {
lock = cast(shared)new Mutex;
length = 0;
this.maxItems = maxItems;
this.blockOnFull = blockOnFull;
}
@property
shared void _(T value) {
bool done;
while(true) {
synchronized(lock) {
if (!done && length < maxItems) {
insert(value);
done = true;
} else if (!blockOnFull) {
throw new ChannelFull("Channel Full");
}
if (length <= maxItems-1) {
break;
}
}
Thread.sleep(dur!"msecs"(5));
}
}
@property
shared T _() {
while(true) {
size_t len;
synchronized(lock) {
len = length;
}
if (len > 0) {
break;
}
Thread.sleep(dur!"msecs"(5));
};
auto r = popFront();
return r;
}
}
auto chan(T)(int n, bool blockOnFull = true) {
return cast(shared)new chan_!T(n, blockOnFull);
}
class ChannelFull : Exception {
this(string msg, string file = __FILE__, ulong line =
cast(ulong)__LINE__, Throwable next = null) {
super(msg,file,line,next);
}
}
