Re: How to recursively accept data from Python server ?
On Friday, 25 June 2021 at 03:27:24 UTC, jfondren wrote: On Friday, 25 June 2021 at 02:55:50 UTC, Utk wrote: Please help me to resolve this issue. Try stracing your program to see exactly what it's doing with the socket, and try std.socket's lastSocketError I tried using ```lastSocketError()```, it gave an error saying *Connection reset by peer*. I'm confused about how the connection is getting reset. Here's the complete code: https://github.com/utkarshb1/Interactive-Register-Debugging/blob/7919612396cf893bb5e9b936c2d8a44663ac6733/apb_slave/testbench/apb.d#L289 I'm receiving that error at line 362. The code should continuously wait for the data from python and once received it should do the computation defined in the code and then again wait for the data until EXIT signal is sent from python.
Re: How to recursively accept data from Python server ?
On Friday, 25 June 2021 at 02:55:50 UTC, Utk wrote: Please help me to resolve this issue. Try stracing your program to see exactly what it's doing with the socket, and try std.socket's lastSocketError
Re: How to recursively accept data from Python server ?
Not sure if you want build interprocess communication utility all by yourself, I'm currently using https://code.dlang.org/packages/apache-thrift and there is also https://code.dlang.org/packages/symmetry_thrift_d BTW grpc is not quiet working https://code.dlang.org/packages/grpc
How to recursively accept data from Python server ?
Hello!
I'm trying to establish a connection between a python server and
a d client. The data is been transferred properly only once and
then the d client doesn't wait for any further data and returns
an error ```Socket.ERROR``` and exists and doesn't wait for
further incoming data from the Python server. From Python, the
data is been sent whenever the user sends a request but instead
of waiting for incoming requests the D client just accepts the
data 1st time and then throws an error and exits.
The code snippet is as below:
```
bool get_req_from_socket(Socket socket) {
ubyte[] rawTrHeader;
ubyte[] buffer;
writeln("Waiting for data from socket");
while (rawTrHeader.length < 16) {
import std.string: format;
buffer.length = 16 - rawTrHeader.length;
auto ret = socket.receive(buffer);
uvm_info("SOCKET", format("Received %d bytes from socket",
ret),
UVM_DEBUG);
if (ret == Socket.ERROR)
uvm_fatal("SOCKET", "Error receiving data from socket");
if (ret == 0)
uvm_fatal("SOCKET", "Got insufficient bytes for TrHeader");
// now slice the buffer to get the part...
rawTrHeader ~= buffer[0 .. ret];
}
...
...
}
```
This is the function (infinitely called while it returns
```true```) that accepts the data and returns ```true``` or
```false``` depending on if data is received or not respectively.
The error I'm receiving is:
```
Waiting for data from socket
UVM_INFO ../testbench/apb.d(375) @ 13: reporter@@apb_seq
[SOCKET] Received -1 bytes from socket
UVM_FATAL ../testbench/apb.d(378) @ 13: reporter@@apb_seq
[SOCKET] Error receiving data from socket
```
Please help me to resolve this issue.
Re: How to call stop from parallel foreach
On 6/24/21 1:41 PM, seany wrote:
> Is there any way to control the number of CPU cores used in
> parallelization ?
Yes. You have to create a task pool explicitly:
import std.parallelism;
void main() {
enum threadCount = 2;
auto myTaskPool = new TaskPool(threadCount);
scope (exit) {
myTaskPool.finish();
}
enum workUnitSize = 1; // Or 42 or something else. :)
foreach (e; myTaskPool.parallel([ 1, 2, 3 ], workUnitSize)) {
// ...
}
}
I've touched on a few parallelism concepts at this point in a presentation:
https://www.youtube.com/watch?v=dRORNQIB2wA=1332s
Ali
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 20:56:26 UTC, Ali Çehreli wrote: On 6/24/21 1:33 PM, Bastiaan Veelo wrote: > distributes the load across all cores (but one). Last time I checked, the current thread would run tasks as well. Ali Indeed, thanks. — Bastiaan.
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 21:05:28 UTC, Bastiaan Veelo wrote: On Thursday, 24 June 2021 at 20:41:40 UTC, seany wrote: Is there any way to control the number of CPU cores used in parallelization ? E.g : take 3 cores for the first parallel foreach - and then for the second one, take 3 cores each -> so 3 + 3 * 3 = 12 cores out of a 16 core system? Thank you. There might be, by using various `TaskPool`s with a smaller number of work threads: https://dlang.org/phobos/std_parallelism.html#.TaskPool.this.2. But I cannot see the benefit of doing this. It will just distribute the same amount of work in a different way. Actually, I think this would be suboptimal as well, as the three outer threads seem to do no real work. — Bastiaan.
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 20:41:40 UTC, seany wrote:
On Thursday, 24 June 2021 at 20:33:00 UTC, Bastiaan Veelo wrote:
By the way, nesting parallel `foreach` does not make much
sense, as one level already distributes the load across all
cores (but one). Additional parallelisation will likely just
add overhead, and have a net negative effect.
— Bastiaan.
Okey. So consider :
foreach(array_elem; parallel(an_array)) {
dothing(array_elem);
}
and then in `dothing()` :
foreach(subelem; array_elem) {
dootherthing(subelem);
}
- Will this ALSO cause the same overhead?
You can nest multiple `foreach`, but only parallelise one like so:
```d
foreach(array_elem; parallel(an_array))
foreach(subelem; array_elem)
dootherthing(subelem);
```
So there is no need to hide one of them in a function.
Is there any way to control the number of CPU cores used in
parallelization ?
E.g : take 3 cores for the first parallel foreach - and then
for the second one, take 3 cores each -> so 3 + 3 * 3 = 12
cores out of a 16 core system? Thank you.
There might be, by using various `TaskPool`s with a smaller
number of work threads:
https://dlang.org/phobos/std_parallelism.html#.TaskPool.this.2.
But I cannot see the benefit of doing this. It will just
distribute the same amount of work in a different way.
— Bastiaan.
Re: How to call stop from parallel foreach
On 6/24/21 1:33 PM, Bastiaan Veelo wrote: > distributes the load across all cores (but one). Last time I checked, the current thread would run tasks as well. Ali
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 20:33:00 UTC, Bastiaan Veelo wrote:
By the way, nesting parallel `foreach` does not make much
sense, as one level already distributes the load across all
cores (but one). Additional parallelisation will likely just
add overhead, and have a net negative effect.
— Bastiaan.
Okey. So consider :
foreach(array_elem; parallel(an_array)) {
dothing(array_elem);
}
and then in `dothing()` :
foreach(subelem; array_elem) {
dootherthing(subelem);
}
- Will this ALSO cause the same overhead?
Is there any way to control the number of CPU cores used in
parallelization ?
E.g : take 3 cores for the first parallel foreach - and then for
the second one, take 3 cores each -> so 3 + 3 * 3 = 12 cores out
of a 16 core system? Thank you.
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 18:23:01 UTC, seany wrote:
I have seen
[this](https://forum.dlang.org/thread/akhbvvjgeaspmjntz...@forum.dlang.org).
I can't call break form parallel foreach.
Okey, Is there a way to easily call .stop() from such a case?
Yes there is, but it won’t break the `foreach`:
```d
auto tp = taskPool;
foreach (i, ref e; tp.parallel(a))
{
// …
tp.stop;
}
```
The reason this does not work is because `stop` terminates the
worker threads as soon as they are finished with their current
`Task`, but no sooner. `parallel` creates the `Task`s before it
presents a range to `foreach`, so no new `Task`s are created
during iteration. Therefore all elements are iterated.
outer: foreach(i, a; parallel(array_of_a)) {
foreach(j, b; parallel(array_of_b)) {
By the way, nesting parallel `foreach` does not make much sense,
as one level already distributes the load across all cores (but
one). Additional parallelisation will likely just add overhead,
and have a net negative effect.
— Bastiaan.
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 20:08:06 UTC, seany wrote:
On Thursday, 24 June 2021 at 19:46:52 UTC, Jerry wrote:
On Thursday, 24 June 2021 at 18:23:01 UTC, seany wrote:
[...]
Maybe I'm wrong here, but I don't think there is any way to do
that with parallel.
What I would do is negate someConditionCheck and instead only
do work when there is work to be done.
Obviously that may or may not be suitable.
But with parallel I don't see any way to make it happen.
The parallel() function is running a taskpool. I should be able
to stop that in any case...
PS :
I have done this :
parallelContainer: while(true) {
outer: foreach(i, a; parallel(array_of_a)) {
foreach(j, b; parallel(array_of_b)) {
auto c = myFunction0(i,j);
auto d = myFunction1(a,b);
auto f = myFunction2(i,b);
auto g = myFunction3(a,j);
if(someConditionCheck(c,d,f,g)) {
break parallelContainer;
}
}
break;
}
Is this safe? Will this cause any problem that I can't foresee
now? Thank you
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 19:46:52 UTC, Jerry wrote: On Thursday, 24 June 2021 at 18:23:01 UTC, seany wrote: [...] Maybe I'm wrong here, but I don't think there is any way to do that with parallel. What I would do is negate someConditionCheck and instead only do work when there is work to be done. Obviously that may or may not be suitable. But with parallel I don't see any way to make it happen. The parallel() function is running a taskpool. I should be able to stop that in any case...
Re: How to call stop from parallel foreach
On Thursday, 24 June 2021 at 18:23:01 UTC, seany wrote:
I have seen
[this](https://forum.dlang.org/thread/akhbvvjgeaspmjntz...@forum.dlang.org).
I can't call break form parallel foreach.
Okey, Is there a way to easily call .stop() from such a case?
Here is a case to consider:
outer: foreach(i, a; parallel(array_of_a)) {
foreach(j, b; parallel(array_of_b)) {
auto c = myFunction0(i,j);
auto d = myFunction1(a,b);
auto f = myFunction2(i,b);
auto g = myFunction3(a,j);
if(someConditionCheck(c,d,f,g)) {
// stop the outer foreach loop here
}
}
}
Thank you
Maybe I'm wrong here, but I don't think there is any way to do
that with parallel.
What I would do is negate someConditionCheck and instead only do
work when there is work to be done.
Obviously that may or may not be suitable.
But with parallel I don't see any way to make it happen.
How to call stop from parallel foreach
I have seen
[this](https://forum.dlang.org/thread/akhbvvjgeaspmjntz...@forum.dlang.org).
I can't call break form parallel foreach.
Okey, Is there a way to easily call .stop() from such a case?
Here is a case to consider:
outer: foreach(i, a; parallel(array_of_a)) {
foreach(j, b; parallel(array_of_b)) {
auto c = myFunction0(i,j);
auto d = myFunction1(a,b);
auto f = myFunction2(i,b);
auto g = myFunction3(a,j);
if(someConditionCheck(c,d,f,g)) {
// stop the outer foreach loop here
}
}
}
Thank you
Re: Are D classes proper reference types?
On Thursday, 24 June 2021 at 12:31:08 UTC, Stefan Koch wrote:
On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:
On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad
wrote:
[...]
(I don't think I've ever seen one); with `scope c = new
Object`, you can have the compiler allocate a class *instance*
on the stack for you, but `c` is still a *ref*.
The dmd frontend uses them all the time to avoid allocation
overhead for Visitors.
I was talking about not having seen a `scope class C { ... }`,
*not* the `scope` storage class as in the example.
Re: is it possible to sort a float range ?
On Thursday, 24 June 2021 at 05:39:22 UTC, jfondren wrote: - ok, it's useful that it's like this Albeit (the grumbling and) the weirdness of it, this is exactly the reason why I am not complaining on such behavior -time will show me.
Re: How to I get pointer to an Array and cast to a void * and back ?
On Thursday, 24 June 2021 at 14:06:11 UTC, seany wrote:
void f() {
a[] * rd;
// DO SOME WORK HERE
this.dataSet = & rd_flattened;
rd = cast (a [] *) dataSet;
write("length of rd is : "); writeln((*rd).length); // <---
this works..
// do some work on rd
this.dataSet = rd;
rd = cast (field.rawData [] *) dataSet;
write("length of rd for a second time is : ");
writeln((*rd).length); // <--- this ALSO works..
}
Now outside `f`, in the same class, i call :
void f2() {
f();
a[] *aa ;
aa = cast (a [] *) this.dataSet; // recall dataset is
public global
// if i print the address of this.dataSet here, this is the
same as inside f()
write("after calling f, count is: ");
writeln((*aa).length); readln();
// here the situation completely blows up . the length is
wrong.
}
What is causing this issue ?
Your variable `a[] rd_flattened;` is a local variable to function
`f()` allocated on the stack. Stack memory expires as soon as you
return from the function. What `f2()` accesses through your
global variable is a dangling pointer, a pointer to the expired
stackframe of `f()`, which is why the `.length` is garbage.
How to I get pointer to an Array and cast to a void * and back ?
I have a struct :
struct a {
int i;
// some more stuff ...
}
In a Class, I define public global `void * dataSet; `
In a function `f`, of the same class: i call :
void f() {
a[] rd_flattened;
a[] * rd;
// DO SOME WORK HERE
this.dataSet = & rd_flattened;
rd = cast (a [] *) dataSet;
write("length of rd is : "); writeln((*rd).length); // <---
this works..
// do some work on rd
this.dataSet = rd;
rd = cast (field.rawData [] *) dataSet;
write("length of rd for a second time is : ");
writeln((*rd).length); // <--- this ALSO works..
}
Now outside `f`, in the same class, i call :
void f2() {
f();
a[] *aa ;
aa = cast (a [] *) this.dataSet; // recall dataset is public
global
// if i print the address of this.dataSet here, this is the
same as inside f()
write("after calling f, count is: "); writeln((*aa).length);
readln();
// here the situation completely blows up . the length is
wrong.
}
I need some help here. Sorry, can't post code. It is proprietary.
What is causing this issue ?
thank you.
Re: Are D classes proper reference types?
On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote: On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad wrote: [...] (I don't think I've ever seen one); with `scope c = new Object`, you can have the compiler allocate a class *instance* on the stack for you, but `c` is still a *ref*. The dmd frontend uses them all the time to avoid allocation overhead for Visitors.
Re: Are D classes proper reference types?
On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote: *scope* classes are deprecated (I don't think I've ever seen one); I used it for my database thing where it is supposed to be destroyed reliably but also uses runtime polymorphism. I now suggest people just stick `scope(exit) .destroy(obj);` any time you use it.
Re: Are D classes proper reference types?
On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad wrote: [...] Yes, class *refs* are always pointers. *scope* classes are deprecated (I don't think I've ever seen one); with `scope c = new Object`, you can have the compiler allocate a class *instance* on the stack for you, but `c` is still a *ref*. `emplace` doesn't allocate, you have to pass the memory explicitly. A class *instance* can also live in the static data segment (`static immutable myStaticObject = new Object;`); `extern(C++)` class instances can also live on the C++ heap/stack etc. etc.
Are D classes proper reference types?
D classes are distinct from structs because they are intended to be bound to a reference (pointer) and not addressed as a value (inline/copying). But for efficiency reasons, scoped classes can stack-allocate, but my understanding is that the compiler can still allocate it on the GC heap? So it is still a reference type, I guess? But how about "emplace", does "emplace" imply the the compiler cannot put it on the GC heap? Yes, I am aware that this is library construct, but still relevant. The reason I am asking this is because implementing reference counting in a clean fashion requires classes to be proper reference types, so that means disabling scoped classes and emplaced classes, and only allow it where a reference/pointer to a class never exceed a reference count of 1. The core of my question is this: is it at all reasonable for "user code" to assume that a class is not allocated on the heap? Is it reasonable to write code that assumes that it is allocated "inline" as with emplace? Because if it is, then then the advantages of having classes as reference types and making them different from structs are lost.
