Re: SumType extraction
On Thursday, 27 June 2024 at 18:51:19 UTC, Josh Holtrop wrote:
Questions:
4. Any other general improvements to my solution?
I know it's kind of an unpopular choice these days but one could
go
with inheritance and polymorphism or instanceof tests. something
along the lines of
```d
import std.stdio : writeln;
class Item
{
public void operationA()
{
}
public void operationB()
{
}
}
class ItemA : Item
{
override public void operationA()
{
writeln("ItemA");
}
}
class ItemB : Item
{
override public void operationB()
{
writeln("ItemB");
}
}
void main(string[] args)
{
auto items = [new ItemA(), new ItemB()];
writeln("operation a:");
foreach (item; items)
{
item.operationA();
}
writeln("operation b:");
foreach (item; items)
{
item.operationB();
}
writeln("instance of:");
foreach (item; items)
{
if (auto itemB = cast(ItemB) item)
{
writeln("Found an ItemB");
}
}
}
```
drawback might be, that if you add a new subtype the compiler will
not warn you that you did not implement one case for one of the
implementations.
Kind regards,
Christian
Re: Partial function application (Currying)
Would https://dlang.org/library/std/functional/curry.html help you? kind regards, Christian
Re: Synchronisation help
On Tuesday, 2 January 2024 at 10:41:55 UTC, Anonymouse wrote: On Monday, 1 January 2024 at 19:49:28 UTC, Jonathan M Davis wrote: [...] Thank you. Yes, `Foo` is a class for the purposes of inheritance -- I left that out of the example. So a completely valid solution is to write a struct wrapper around an AA of the type I need, overload the required operators, and then just drop-in replace the current AA? All array operations would then transparently be between lock and unlock statements. [...] I guess this can break synchronisation between the two if I replace the `Mutex` in either thread. Are there any other obvious caveats? It might be, that this locking scheme is too narrow. E.g. you might want to have an "atomic" testAndSet on the AA e.g. check if an element is in and iff its not in put it there. Kind regards, Christian
Re: How to implement filterMap
On Saturday, 30 December 2023 at 18:08:55 UTC, Alexandru Ermicioi wrote: On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote: Is there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range). Perhaps try map to null cases that exception is thrown then filter out those, and then unwrap from nullable type: ```d myRange.map!(e => e.to!T.asNullForException).filter!(e => !e.isNull).map!(e => e.get).myOtherAlgorithm() ``` where asNullForException will return empty `Nullable` in case conversion fails. asNullForException could be nicely implemented with lazy parameters i guess. then its back to the map/filter/map chain that motivated filter_map in rust ... thanks for this idea... kind regards, Christian
Re: How to implement filterMap
On Saturday, 30 December 2023 at 01:22:31 UTC, Siarhei Siamashka wrote: On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote: Is there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range). It's probably not a good idea to do this in general. Expecting a lot of exceptions handling happening during normal program execution (if you want to filter out roughly half of the input array) will result in a major performance loss. Exceptions are best left to just do error handling on a very rarely used code path for troubleshooting purposes. Thanks for the feedback. This might be true, but in my example I would parse the input always with conv.to, so I would need to handle the exception(s). The "original" https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map works with the Option(Some/None) ... Kind regards, Christian
How to implement filterMap
Is there a way to implement filterMap (meaning do mapping of a
range, but if something happens during the map, leave this
element out of the resulting range).
I have two solutions (one is with evaluating the mapping function
several times), and one tries to store the result for the next
front call like this.
Both do not seem very clean for all possible types. Here the
version that tries to cache the mapping results.
```d
template filterMap(mapping...) if (mapping.length == 1)
{
auto filterMap(Range)(Range range)
{
import std.range : ElementType, empty;
import std.functional : unaryFun;
alias RangeElement = ElementType!Range;
alias mappingFunction = unaryFun!mapping;
typeof(mappingFunction(RangeElement.init)) result;
void findNext() {
while (!range.empty)
{
try
{
result = mappingFunction(range.front);
break;
}
catch (Exception e)
{
range.popFront;
}
}
}
findNext();
struct FilterMap
{
bool empty()
{
return range.empty;
}
auto ref front()
{
return result;
}
void popFront()
{
range.popFront;
findNext();
}
}
return FilterMap();
}
}
@("filterMap") unittest {
import std.conv : to;
import std.array : array;
["1", "2", "abc", ""].filterMap!(s => s.to!int *
2).array.should == [2, 4];
}
```
Kind regards,
Christian
Re: Non-blocking keyboard input
On Wednesday, 27 December 2023 at 14:41:05 UTC, Christian Köstlin wrote: One option (not tested) should be to close stdin so that readln then returns null or something on eof. Shutting down threads is always tricky. It would be great if there would be one or two (perhaps one synchronous, one asynchronous) main io-frameworks for dlang (also as dub packages) that cover the most common use-cases. Kind regards, Christian I tested this now, but it still blocks in readln ... Kind regards, Christian
Re: Non-blocking keyboard input
On Wednesday, 27 December 2023 at 05:07:04 UTC, Joe wrote: ??? Surely there there is a one liner library solution for this? I have a program that spawns a thread for debugging information and uses the keyboard input which allows me to display the information. If I use getchar or readline then it blocks the thread. This is generally fine because that is all the thread does. The problem is that it also blocks the program termination as the main thread will not exit while the thread is running which is is because it's waiting on keyboard input stuck on getchar or fgetc or whatever. If I terminate the threads using thread_term then it terminates the program but the program then does not return the return code that it was successfully finished(because it was prematurely terminated by thread_term. Surely there is some type of peek for keyboards in D? I can't seem to get kbhit or because the library is not included(well, I tried to use one from dmc but it seemed to be invalid. snn.lib IIRC). This shouldn't be hard... yet it is. One option (not tested) should be to close stdin so that readln then returns null or something on eof. Shutting down threads is always tricky. It would be great if there would be one or two (perhaps one synchronous, one asynchronous) main io-frameworks for dlang (also as dub packages) that cover the most common use-cases. Kind regards, Christian
Re: Behaves different on my osx and linux machines
On Wednesday, 27 December 2023 at 14:03:06 UTC, Kagamin wrote: Maybe you're not supposed to print text while reading? In parallel I have contacted schveiguy on discord and he found the culprid. But we do not have a solution yet. It probably will result in a bugreport at https://issues.dlang.org/. Kind regards, Christian
Re: Behaves different on my osx and linux machines
On Friday, 22 December 2023 at 15:02:42 UTC, Kagamin wrote:
Add more debugging?
```
bool done = false;
while (!done) {
writeln(1);
auto result = ["echo", "Hello World"].execute;
if (result.status != 0)
{
writeln(2);
throw new Exception("echo failed");
}
writeln(result.output);
receiveTimeout(dur!"msecs"(-1),
(LinkTerminated t) {
writeln("Done");
done = true;
},
);
writeln(3);
}
writeln("ByeBye");
```
Thanks for the suggestion. But it still behaves the same:
Output:
```
Running vibe-io
1
Please enter something:
asdf
You entered asdf
Please enter something:
Hello World
3
1
asdf
You entered asdf
Please enter something:
Hello World
3
1
qwer
You entered qwer
Please enter something:
Hello World
3
1
```
Behaves different on my osx and linux machines
I have this somehow reduced program that behaves differently on osx and
linux.
```d
void stdioMain()
{
import std.stdio : readln, writeln;
import std.concurrency : spawnLinked, receive, receiveTimeout,
LinkTerminated;
import std.variant : Variant;
import std.string : strip;
import std.process : execute;
import core.time : dur;
auto input = spawnLinked(() {
string getInput() {
writeln("Please enter something:");
return readln().strip();
}
string line = getInput();
while (line != "quit")
{
writeln("You entered ", line);
line = getInput();
}
});
bool done = false;
while (!done) {
auto result = ["echo", "Hello World"].execute;
if (result.status != 0)
{
throw new Exception("echo failed");
}
writeln(result.output);
receiveTimeout(dur!"msecs"(-1),
(LinkTerminated t) {
writeln("Done");
done = true;
},
);
}
writeln("ByeBye");
}
void main(string[] args)
{
stdioMain();
}
`
It should just read from stdin in one separate thread and on the main
thread it just executes one `echo` after the other (and tries to find
out, if the other thread finished).
In linux this behaves as expected, meaning it prints a lot of hello worlds.
In osx on the other hand it prints 'please enter something', waits for
my input, prints it, and outputs one hello world.
What did I do wrong?
Kind regards,
Christian
p.s.: i am using ldc-1.35.0 osx version is 13.6.
Re: How do I install a package globally?
On Saturday, 11 November 2023 at 23:28:18 UTC, Trevor wrote: On Saturday, 11 November 2023 at 07:12:21 UTC, Christian Köstlin wrote: On Saturday, 11 November 2023 at 01:50:54 UTC, Trevor wrote: I'm just getting in to D , coming from a C and Python background. I've had a play with DUB and adding packages to my project, but it seems like there should be a way to install packages so they can be used in any D program I compile without creating a whole package. For example in Python you can just go "pip install abc" and then any script can use abc. How does one install packages globally, and how can I write programs that use the third-party packages without being wrapped in a dub file? There is no way to install a dependency for all your d programs globally. In general you also should be careful about which dependencies you pull in and control them tightly. In general the approach taken by dub is slightly different. The dependencies you are using in all your programs are put to `$HOME/.dub/packages/...` and are in theory available for all your programs, but you still have to tell dub, that you want to use one of them. This is similar to python in that, pip puts your dependencies somewhere in your `$PYTHONHOME` or `$PYTHONPATH` (I am not 100% sure about that). You can tell dub (and dub then tells the compiler where to find the dependencies) in two way: - Create a complete dub project with dub.json/sdl and add the dependency there (e.g. with `dub add` or while creating the project or by adding them in your editor to the dub.json/sdl) - Create a self executing d file (that is run when you just call it like a script and compiled on the fly). See https://dub.pm/advanced_usage#single-file for details. Another way dub is able to "install" something is by using `dub fetch` to pull a package with a binary in it (e.g. `dub fetch dfmt`). You then can run this executable by `dub run` (e.g. `dub run dfmt -- ... here go the arguments to the dfmt tool). Kind regards, Christian Thanks for the detailed reply. I guess what I'd like to do is not create a DUB package for every little project I work on. It seems like most modern languages require a package/dependency manager though. Being able to install libraries globally would avoid this but I can how that can cause it's own set of issues. It doesn't seem efficient in terms of bandwidth and hard disk space to have a new copy of a library for each project that uses it? Did you try https://dub.pm/advanced_usage#single-file for your usecase. This one installs the dependencies once in `$HOME/.dub` and compiles them on the fly. - For small projects this is I think quite nice. - For medium projects I like how dub works at the moment with a `dub.sdl/json` in the projects folder and the shared cache of dependencies in `$HOME/.dub`. - For really big problems I would like to have a way to install the dependencies really locally to the project (just to have more control over them, e.g. like ruby's `bundle package` command works). Kind regards, Christian
Re: How do I install a package globally?
On Saturday, 11 November 2023 at 01:50:54 UTC, Trevor wrote: I'm just getting in to D , coming from a C and Python background. I've had a play with DUB and adding packages to my project, but it seems like there should be a way to install packages so they can be used in any D program I compile without creating a whole package. For example in Python you can just go "pip install abc" and then any script can use abc. How does one install packages globally, and how can I write programs that use the third-party packages without being wrapped in a dub file? There is no way to install a dependency for all your d programs globally. In general you also should be careful about which dependencies you pull in and control them tightly. In general the approach taken by dub is slightly different. The dependencies you are using in all your programs are put to `$HOME/.dub/packages/...` and are in theory available for all your programs, but you still have to tell dub, that you want to use one of them. This is similar to python in that, pip puts your dependencies somewhere in your `$PYTHONHOME` or `$PYTHONPATH` (I am not 100% sure about that). You can tell dub (and dub then tells the compiler where to find the dependencies) in two way: - Create a complete dub project with dub.json/sdl and add the dependency there (e.g. with `dub add` or while creating the project or by adding them in your editor to the dub.json/sdl) - Create a self executing d file (that is run when you just call it like a script and compiled on the fly). See https://dub.pm/advanced_usage#single-file for details. Another way dub is able to "install" something is by using `dub fetch` to pull a package with a binary in it (e.g. `dub fetch dfmt`). You then can run this executable by `dub run` (e.g. `dub run dfmt -- ... here go the arguments to the dfmt tool). Kind regards, Christian
Re: Weird bug in std.logger? Possible memory corruption
On Wednesday, 1 November 2023 at 14:15:55 UTC, matheus wrote:
On Tuesday, 31 October 2023 at 21:19:34 UTC, Arafel wrote:
...
Assigning the value to a variable works as expected:
```d
import std.logger : info;
void main() {
auto s = foo();
info(s);
}
auto foo() {
info("In foo");
return "Hello, world.";
}
```
...
Unless you do:
string s;
info(s=foo());
I think this is a bug, or at least very weird behavior.
Matheus.
It's really weird: https://run.dlang.io/is/fIBR2n
Re: Removing an element from a DList
On Tuesday, 17 October 2023 at 17:27:19 UTC, Joakim G. wrote:
For some reason I cannot remove an element from a DList. I
tried several range approaches but to no avail. I'm a noob.
In the end I did this:
```
private void removeFromWaitingQueue(uint jid) {
auto arr = waitingQueue[].array;
arr = arr.remove!(job => job.jid == jid);
waitingQueue.clear();
foreach(job; arr) {
waitingQueue.insertBack(job);
}
}
```
Any hints?
/J
```d
import std.container : DList;
import std.algorithm : find;
import std.range : take;
import std.stdio : writeln;
import std.range : take;
struct Job
{
int id;
}
int main(string[] args)
{
auto list = DList!Job(Job(1), Job(2), Job(3));
int idToDelete = 2;
auto toRemove = list[].find!(job => job.id ==
idToDelete).take(1);
if (!toRemove.empty)
list.linearRemove(toRemove);
foreach (i; list[])
{
writeln(i);
}
return 0;
}
```
works for me ... see https://run.dlang.io/is/OtX820 for a live
version.
Kind regards,
Christian
Re: Vibe.d download function, how to get callback when done or error?
On 24.09.23 12:01, j...@bloow.edu wrote: On Saturday, 23 September 2023 at 20:20:31 UTC, Christian Köstlin wrote: On 23.09.23 14:07, j...@bloow.edu wrote: I'm using download(url, filename) to download files in vibe.d. The issue is that I do not know when the download is finished or errors. There is a callback for the streaming side but not for the file download. A small test program shows, that if the function return normally the transfer was done (and the file saved). The function raises an exception if there is e,g, an http error status communicated. I am not sure what happens if the download is interrupted in the middle. I guess there will be an exception, but the file might be written partially. Kind regards, Christian I can't do any testing because overnight for no explicable reason the app will not compile any more. If it is synchronous then I should be able to do what I need without issue. When I initially was using it and the way the app works it seemed like it was running in parallel because I didn't see it execute synchronously because the tasks were relatively short and I just thought it was asynchronous for some reason(I didn't give it any thought at the time until I got errors in the download and needed to make the code more robust but I can't do anything until I figure out why the code no longer compiles(I made a post about it but it's not showing up yet). I recommend to use `https://httpbin.org/drip?duration=20&numbytes=100&code=200&delay=0` as url to test vibes download. Kind regards, Christian
Re: Vibe.d download function, how to get callback when done or error?
On 24.09.23 12:01, j...@bloow.edu wrote:
On Saturday, 23 September 2023 at 20:20:31 UTC, Christian Köstlin wrote:
On 23.09.23 14:07, j...@bloow.edu wrote:
I'm using download(url, filename) to download files in vibe.d.
The issue is that I do not know when the download is finished or
errors. There is a callback for the streaming side but not for the
file download.
A small test program shows, that if the function return normally the
transfer was done (and the file saved).
The function raises an exception if there is e,g, an http error status
communicated.
I am not sure what happens if the download is interrupted in the
middle. I guess there will be an exception, but the file might be
written partially.
Kind regards,
Christian
I can't do any testing because overnight for no explicable reason the
app will not compile any more.
If it is synchronous then I should be able to do what I need without
issue. When I initially was using it and the way the app works it seemed
like it was running in parallel because I didn't see it execute
synchronously because the tasks were relatively short and I just thought
it was asynchronous for some reason(I didn't give it any thought at the
time until I got errors in the download and needed to make the code more
robust but I can't do anything until I figure out why the code no longer
compiles(I made a post about it but it's not showing up yet).
minimal program to test the behavior (although with fibers it might be
tricky to see whats really going on .. perhaps try to do two downloads
after each other and see how the program behaves):
```d
import vibe.vibe;
import std.stdio : writeln;
void main(string[] args)
{
try {
download(args[1], args[2]);
} catch (Exception e)
{
writeln("puh: ", e);
}
}
```
Kind regards,
Christian
Re: Vibe.d download function, how to get callback when done or error?
On 23.09.23 14:07, j...@bloow.edu wrote: I'm using download(url, filename) to download files in vibe.d. The issue is that I do not know when the download is finished or errors. There is a callback for the streaming side but not for the file download. A small test program shows, that if the function return normally the transfer was done (and the file saved). The function raises an exception if there is e,g, an http error status communicated. I am not sure what happens if the download is interrupted in the middle. I guess there will be an exception, but the file might be written partially. Kind regards, Christian
Re: change object class
On 23.09.23 05:11, Vitaliy Fadeev wrote:
On Friday, 22 September 2023 at 19:50:17 UTC, Christian Köstlin wrote:
another option could be to model your own VTable in a struct like this:
https://run.dlang.io/is/3LTjP5
Kind regards,
Christian
Thank, Christian !
True nice tasty solution with ```VTable```!
And further... the project is growing.
```
void Draw()
{
DrawBG();
DrawFG();
}
```
And we want use ```DrawBG()``` code from ```initial``` in other states,
like ```Selected```.
How to use some functions from ```initial``` via ```VTable``` ?
I see solution in ```classes``` and methods with ```override``` keyword.
```VTable``` does the same thing as ```__vptr``` ?
VTable is your structure .. it does exactly what you want it to do.
__vptr is the internal implementation of virtual methods in the dlang
object model.
Line 20 and 21 in my example initialize the two `VTable`s Initial and
Hovered. You can change VTable to contain two function pointers and
initialize those as you like for the instances of the VTable structs.
e.g.
```d
struct DrawVTable
{
void function(Chip, Renderer) background;
void function(Chip, Renderer) foreground;
}
// define functions to draw the different fore and backgrounds
...
...
VTable initial = VTable(&drawInitialBackground, &drawInitialForeground);
VTable hovered = VTable(&drawHoveredBackground, &drawHoveredForeground);
VTable selected = VTable(&drawInitialBackground, &drawHoveredForegtround);
```
Kind regards,
Christian
Re: change object class
On 23.09.23 05:25, Vitaliy Fadeev wrote: On Friday, 22 September 2023 at 19:50:17 UTC, Christian Köstlin wrote: On 17.09.23 17:05, Vitaliy Fadeev wrote: Hi! You could model it oop style like this: https://run.dlang.io/is/MJb5Fk This solution might not be to your taste, as it involves interfaces, and classes and objects and garbage (all the news) ... another option could be to model your own VTable in a struct like this: https://run.dlang.io/is/3LTjP5 Kind regards, Christian ```Behavior``` is beautiful code! But it contains a second ```new``` when ```Chip``` is created. One ```new``` is possible? Christian, really nice code! Here a solution with less `new`s: https://run.dlang.io/is/iV1qVq. It really depends on the program that you are doing if creating those news is a problem. Kind regards, Christian
Re: change object class
On 17.09.23 17:05, Vitaliy Fadeev wrote:
Hi!
I want to change a method ```Draw``` on a custom object when the
```MouseIn``` event occurs.
This is known as "Change State" of the object: ```Init``` -> ```Hovered```.
I want to change the state of an object by changing its class, like this:
```d
this.__vptr = typeid(CLS).vtbl.ptr;
```
I have read the ABI and am confident in replacing ```__vptr``` as long
as the classes contain the same fields and the same interfaces.
Example:
```d
// O
// to!state
// State_Init : O
// Draw
// State_Hovered : O
// Draw
//
// o.to!State_Hovered
// o.to!State_Init
class O
{
void to(CLS)()
{
// if (same fields && same interfaces && same instance size)
this.__vptr =
cast(immutable(void*)*)typeid(CLS).vtbl.ptr;
}
}
State_Init : O
void Draw() { /* ... */ }
State_Hovered : O
void Draw() { /* ... */ }
```
when MouseIn:
```d
...
o.to!State_Hovered();
...
```
when MouseOut:
```d
...
o.to!State_Init();
...
```
It works! But I want to ask how to make this 100% the best of the best?
What should I consider before changing ```__vptr``` ?
You could model it oop style like this:
https://run.dlang.io/is/MJb5Fk
This solution might not be to your taste, as it involves interfaces, and
classes and objects and garbage (all the news) ...
another option could be to model your own VTable in a struct like this:
https://run.dlang.io/is/3LTjP5
Kind regards,
Christian
Re: Json Help
On 10.09.23 13:06, Vino wrote:
Hi All,
Request your help on the below code,I am trying to convert the below
string to json and it always throws the error, if the below can be
accomplished with any other json package even that is fine, I tired only
the std.json package.
Test Program: Works
import std.json;
import std.stdio: writeln;
void main () {
auto sadrsJson = parseJSON(`{ "TestHost": { "Building1": { "RackNo":
123}}}`);
writeln(sadrsJson);
}
```
Program: Does not Work
```
import std.json;
import std.stdio;
import std.container;
import std.socket: Socket;
import std.stdio: writeln;
void main () {
string host = Socket.hostName;
string registry = "Test"
Array!string output;
Array!string errors;
output.insertBack("TESTOUTPUT");
errors.insertBack("TESTERRORS");
auto sadrsJson = parseJSON({ host: { registry: { "OUTPUT": output[],
"ERROR": errors[]}}});
writeln(sadrsJson);
}
From,
Vino.B
`parseJSON` takes a string as arguments and tries to parse it as JSON.
The first example works, because your argument is a string and in JSON
format.
The argument in the second example is not a string, and if it would be a
string, its not JSON (e.g. `host:` is not `"host":`)...
Kind regards,
Christian
Re: pipeProcess output to hash string
On 09.09.23 17:44, Vino wrote:
Hi All,
Request your help on how to convert the output of
std.process.pipeProcess to hash string
```
auto test(in Redirect redirect=Redirect.stdout | Redirect.stderr) {
import std.process;
import std.digest.crc;
import std.stdio: writeln;
result = std.process.pipeProcess("whoami" ~ "/?", redirect);
auto content = result.stdout.readln;
auto hash = crc32Of(content);
return hash;
}
void main() {
writeln(test);
}
```
Output: All writes the below.
```
[0, 0, 0, 0]
```
Required: Read the completed output and convert it to a single hash string.
From,
Vino.B
Good that you could solve your problem already.
Just three remarks:
First I would recommend to use `std.process : execute` instead of
`pipeProcess` in this usecase, as this will wait properly for the
process to exit and it also will collect its output.
Second its always good to test the exit status of the process ... just
in case.
Third I would not look at `stderr` too much .. well behaved commandline
tools should not put data for machines there.
Kind regards,
Christian
Re: parallel threads stalls until all thread batches are finished.
On 29.08.23 00:37, j...@bloow.edu wrote:
Well, I have 32 cores so that would spawn 64-1 threads with hyper
threading so not really a solution as it is too many simultaneous downs
IMO.
"These properties get and set the number of worker threads in the
TaskPool instance returned by taskPool. The default value is totalCPUs -
1. Calling the setter after the first call to taskPool does not changes
number of worker threads in the instance returned by taskPool. "
I guess I could try to see if I can change this but I don't know what
the "first call" is(and I'm using parallel to create it).
The call that is interesting in your case is the call to `taskPool` in
your foreach loop. So if you call `defaultPoolThreads(8)` before your
loop you should be good.
Seems that the code should simply be made more robust. Probably a just a
few lines of code to change/add at most. Maybe the constructor and
parallel should take an argument to set the "totalCPUs" which defaults
to getting the total number rather than it being hard coded.
I currently don't need or have 32+ downlaods to test ATM so...
this() @trusted
{
this(totalCPUs - 1);
}
/**
Allows for custom number of worker threads.
*/
this(size_t nWorkers) @trusted
{
Basically everything is hard coded to use totalCPU's and that is the
ultimate problem. Not all tasks should use all CPU' >
What happens when we get 128 cores? or even 32k at some poin >
It shouldn't be a hard coded value, it's really that simple and where
the problem originates because someone didn't think ahead.
You have the option to not use the default value.
Kind regards,
Christian
Re: parallel threads stalls until all thread batches are finished.
On 26.08.23 05:39, j...@bloow.edu wrote: On Friday, 25 August 2023 at 21:31:37 UTC, Ali Çehreli wrote: On 8/25/23 14:27, j...@bloow.edu wrote: > "A work unit is a set of consecutive elements of range to be processed > by a worker thread between communication with any other thread. The > number of elements processed per work unit is controlled by the > workUnitSize parameter. " > > So the question is how to rebalance these work units? Ok, your question brings me back from summer hibernation. :) This is what I do: - Sort the tasks in decreasing time order; the ones that will take the most time should go first. - Use a work unit size of 1. The longest running task will start first. You can't get better than that. When I print some progress reporting, I see that most of the time N-1 tasks have finished and we are waiting for that one longest running task. Ali "back to sleep" I do not know the amount of time they will run. They are files that are being downloaded and I neither know the file size nor the download rate(in fact, the actual download happens externally). While I could use work unit of size 1 then problem then is I would be downloading N files at once and that will cause other problems if N is large(and sometimes it is). There should be a "work unit size" and a "max simultaneous workers". Then I could set the work unit size to 1 and say the max simultaneous workers to 8 to get 8 simultaneous downloads without stalling. I think thats what is implemented atm ... `taskPool` creates a `TaskPool` of size `defaultPoolThreads` (defaulting to totalCPUs - 1). The work unit size is only there to optimize for small workloads where task / thread switching would be a big performance problem (I guess). So in your case a work unit size of 1 should be good. Did you try this already? Kind regards, Christian
Re: Giant template - changing types everywhere
On 14.07.23 18:51, Steven Schveighoffer wrote: On 7/14/23 12:40 PM, Christian Köstlin wrote: Would Eponymous Templates (https://dlang.org/spec/template.html#implicit_template_properties) work with the wrapping template? Only if all the functions are named the same as the template. With eponymous templates, you no longer get access to any of the members of the template aside from the eponymous member(s). -Steve I see ... thanks for the explanation!!! Kind regards, Christian
Re: Giant template - changing types everywhere
On 14.07.23 16:15, Steven Schveighoffer wrote: On 7/14/23 1:51 AM, Cecil Ward wrote: On Friday, 14 July 2023 at 05:09:58 UTC, Cecil Ward wrote: On Friday, 14 July 2023 at 05:05:27 UTC, Cecil Ward wrote: On Friday, 14 July 2023 at 05:03:31 UTC, Cecil Ward wrote: The way I can see it going is a giant template encompassing pretty much the whole file. Does that mean that the caller who calls my one public function does so by passing the type dchar or wchar ? And then we generate the strings from that. It might be rather more natural for the caller to pass one of the string types into the template. That’s where I get rather more confused, say caller calls Transform(dstring)(dstring str) or can they just do Transform( "str"d ) and it would work out that the type is immutable dchar[] ? Perhaps I should just make up a small example file with two functions in it to see if I can get the syntax right? If I wrap the whole thing with a template declaration of the xchar type, then can I get away with no changes to the individual function definitions? I tried it, wrapped the whole thing in a template definition and it compiled, but then my test file which calls Transform( someDString ) failed to compile with errors saying it couldn’t find the definition of Transform in the other module, which is or was public. It’s as if it is no longer public because it’s now inside the template. So templates don't automatically instantiate, you have to specify them. And then if your function is inside the template, to access it, you will need to do: ```d GiantTemplate!dstring.Transform(str); ``` But this is not a usual way of creating API. Instead, you should template individual functions on the string type. Depending on what you are doing, you can use: ```d T Transform(T)(T val) T[] Transform(T)(T[] val) ``` The first will cover cases where you have custom string types that aren't arrays, the second will just capture the array type, and ensures that the parameter/return is an array. When you make template functions like this, a feature of D called Implicit Function Template Instantiation (IFTI) will automatically instantiate the template for you, so you don't have to specify the template parameters. You just call `Transform(str)` and it works. With the wrapping template solution, this is not available -- you must explicitly instantiate the wrapper. If you are having problems, it is nearly impossible to diagnose without some actual code to look at. -Steve Would Eponymous Templates (https://dlang.org/spec/template.html#implicit_template_properties) work with the wrapping template? Kind regards, Christian
Re: Mixin and compile-time functions for code generation
On 24.06.23 18:31, Cecil Ward wrote:
I have a function that can be run at compile-time and which will be able
to output code to be injected into the D source code stream. Can I get
mixin whatever to do this for me? Mixin with a function that runs at
compile-time and creates the required source ? Like D’s solution for a
replacement for function-style C preprocessor macros ? - but far more
advanced and capable ?
I need to re-read Ali Çehreli’s excellent book for the third time.
Is this what you want todo:
```d
import std.string : format, capitalize;
import std.stdio : writeln;
string getter(string v)
{
return "auto get%s() { return %s; }".format(v.capitalize, v);
}
class T
{
int abc;
mixin(getter("abc"));
}
void main() {
auto t = new T();
writeln(t.getAbc());
}
```
Re: serve-d and emacs
I tried to reproduce my old eglot experiment, and for me serve-d was not even compiling with the newest dmd. Which versions are you using? Kind regards, Christian
How do you work with unittest libraries in dub packages?
Recently Dmytro Katyukha brought up an issue in one of my dub packages that is supposed to be used as a library. He even went the whole way and came up with a simple reduced example: https://gitlab.com/gizmomogwai/colored/-/merge_requests/3#note_1341026928. The problem here is, that my dub package uses unit_threaded in the unittest configuration, and only imports unit_threaded in a `version (unittest)` block. Previously I imported unit_threaded only in the individual testfunctions themselves, but I thought to reduce some typing. When my library is now used in another project (that does not depend on unit_threaded) `dub test` seems to still compile my library with the unittest version, but it would not run the unittests of my library. So in the end `dub test` fails for the dependent project (as it is not providing unit_threaded). Is this behavior expected? What would be best practice to work around that? Kind regards, Christian
How to work with phobos github projects
Recently I was looking in contributing to dlang/phobos and found the github subprojects for phobos (https://github.com/dlang/phobos/projects?type=classic) which include a project to improve the public examples for phobos (https://github.com/dlang/phobos/projects/1). I looked at one of the cards there (https://github.com/dlang/phobos/projects/1#card-7028566) which wants to improve std.container.array's public examples. Looking at the source and the dscanner results for std/container/array.d it is visible, that the modules' example already contain the examples, that dscanner is missing. So one way would be to move the examples to the place that dscanner expects or come perhaps up with new examples. Anyways the solution to that is not really clear, and I would like to discuss with someone who is more familiar with phobos what todo (perhaps even just drop the issue in github). Normally one can comment on github issues, but I found no such mechanism for those projcts/cards. So how would one move forward there? Kind regards, Christian
Re: Simplest way to convert an array into a set
On 13.02.23 19:04, Matt wrote: Obviously, there is no "set" object in D, but I was wondering what the quickest way to remove duplicates from an array would be. I was convinced I'd seen a "unique" method somewhere, but I've looked through the documentation for std.array, std.algorithm AND std.range, and I've either missed it, or my memory is playing tricks on me. That, or I'm looking in the wrong place entirely, of course You are looking probably for https://dlang.org/library/std/algorithm/iteration/uniq.html. Kind regards, Christian
Re: Need some technical help an object oriented wrapper I am creating for bindbc.sfml
On 24.01.23 04:59, thebluepandabear wrote: Regards, thebluepandabear Btw I understand this question is extremely complex, don't want to pressure anyone to help me because of that... but any sort of assistance or leads would be greatly... greatly apprecaited... I do not know anything about sfml, but could you try a simpler shape, e.g. circle (that is a primitive on the native side). perhaps the winding order of your vertices is wrong? Kind regards, Christian
Re: Coding Challenges - Dlang or Generic
On 10.01.23 23:30, Paul wrote: On Tuesday, 10 January 2023 at 01:31:28 UTC, Ali Çehreli wrote: On 1/9/23 16:17, Paul wrote: > coding challenges Perhaps the following two? https://rosettacode.org/ https://adventofcode.com/ Ali Excellent. Thanks. For this years advent-of-code Steven Schveighoffer (https://github.com /schveiguy/adventofcode/tree/master/2022) has a complete set of dlang solutions. Kind regards, Christian
Re: Coding Challenges - Dlang or Generic
On 10.01.23 23:22, monkyyy wrote: On Tuesday, 10 January 2023 at 19:10:09 UTC, Christian Köstlin wrote: On 10.01.23 01:17, Paul wrote: There is also https://exercism.org/tracks/d with some tasks for dlang. Kind regards, Christian Its all converted code; worthless I was not aware, that the question was searching for solutions in dlang. At least exorcism gives a skeleton for a dlang project (including unittest) to solve the task. Kind regards, Christian
Re: Coding Challenges - Dlang or Generic
On 10.01.23 01:17, Paul wrote: There is also https://exercism.org/tracks/d with some tasks for dlang. Kind regards, Christian
Advent of Code 2022
Is anybody participating with dlang in the advent of code 22? It would be interesting to discuss dlang specific things from the puzzles. Kind regards, Christian
Re: Gotcha with photos' documentation
On 09.12.22 19:55, H. S. Teoh wrote:
On Fri, Dec 09, 2022 at 12:51:27PM +0100, Christian Köstlin via
Digitalmars-d-learn wrote:
On 09.12.22 02:27, H. S. Teoh wrote:
[...]
https://github.com/dlang/phobos/pull/8646
[...]
Thanks a lot ...
that was fast.
It only took a minute to fix. :-D
Is there also an implementation that works on non sorted ranges?
[...]
Not as far as I know, because on a non-sorted range you'd have a much
higher time/space complexity. You could sort the range first, but
that's O(n log n) time and possibly O(n) space if you want to preserve
the original range.
Alternatively, you could use an AA to keep track of items already seen,
but that's also O(n) space in the worst case. Ostensibly it'd be O(n)
time as well, but that depends on your data set (some pathological cases
may trigger poorer AA performance). It will also trigger GC allocations.
Something along these lines:
auto noDuplicates(R)(R range)
if (isInputRange!R)
{
bool[ElementType!R] aa;
return range.filter!((e) {
if (e in aa)
return false;
aa[e] = true;
return true;
});
}
T
Thanks for the sketch ... sure ... for pathological cases all hope is
lost :).
Kind regards,
Christian
Re: Gotcha with photos' documentation
On 09.12.22 02:27, H. S. Teoh wrote: On Thu, Dec 08, 2022 at 05:21:52PM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] I'll see if I can reword this to be more explicit. [...] https://github.com/dlang/phobos/pull/8646 T Thanks a lot ... that was fast. Is there also an implementation that works on non sorted ranges? Kind regards, Christian
Gotcha with photos' documentation
Recently I stumbled upon a small issue in dlang's docs. I wanted to look up uniq in std.algorithm. Started from https://dlang.org/phobos/std_algorithm.html and clicked uniq, no problem, all good. But my code did not work. After some debugging I saw, that for some inputs uniq just did not work. I clicked around some more in the online docs and finally arrived at https://dlang.org /phobos/std_algorithm_iteration.html which also lists uniq at the beginning of the page and mentions (only there) that the input needs to be sorted. With that all was back to good :) Would it make sense to mention this prerequisite also at the main documentation of uniq (https://dlang.org/phobos/std_algorithm_iteration.html#uniq) Kind regards, Christian
Is it just me, or does vibe.d's api doc look strange?
Please see this screenshot: https://imgur.com/Ez9TcqD of my browser (firefox or chrome) of https://vibed.org/api/vibe.web.auth/ Kind regards, Christian
Re: Makefiles and dub
On 05.11.22 12:38, rikki cattermole wrote: We have a few build formats that dub can generate for you automatically: ``` visuald - VisualD project files sublimetext - SublimeText project file cmake - CMake build scripts build - Builds the package directly ``` Unfortunately none of them are make, it would be nice to have that if you are looking to contribute! If cmake works, then cmake could generate a normal makefile :) Kind regards, Christian
Re: Make IN Dlang
On 02.11.22 17:24, Kagamin wrote:
Another idea is to separate the script and interpreter then compile them
together.
```
--- interp.d ---
import script;
import ...more stuff
...boilerplate code
int main()
{
interpret(script.All);
return 0;
}
--- script.d ---
#! ?
module script;
import mind;
auto All=Task(...);
...more declarative tasks
--- run ---
dmd /usr/local/interp.d /path/to/script.d
```
Thanks, have to think a little about that :)
Kind regards,
Christian
Re: Make IN Dlang
On 02.11.22 20:16, H. S. Teoh wrote:
On Wed, Nov 02, 2022 at 03:08:36PM +, JN via Digitalmars-d-learn wrote:
On Tuesday, 1 November 2022 at 23:40:22 UTC, Christian Köstlin wrote:
sh("touch %s".format(t.name));
One of the problems of many Make-like tools is that they offer lots of
freedom, especially when allowing you to launch arbitrary shell
commands. But this also comes with drawbacks, because this touch
command will instantly break Windows builds, might also be a problem
on some non-Linux platforms like macOS. Declarative approach from
tools like dub might be restrictive, but it also lets me as a user
know that I can download an arbitrary dub project and 99% chance it
will just compile out of the box on Windows.
IMO, the ideal situation is a hybrid situation: the underlying build
mechanism should be purely declarative, because otherwise the complexity
just goes out of control and you end up with non-portability,
non-reproducible builds, intractibility of static analysis (e.g., for an
external tool to understand what the build does). However, quite often
in a large project you need to perform some complex tasks, and doing
this declaratively can be too cumbersome. So what you want is a
procedural element to the build description that *generates* the
underlying declarative elements of the build. The procedural part does
not perform any build actions; its job is to generate the declarative
build description. The actual build is carried out based on this
declarative build description.
T
Thats an interesting approach. Reggae goes a little bit in that
direction, right?
Kind regards,
Christian
Re: Make IN Dlang
On 02.11.22 04:07, rikki cattermole wrote: Something to consider: dub can be used as a library. You can add your own logic in main to allow using your build specification to generate a dub file (either in memory or in file system). Nice ... I will perhaps give that a try! Kind regards, Christian
Re: Make IN Dlang
On 02.11.22 03:25, Tejas wrote: On Tuesday, 1 November 2022 at 23:40:22 UTC, Christian Köstlin wrote: Dear dlang-folk, one of the tools I always return to is rake (https://ruby.github.io/rake/). For those that do not know it, its a little like make in the sense that you describe your build as a graph of tasks with dependencies between them, but in contrast to make the definition is written in a normal programming language (in this case ruby) with all features of it. [...] Sounds pretty similar to [tup](https://gittup.org/tup/) I am a great admirer of tup (especially because they incorporated as the first system (that I know of) a filesystem monitor outside of IDEs). Its language is make like I would say, for that I do not like it that much. Reggae, the build system mentioned by Adam, supports tup as a backend, so you could use that as well +1 Kind regards, Christian
Re: Make IN Dlang
On 02.11.22 00:51, Adam D Ruppe wrote: I don't have specific answers to your questions but your goal sounds similar to Atila's reggae project so it might be good for you to take a look at: https://code.dlang.org/packages/reggae Hi Adam, thanks for the pointer. I forgot about reggae ;-) From the documentation it looks more high level and already defines things like libraries and so on. It's also very ambitious in that it tries to support different backends! Mind is more low level (although some language specific things could be added on top of it. I like reggaes approach in reading in the "source" and creating something new from it (perhaps with dub as library) this could get rid of some of the boilerplate of what I have now. Kind regards, Christian
Make IN Dlang
Dear dlang-folk,
one of the tools I always return to is rake
(https://ruby.github.io/rake/). For those that do not know it, its a
little like make in the
sense that you describe your build as a graph of tasks with dependencies
between them, but in contrast to make the definition is written in
a normal programming language (in this case ruby) with all features of
it.
Dlang is also quite expressive, so I thought why not define the build in
Dlang.
The result is mind (https://gitlab.com/gizmomogwai/mind). An example
mindfile looks like this:
```
#!/usr/bin/env dub
/+ dub.sdl:
name "mindfile"
dependency "mind" version="~master"
...further dependencies...
+/
import mind;
import std.stdio : writeln;
import std.format : format;
import std.range : iota;
import std.algorithm : map;
import std.array : array;
import core.thread.osthread : Thread;
import core.time : msecs;
int main(string[] args)
{
description("Do all");
auto all = task("all", null, (t) {});
for (int i=0; i<1000; ++i)
{
auto fileName = "out/file-%s.txt".format(i);
all.enhance(fileName);
description(fileName);
file(fileName, null, (t)
{
Thread.sleep(100.msecs);
sh("touch %s".format(t.name));
},
);
}
return mindMain(args);
}
```
This uses dub's single file feature
(https://dub.pm/advanced_usage#single-file) to get the helper library
and "execution engine" (mind).
The main functions defines a bunch of tasks or file-tasks and finally
forwards to the main function of the library for parallel (if possible)
execution of the tasks.
At the moment this is still in an experimental stage, but has nice
features such as:
- colorized --help (thanks to argparse)
- --tasks to show all defined (and commented tasks)
- parallel execution
I am still trying to find answers to the following questions:
1. Is it somehow possible to get rid of the dub single file scheme, and
e.g. interpret a full dlang script at runtime?
2. How can the program as is made nicer/shorter? (one could just import
std, or make use of https://code.dlang.org/packages/scriptlike)?
3. Does this make sense at all, or should we just use make/rake? (dub
also uses a custom build.d file).
Kind regards,
Christian
Re: rotate left an array
If you are ok with using things from std.range you could use something
like this:
```d
import std.range : cycle, drop, take;
import std.stdio : writeln;
int main(string[] args)
{
auto r = [1, 2, 3, 4, 5, 6, 7, 8];
writeln(r.cycle.drop(3).take(r.length));
return 0;
}
```
Kind regards,
Christian
Re: Is there a way to mark a dub package as linux only?
On 27.09.22 13:07, Ahmet Sait wrote:
On Monday, 26 September 2022 at 20:57:06 UTC, Christian Köstlin wrote:
Or posix only? Or not windows?
Kind regards,
Christian
Not necessarily a dub solution but you can do something like this:
```d
version(Posix) { }
else
static assert(0, "Unsupported platform.");
```
This will result in a compiler error while targetting a non-posix platform.
Thanks a lot ... but thats a little late for me :) as someone already
might have added my library and was looking forward on using it.
Is there a way to mark a dub package as linux only?
Or posix only? Or not windows? Kind regards, Christian
Anaphoric "macros"
Hi, I just stumbled upon anaphoric macros (https://en.wikipedia.org/wiki/Anaphoric_macro) in elisp. Seems that the dlang feature e.g. `map!"a*2"` is something similar to that, although I never read about it phrased like that before. Kind regards, Christian
Re: Setting import paths - project (dub) and also rdmd or dmd
On 19.09.22 16:24, David wrote: Hi, New to D (and enjoying the learning..) I've probably missed something obvious but I'm slightly confused with the best way to achieve a simple build. I like to keep my reusable modules in a directory outside of the project directory so I can use them on another project for example. I do :- import externalmodulename; rdmd app.d and get a fail. No problems, I can run rdmd thus :- rdmd -I../EXTERNALMODULES app.d and it all compiles nicely. Same goes for dmd just use dmd -I and it compiles the code clean. I'd normally just have a CPPFLAGS=... line in a classic Makefile to do this for me but can't see a native D way of doing it. I looked at dub and that that :- dub add-path ../EXTERNALMODULES would do the trick but it doesn't make a difference (the path is added to the relevant json file in ~/.dub), I also tried add-local but no difference. I was expecting it to add something to dub.sdl but that hasn't changed since I created the project. TLDR: How do I automatically specify the -I to the compilers so I don't need to specify it manually each time ? Regards, D Another option that goes completely around the -I issue is if you make your module into a full dub package and add this with dub add-local. Then it becomes available as a regular dub dependency, which can be used in dub builds as well as in rdmd builds (see https://dub.pm/advanced_usage#single-file). Not sure how to solve the dmd thing though, besides what Steven proposed. Kind regards, Christian
Re: dub lint
On 16.09.22 02:23, rikki cattermole wrote: https://github.com/dlang/dub/issues/2483 Also the double --config option is already in a bugreport (quite old), but not fixed as far as i can see: https://github.com/dlang/dub/issues/1940 Kind regards, Christian
Re: dub lint
On 16.09.22 01:14, Christian Köstlin wrote: On 16.09.22 00:14, Ali Çehreli wrote: On 9/15/22 15:04, Ali Çehreli wrote: > Is there a way to silence specific 'dub lint' warnings? Answering myself, I don't think it's possible but luckily my catching an Error was in unittests only so I can do either of the following to skip unittest code when linting: a) Pass --skipTests to dscanner (what 'dub lint' runs behind the scenes) along with --styleCheck, which --skipTests requires dub lint -- --skipTests --styleCheck b) Pass --styleCheck indirectly through 'dub lint', which has its own spelling for it :), but --skipTests is still required of course: dub lint --style-check -- --skipTests Ali There is `dub run dscanner -- --defaultConfig` which creates a default config in `~/.config/dscanner/dscanner.ini` (for linux and osx). There one can disable only the checks for bad exception handling ``` ; Check for poor exception handling practices exception_check="disabled" ``` and then its possible to move this file e.g. to the project folder and use in with `dub lint --config=dscanner.ini` ... interestingly `dub lint --help` shows two different options for `--config`: ``` dub lint --help USAGE: dub lint [[@]] [] [-- ] .. .. --config=VALUE Use the given configuration file. .. .. -c --config=VALUE Builds the specified configuration. .. .. ``` Kind regards, Christian p.s. phobos for example includes its own dscanner.ini file in https://github.com/dlang/phobos/blob/master/.dscanner.ini
Re: dub lint
On 16.09.22 00:14, Ali Çehreli wrote: On 9/15/22 15:04, Ali Çehreli wrote: > Is there a way to silence specific 'dub lint' warnings? Answering myself, I don't think it's possible but luckily my catching an Error was in unittests only so I can do either of the following to skip unittest code when linting: a) Pass --skipTests to dscanner (what 'dub lint' runs behind the scenes) along with --styleCheck, which --skipTests requires dub lint -- --skipTests --styleCheck b) Pass --styleCheck indirectly through 'dub lint', which has its own spelling for it :), but --skipTests is still required of course: dub lint --style-check -- --skipTests Ali There is `dub run dscanner -- --defaultConfig` which creates a default config in `~/.config/dscanner/dscanner.ini` (for linux and osx). There one can disable only the checks for bad exception handling ``` ; Check for poor exception handling practices exception_check="disabled" ``` and then its possible to move this file e.g. to the project folder and use in with `dub lint --config=dscanner.ini` ... interestingly `dub lint --help` shows two different options for `--config`: ``` dub lint --help USAGE: dub lint [[@]] [] [-- ] .. .. --config=VALUEUse the given configuration file. .. .. -c --config=VALUEBuilds the specified configuration. .. .. ``` Kind regards, Christian
Re: How check if destructor has been called?
On 13.09.22 19:13, Ben Jones wrote:
On Tuesday, 13 September 2022 at 14:06:42 UTC, Injeckt wrote:
Hi, I'm trying to check if destructor has been called, but when I'm
deleting class object I didn't get any calls from destructor.
myclass.d
~this() {
this.log("\nDestructor\n");
this._free_trash();
}
main.d
try {
server.server_init(server);
} catch (Exception e) {
server.log(e.msg);
server = null;
}
Classes are allocated on the GC heap, so even though you're setting the
reference to null, it's not being collected at that point. You can use
`destroy` to make sure the destructor is called (see here:
https://dlang.org/spec/class.html#destructors)
Or you could make your instance `scope` and then it basically follows
the same lifetime rules as `struct`s.
Some more things to watch out are mentioned in
https://p0nce.github.io/d-idioms/#The-trouble-with-class-destructors.
In one of my programs I had problems with allocating in GC called
destructors (by doing "harmless" writeln debugging).
Kind regards,
Christian
How do you work with lst files?
I want to ask around how you from the dlang community work with .lst coverage files? For me those files are really one of the best formats as they are (really, in contrast to some xml things) human readable and someone added them to codecov. My setup at the moment consists of a small tool that I can plug into postRunCommands (e.g. of the unittest dub configuration) and that converts those files to "normal" dmd style warnings, so that my editor (in my case emacs, but the same works with the vs code plugin) can pick them up. looking forward to learn about your setup! Kind regards, Christian
Re: Recommendation for parallelism with nested for loops?
On 20.08.22 12:28, Christian Köstlin wrote:
On 19.08.22 03:49, Shriramana Sharma wrote:
Hello. I want to parallelize a computation which has two for loops,
one nested within another. All inner-loop-param+outer-loop-param
combinations can be computed independent of one another.
As I suspected,
[https://forum.dlang.org/post/xysyidbkjdinclmrx...@forum.dlang.org](this forum post) says that only one loop can be parallelized. Will it be an error or inefficient or useless if I try to do both?
Also, what is the best way to do parallelism in such a situation?
You could also do a custom range that makes a one-dimensional range (aka
iota out of your nested loops) and then process this with parallel.
Another way (more similar to Ali's solution) would be to write the
nested loops as you have them, but collect the parameters for the work
in an array or something and then process this array in parallel.
Kind regards,
Christian
Actually phobos brings already all that is required (especially the
cartesian product):
```d
import std;
void doSomething(int i, string s)
{
writeln("%s-%s".format(i, s));
}
void main()
{
foreach (t; cartesianProduct(iota(1, 4), ["abc", "def",
"ghi"]).parallel) {
doSomething(t.expand);
}
}
```
kind regards,
Christian
Re: Recommendation for parallelism with nested for loops?
On 19.08.22 03:49, Shriramana Sharma wrote: Hello. I want to parallelize a computation which has two for loops, one nested within another. All inner-loop-param+outer-loop-param combinations can be computed independent of one another. As I suspected, [https://forum.dlang.org/post/xysyidbkjdinclmrx...@forum.dlang.org](this forum post) says that only one loop can be parallelized. Will it be an error or inefficient or useless if I try to do both? Also, what is the best way to do parallelism in such a situation? You could also do a custom range that makes a one-dimensional range (aka iota out of your nested loops) and then process this with parallel. Another way (more similar to Ali's solution) would be to write the nested loops as you have them, but collect the parameters for the work in an array or something and then process this array in parallel. Kind regards, Christian
Re: How to use exceptions
On 13.08.22 17:00, kdevel wrote:
"Exception enrichment" would be my wording which is supported by google
[1].
There is also the notion of "exception context" [2] and "contexted
exception" [3].
Thats really a good word! Especially it describes better what the java
guys are doing by adding information to an existing object and
rethrowing, without creating a new exception, covering the tracks of the
original exception by wrapping it and throwing this.
Can we have that for dlang exceptions as well (I mean as additional api
in the baseclass)? It would not break old code, as it would be
additional api?
The current filename imposes a context on its processing. The exception
thrown from readText knows that filename and propagates it within the
exception msg. Unfortunately exceptions are frequently only "stringly
typed", e.g.
yes .. thats a shame. I think it is, because its much shorter than
defining a new member, a new constructor, a new message method in the
new exception subclass.
394: throw new JSONException("JSONValue is not a number type");
406: throw new JSONException("JSONValue is not a an integral type");
This should have been made into an exception hierarchy with the faulty
value beeing a data member of the respective class. That would nicely
support the crafting of a message for the user in the presentation layer.
I do agree.
Of course the UTF8Exception must be enriched with the filename in the
application code.
I do not agree. As for me it looks like `readText` works with kind of
two API's internally. One is the file API that can throw, and forwards
information nicely (about which file has the problem). And then some
UTF-8 processing based on data from that file (which should also forward
the information of the file, as the end user of this API did not even
touch a file at all (only a string representing a filename)).
But for sure there are different scenarios where the enrichment needs to
happen at call site.
3. localization is a fantastic example of data that needs to be added
to almost every exception that raises from below.
How would you handle a FileException.msg = "myconfig.cnf: Permission
denied"? Run regular expressions over the msg? Something is
fundamentally wrong if any kind of parsing of error strings is required.
Thats what I mean with "that raises from below". I think the
FileException is not supposed to be on the top of the exception stack,
but a more application level exception (e.g. Cannot process
configuration file (as Configuration file is something on application
level)).
Then there is another type of user who shall not be faced with any kind
of specific error message: The client of a company which runs a web
application. This case is easy to implement: Catch any Exception and
replace it with an "The Unforeseen happend, id#" message. Leave the code
as it is and let the stack trace go into the error_log. The devops
people of the company probably understand english messages.
One could argue, that every exception stacktrace that comes to
stdout/-err or a user interface is already a bug. It might be helpful
for developers and system administrators, but not for end users. So your
solution is perhaps inconvinient for developers, but good in general.
[1]
https://jenkov.com/tutorials/java-exception-handling/exception-enrichment.html
[2]
https://docs.microsoft.com/en-us/dotnet/api/system.web.mvc.exceptioncontext?view=aspnet-mvc-5.2
[3]
https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/exception/ContextedException.html
thanks again for the links ... nice read!!!
Kind regards,
Christian
Re: How to use exceptions
On 13.08.22 15:00, kdevel wrote:
On Friday, 12 August 2022 at 21:41:25 UTC, Christian Köstlin wrote:
which would enable something like
```d
return s
.readText
.parseJSON
.contextWithException((UTFException e) {
return new Exception("Cannot process UTF-8 in config
file%s\n %s".format(s, e.msg), e);
})
.contextWithException((FileException e) {
return new Exception("Cannot process config file%s\n
%s".format(s, e.msg), e);
});
```
This is not as DRY as it could be. Furthermore I would try implement the
error handling completely outside the main execution path, ideally in a
wrapper around a the old main function (renamed to main_). This approach
becomes problematic if exceptions of the same class can be thrown from
two functions of the chain. >
Your code is printing e.msg. How to you localize that string?
Those 3 points are exactly touching the problems (I am ignoring the DRY
thing for now, as this is a side battle):
1. error handling in main path: exactly thats what I would like todo.
but for that the errors that raise need to have meaningful information.
this is exactly what I am trying in those context* functions ... they do
not do error handling, but more something like error enhancement (by
fixing up the error information).
2. yes ... for that the chain would need to be broken up, then you can
react on the same exception classes of different members of the chain
differently ... for that I do not see a nice way to write it in d.
3. localization is a fantastic example of data that needs to be added to
almost every exception that raises from below. my example with the
strings is just an example, it could also be that by conventions of your
application framework all context* functions raise
LocalizableExceptions, that somehow can then be mapped to different
languages with their own api. the generalalized main function then would
handle those exception classes differently from "normal" exceptions.
hope that makes sense.
kind regards,
christian
Re: How to use exceptions
On 12.08.22 23:05, Christian Köstlin wrote:
On 12.08.22 01:50, H. S. Teoh wrote:
...
>
The OP's idea of wrapping throwing code with a function that tacks on
extra information is a good idea. Perhaps the use of strings isn't
ideal, but in principle I like his idea of exceptions acquiring
higher-level information as it propagates up the call stack.
Thanks for the kind words ... I am still thinking how to put this
into a nicer API ... as it is in my current demo, its not much to type
out so I am happy with that. Actually having read the source of
ifThrown, its amazing how lazy makes wrapping easy!
One thing that can be done is to templateize the exception handler so
that only exceptions of a certain type are handled.
```d
auto contextWithException(T, E)(lazy scope T expression, Exception
delegate(E) handler)
{
Exception newException;
try
{
return expression();
}
catch (E e)
{
newException = handler(e);
}
throw newException;
}
```
which would enable something like
```d
return s
.readText
.parseJSON
.contextWithException((UTFException e) {
return new Exception("Cannot process UTF-8 in config
file%s\n %s".format(s, e.msg), e);
})
.contextWithException((FileException e) {
return new Exception("Cannot process config file%s\n
%s".format(s, e.msg), e);
});
```
Not sure if that makes it any better though, as I have the feeling that
most exceptions need to be wrapped at least once on their way to the end
user.
kind regards,
Christian
Run dub build with all generated files in /tmp
Sometimes I do not only rely on git to transport dub projects from computer to computer, but also on Dropbox or Syncthing or similar tools. For that it would be great if it would be possible to do all dub commands (e.g. build) in a way, that they are not touching the current working directory. For `dub run` I found the option `--temp-build` that builds to /tmp, but I did not find a similar option for dub build. Is there some way to have all files generated by dub out of the project folder? Kind regards, Christian
Re: How to use exceptions
On 12.08.22 01:50, H. S. Teoh wrote: ... > The OP's idea of wrapping throwing code with a function that tacks on extra information is a good idea. Perhaps the use of strings isn't ideal, but in principle I like his idea of exceptions acquiring higher-level information as it propagates up the call stack. Thanks for the kind words ... I am still thinking how to put this into a nicer API ... as it is in my current demo, its not much to type out so I am happy with that. Actually having read the source of ifThrown, its amazing how lazy makes wrapping easy!
Re: How to use exceptions
On 12.08.22 01:06, Adam D Ruppe wrote: You might find my recent blog post interesting too: http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept and a draft of some more concepts: http://arsd-official.dpldocs.info/source/arsd.exception.d.html I also find the lack of information disturbing, but I also hate formatting it all into strings, so I wanted to experiment with other options there that avoid the strings. Yes ... I already read your blog post. In general I am with you there as it makes sense to have either dedicated exception classes or something more along the lines of what you describe in your post.
How to use exceptions
Dear d-lang experts,
lets say in general I am quite happy with exceptions.
Recently though I stumbled upon two problems with them:
1. Its quite simple to loose valuable information
2. Its hard to present the exception messages to end users of your
program
Let me elaborate on those:
Lets take a simple config parser example (pseudocode, the real code is
in the end):
```d
auto parseConfig(string filename)
{
return s
.readText
.parseJSON;
}
..
void main()
{
...
auto config = parseConfig("config.json");
run(config);
...
}
```
Lets look at what we would get in terms of error messages for the user
of this program (besides that a full stacktrace is printed, which is
nice for the developer of the program, but perhaps not so nice for the
end user)
- file is not there: config.json: No such file or directory
nice ... its almost human readable and one could guess that something
with the config file is amiss
- file is not readable for the user: config.json: Permission denied
nice ... quite a good explanation as well
- file with invalid UTF-8: Invalid UTF-8 sequence (at index 1)
not so nice ... something is wrong with some UTF-8 in the program,
but where and in which area of the program
- file with broken json: Illegal control character. (Line 4:0)
not nice ... line of which file, illegal control character does
not even sound like json processing.
Arguably readText could behave a little better and not throw away
information about the file its working with, but for parseJSON the
problem is "bigger" as it is only working with a string in memory not
with a file anymore, so it really needs some help of the application
developer I would say. When browsing through phobos I stumbled upon the
genius std.exception.ifThrown function, that allows for very nice
fallbacks in case of recoverable exceptions. Building on that I came up
with the idea (parhaps similar to Rusts error contexts) to use this
mechanism to wrap the exceptions in better explaining exceptions. This
would allow to provide exceptions with the information that might
otherwise be lost aswell as lift the error messages onto a end user
consumable level (at least if only the topmost exception message is
looked at).
```d
#!/usr/bin/env rdmd
import std;
auto contextWithString(T)(lazy scope T expression, string s)
{
try
{
return expression();
}
catch (Exception e)
{
throw new Exception("%s\n%s".format(s, e.msg));
}
assert(false);
}
auto contextWithException(T)(lazy scope T expression, Exception
delegate(Exception) handler)
{
Exception newException;
try
{
return expression();
}
catch (Exception e)
{
newException = handler(e);
}
throw newException;
}
// plain version, no special error handling
JSONValue readConfig1(string s)
{
// dfmt off
return s
.readText
.parseJSON;
// dfmt.on
}
// wraps all exceptions with a description whats going on
JSONValue readConfig2(string s)
{
// dfmt off
return s
.readText
.parseJSON
.contextWithString("Cannot process config file %s".format(s));
// dfmt on
}
// tries to deduplicate the filename from the exception messages
// but misses on utf8 errors
JSONValue readConfig3(string s)
{
// dfmt off
auto t = s
.readText;
return t
.parseJSON
.contextWithString("Cannot process config file %s".format(s));
// dfmt on
}
// same as 3 just different api
JSONValue readConfig4(string s)
{
// dfmt off
auto t = s
.readText;
return t
.parseJSON
.contextWithException((Exception e) {
return new Exception("Cannot process config file%s\n
%s".format(s, e.msg));
});
// dfmt on
}
void main()
{
foreach (file; [
"normal.txt",
"missing.txt",
"broken_json.txt",
"not_readable.txt",
"invalid_utf8.txt",
])
{
writeln("=");
size_t idx = 0;
foreach (kv; [
tuple("readConfig1", &readConfig1),
tuple("readConfig2", &readConfig2),
tuple("readConfig3", &readConfig3),
tuple("readConfig4", &readConfig4),
])
{
auto f = kv[1];
try
{
if (idx++ > 0)
writeln("-");
writeln("Working on ", file, " with ", kv[0]);
f("testfiles/%s".format(file));
}
catch (Exception e)
{
writeln(e.msg);
}
}
}
}
```
What do you guys think about that?
Full dub project available at
g...@github.com:gizmomogwai/d-exceptions.git ... If you want to play with
it, please run ./setup-files.sh first (which will create a file that is
not readable).
One thing I
Re: Combining JSON arrays into a single JSON array -- better way than this?
On 2022-08-01 06:24, ikelaiah wrote:
Hi,
I've written a cli tool to merge JSON files (containing JSON array) in
the current folder as a single JSON file.
My algorithm:
1. Create a string to store the output JSON array as a string,
2. read each file
3. read each object in JSON array from input file
4. append the string representation of each JSON object in a string
5. Parse the result string as JSON
6. Save string representation of point 6 above.
**Question**
While this works, is there a more concise way than this?
Thank you for your time.
```d
module combinejsonv2;
import std.file;
import std.stdio;
import std.json;
import std.array;
import std.algorithm.searching;
void main()
{
// can't I use this for JSON array?
JSONValue jj;
// create a string opening [ for JSON array
string stringResult = "[";
foreach (string filename; dirEntries(".", "*.json", SpanMode.shallow))
{
// if filename contains 'output' in it, ignore
if(canFind(filename, "output")) {
std.stdio.writeln("ignoring: " ~ filename);
continue;
}
// show status to console
std.stdio.writeln("processing: " ~ filename);
// read JSON file as string
string content = std.file.readText(filename);
// parse as JSON
JSONValue j = parseJSON(content);
foreach (JSONValue jsonObject; j.array) {
// Combine objects from files into a single list of JSON
object
// std.stdio.writeln(jsonObject.toPrettyString);
stringResult ~= jsonObject.toString;
stringResult ~= ",";
}
}
// create closing ] for the JSON array
stringResult ~= "]";
// parse the string as a JSON object
JSONValue jsonResult = parseJSON(stringResult);
// write to file
std.file.write("output-combined.json", jsonResult.toPrettyString);
}
```
An arguably shorter solution (that drops some of your logging) could be:
```d
import std;
void main() {
dirEntries(".", "*.json", SpanMode.shallow)
.filter!(f => !f.name.canFind("output"))
.map!(readText)
.map!(parseJSON)
.fold!((result, json) { result ~= json.array; return result; })
.toPrettyString
.reverseArgs!(std.file.write)("output-combined.json");
}
```
not sure if you are looking for this style though.
kind regards,
Christian
How to work with coverage data
Hi dlang lovers, I recently wanted to improve how I work with coverage data (locally). For that I came up with a small program, that can be either called after a `dub test --coverage` or that can be automatically executed after the unittest with `postRunCommands "$DUB run lst2errormessages"` in `dub.sdl` and that converts the generated lst files to error messages compatible with dmd output. That way you can integrate them into your favorite editor, if it already has some dmd error output support. I use it mainly in emacs compile-mode or flycheck-mode (the latter with another small addition), but its also possible to run this in vscode + d-code. Is there any interest in a more detailed write up of this. Anyways, the code is available at https://gitlab.com/gizmomogwai/lst2errormessages Kind regards, Christian
Re: Fetching licensing info for all dependencies of a DUB project
On 2022-06-28 14:34, Guillaume Piolat wrote: On Monday, 27 June 2022 at 21:36:31 UTC, Christian Köstlin wrote: I played around with the idea and came up with a small dub package, that is not (yet) uploaded to the dub registry. Source is available at https://github.com/gizmomogwai/packageinfo, feedback very welcome. I've done something similar not for licences but for code amount, to extract from a DUB project: - DUB packages used by project - source files used by project - and their LOC count This is a D forums exclusive: https://pastebin.com/RFbFCgR2 Keep your debt in check! Interesting ... there is also dscanner that can count lines of code. in another toy project of mine I create a dependency dot graph, might be good to collect all those things together ... there is also https://github.com/funkwerk/cogito that collect another metric of projects.
Re: Fetching licensing info for all dependencies of a DUB project
On 2020-05-12 15:23, Paul Backus wrote: On Tuesday, 12 May 2020 at 13:08:01 UTC, Joseph Rushton Wakeling wrote: On Tuesday, 12 May 2020 at 12:59:14 UTC, Paul Backus wrote: You should be able to get this information from the JSON output of `dub describe`. Cool, thanks. Much appreciated :-) Has anyone created any tools to condense that into a licensing report? No worries if not, just curious. Not that I know of. If you end up making one yourself, it might be worth posting in the Announce forum. Hi all, I played around with the idea and came up with a small dub package, that is not (yet) uploaded to the dub registry. Source is available at https://github.com/gizmomogwai/packageinfo, feedback very welcome. Kind regards, Christian
Re: Whats the proper way to write a Range next function
On 2022-06-15 19:36, JG wrote:
On Wednesday, 15 June 2022 at 17:30:31 UTC, JG wrote:
On Wednesday, 15 June 2022 at 13:52:24 UTC, Christian Köstlin wrote:
the naive version would look like
```d
auto next(Range)(Range r) {
r.popFront;
return r.front;
}
```
But looking at a mature library e.g.
https://github.com/submada/btl/blob/9cc599fd8495215d346ccd62d6e9f1f7ac140937/source/btl/vector/package.d#L229
is looks like there should be tons of annotations/attributes on it.
Kind regards,
Christian
I not sure of your use case. But you need to check if the range is
empty before and after calling popFront (and decide what to if it is).
Also unless you want the original range passed in mutated you should
call save (assuming you have a forward range) before calling your next
or you need to modify next so it calls save.
This is what I would write for next (which should only be called after
checking if the range is empty).
It produces a range whose front points at the next element.
```d
auto next(Range)(Range r) {
auto ret = r.save;
ret.popFront;
return ret;
}
```
Thanks for the explanation!
I actually want to mutate the original range and I know, that there are
at least 2 elements in my range.
e.g. the program
```d
import std.stdio;
import std.range;
auto next(Range)(Range r) {
r.popFront;
return r.front;
}
int main(string[] args) {
auto i = iota(10);
writeln(i.next);
writeln(i.next);
return 0;
}
```
prints:
```
1
1
```
whereas
```d
import std.stdio;
import std.range;
auto next(Range)(ref Range r) {
r.popFront;
return r.front;
}
int main(string[] args) {
auto i = iota(10);
writeln(i.next);
writeln(i.next);
return 0;
}
```
prints
```
1
2
```
which is what I would want. So it depends if the Range is a value or
reference type ...
Whats the proper way to write a Range next function
the naive version would look like
```d
auto next(Range)(Range r) {
r.popFront;
return r.front;
}
```
But looking at a mature library e.g.
https://github.com/submada/btl/blob/9cc599fd8495215d346ccd62d6e9f1f7ac140937/source/btl/vector/package.d#L229
is looks like there should be tons of annotations/attributes on it.
Kind regards,
Christian
Re: Templatized delegates
On 2022-05-31 23:15, Andrey Zherikov wrote:
I have tightly coupled code which I'd like to decouple but I'm a bit stuck.
For simplicity, I reduced the amount of code to something simple to
understand. So I have a struct `S` that has templated member function
that does something. On the other side I have delegate holder `R` - this
delegate takes `S` object as an argument and calls that function. My
goal is to remove dependency from `R` to `S`.
Here is code example:
```d
import std.stdio: writeln;
struct S
{
// function that should be called from delegate
void doSomething(T)(T value) { value.writeln; }
}
alias DG = void delegate (ref S s);
auto createDG(T)(T value)
{
return delegate (ref S s) { s.doSomething(value); };
}
struct R
{
DG dg;
this(int value)
{
dg = createDG(value);
}
}
void main()
{
auto r = R(5);
S s;
r.dg(s);
}
```
An obvious way is to add a type template parameter to `R`, `DG` and
`createDG` but I would like to avoid this. Is there another way to do so?
I think ideal solution would be having templatized delegate `void
delegate (S)(ref S s)` and then call `r.dg!S(s)` but it's not available:
`alias DG = void delegate (S) (ref S s)` gives unclear `Error: function
declaration without return type. (Note that constructors are always
named 'this')` message.Would it help to not create the delegate in R's constructor, but feed
the delegate into it (and create the delegate outside). This would also
resemble your description (R is a delegate holder) more.
Re: Tracing/Profiling D Applications
On 2022-05-29 23:00, Ali Çehreli wrote: On 5/29/22 13:53, Christian Köstlin wrote: > According to > https://www.schveiguy.com/blog/2022/05/comparing-exceptions-and-errors-in-d/ > its bad to catch Errors ... Correct in the sense that the program should not continue after catching Error. > so dowork should catch only Exception? It should catch Error as well. Otherwise you will have no idea why a thread disappeared. You may agree with me here: https://youtu.be/dRORNQIB2wA?t=1950 Thanks a lot for this explanation! Probably a try catch (Throwable) is a good thing to have in any thread that does not only use nothrows ... I catch'ed Throwable in the code just because it's a quick little experiment. > it might be better > to crash directly ... Sounds good but only after leaving the reason behind somehow. Ali Kind regards, Christian
Re: Tracing/Profiling D Applications
On 2022-05-29 23:08, Ali Çehreli wrote: On 5/29/22 13:47, Christian Köstlin wrote: > Our discussion with using TLS for the > collectors proposed to not need any lock on the add method for > collector, because its thread local and with that thread safe? It would be great that way but then the client changed the requirements on us: On 5/26/22 12:54, Christian Köstlin wrote: > I want to be able to dump tracings even while the program is still running. :p If the collected data were TLS, then the dumping thread should be able to ask each thread to provide data collected so far. That either requires synchronization, which I did, which necessitated shared Collector objects; or messaging, which would require each thread checking their Concurrency message box. I don't know... A third option came to me: Each thread periodically puts their data to a common location and dumper dumps whatever is aggregated up to that point. This would reduce contention. Thats also a nice idea, e.g. the collectors could aggregate their data and only propagate it once every second or every 10 seconds, or every 1000 events or something! Will think about that some more! Thanks a lot! Christian
Re: Execute the Shell command and continue executing the algorithm
On 2022-05-30 15:25, Ali Çehreli wrote:
On 5/30/22 04:18, Alexander Zhirov wrote:
> I want to run a command in the background
The closest is spawnShell:
import std.stdio;
import std.process;
import core.thread;
void main() {
auto pid = spawnShell(`(sleep 1 & echo SLEEP >> log)`);
Thread.sleep(5.seconds);
kill(pid);
writeln("Terminated with ", wait(pid));
}
I am not good at shell scripting but I had to change your && to & to see
anything in log.
I think this runs sleep 1 in the background and emits the echo directly.
As std.process documentation explains, the value returned by wait() (and
more) are platform dependent.
Ali
Re: Tracing/Profiling D Applications
On 2022-05-29 20:52, Ali Çehreli wrote:
On 5/27/22 06:55, Christian Köstlin wrote:
> I wonder how I can synchronize the "dumping" and the
> collection of the threads. Would be cool to have an efficient lockless
> implementation of appender ...
That turned out to be nontrivial.
The following is a draft I played with. Collector collects and Dumper
dumps. They use a SpinLock, an unpublished feature of core.internal for
locking. The implementation of spinlock (e.g. at
/usr/include/dlang/dmd/core/internal/spinlock.d) has a reference to
"test and test-and-set (TTAS)":
https://en.wikipedia.org/wiki/Test_and_test-and-set
I learned about TTAS from Rikki Cattermole yesterday at TeaConf. :)
The code is attached and works on my system.
Ali
import std;
import std.datetime.stopwatch;
import core.thread;
import core.atomic;
import core.internal.spinlock;
enum workerCount = 8;
enum threadRunTime = 4.seconds;
enum mainRunTime = threadRunTime + 1.seconds;
shared struct ScopeLock {
@disable this(this);
@disable void opAssign(ref const(typeof(this)));
SpinLock * lock;
this(shared(SpinLock) * lock) {
this.lock = lock;
lock.lock();
}
~this() {
lock.unlock();
}
}
struct Collector {
long[] data;
shared(SpinLock) lock;
auto scopeLock() shared {
return ScopeLock(&lock);
}
// Adds a data point to this collector.
void add(long i) shared {
auto sl = scopeLock();
/// Some crazy way of adding data points. Real code should
// make more sense.
data ~= i;
}
// Adds the data of this collector to the specified array
// array. Again, real code should use a more sophisticated
// method.
void aggregate(ref long[] where) shared {
auto sl = scopeLock();
where ~= data.sum;
data.length = 0;
(cast(long[])data).assumeSafeAppend();
}
}
// A variable to help us trust the code. We will print this at
// the end of main.
long allThatHasBeenDumped = 0;
// Used only for validating the code.
shared long allCollectedByThreads;
synchronized class Dumper {
private:
shared(Collector)*[] collectors;
void register(shared(Collector) * collector) shared {
writeln("registering ", collector);
collectors ~= collector;
}
// Dumps current results.
void dump(File output) shared {
long[] data;
foreach (collector; collectors) {
collector.aggregate(data);
}
const allData = data.sum;
if (allData != 0) {
stdout.writefln!"Just collected:%-(\n %,s%)"(data);
allThatHasBeenDumped += allData;
}
}
}
shared(Dumper) dumper;
shared static this() {
writeln("Making a Dumper");
dumper = new Dumper();
}
shared(Collector) * collector;
static this() {
writeln("Making a Collector");
collector = new shared(Collector)();
dumper.register(cast(shared)collector);
}
// Main thread function
void doWork() {
try {
doWorkImpl();
} catch (Throwable exc) {
stderr.writeln("Caught Throwable: ", exc.msg);
}
}
// The implementation of each thread.
void doWorkImpl() {
auto sw = StopWatch();
sw.start();
long i = 0;
while (sw.peek < threadRunTime) {
(cast(shared)collector).add(i);
++i;
}
--i;
auto total = i * (i + 1) / 2;
writefln("Thread collected %,s items equaling %,s with %s",
i, total, collector);
atomicOp!"+="(allCollectedByThreads, total);
}
void main() {
writeln("main started");
iota(workerCount).each!(_ => spawn(&doWork));
auto sw = StopWatch();
sw.start();
while (sw.peek < mainRunTime) {
dumper.dump(stdout);
Thread.sleep(100.msecs);
}
// One final collection (and dump):
dumper.dump(stdout);
assert(allThatHasBeenDumped == allCollectedByThreads);
}
According to
https://www.schveiguy.com/blog/2022/05/comparing-exceptions-and-errors-in-d/
its bad to catch Errors ... so dowork should catch only Exception? Or is
this a special case to just log the error per thread and be done with
it? still if not everything is cleaned up correctly it might be better
to crash directly ...
Kind regards,
Christian
Re: Tracing/Profiling D Applications
On 2022-05-29 20:52, Ali Çehreli wrote:
On 5/27/22 06:55, Christian Köstlin wrote:
> I wonder how I can synchronize the "dumping" and the
> collection of the threads. Would be cool to have an efficient lockless
> implementation of appender ...
That turned out to be nontrivial.
The following is a draft I played with. Collector collects and Dumper
dumps. They use a SpinLock, an unpublished feature of core.internal for
locking. The implementation of spinlock (e.g. at
/usr/include/dlang/dmd/core/internal/spinlock.d) has a reference to
"test and test-and-set (TTAS)":
https://en.wikipedia.org/wiki/Test_and_test-and-set
I learned about TTAS from Rikki Cattermole yesterday at TeaConf. :)
The code is attached and works on my system.
Ali
import std;
import std.datetime.stopwatch;
import core.thread;
import core.atomic;
import core.internal.spinlock;
enum workerCount = 8;
enum threadRunTime = 4.seconds;
enum mainRunTime = threadRunTime + 1.seconds;
shared struct ScopeLock {
@disable this(this);
@disable void opAssign(ref const(typeof(this)));
SpinLock * lock;
this(shared(SpinLock) * lock) {
this.lock = lock;
lock.lock();
}
~this() {
lock.unlock();
}
}
struct Collector {
long[] data;
shared(SpinLock) lock;
auto scopeLock() shared {
return ScopeLock(&lock);
}
// Adds a data point to this collector.
void add(long i) shared {
auto sl = scopeLock();
/// Some crazy way of adding data points. Real code should
// make more sense.
data ~= i;
}
// Adds the data of this collector to the specified array
// array. Again, real code should use a more sophisticated
// method.
void aggregate(ref long[] where) shared {
auto sl = scopeLock();
where ~= data.sum;
data.length = 0;
(cast(long[])data).assumeSafeAppend();
}
}
// A variable to help us trust the code. We will print this at
// the end of main.
long allThatHasBeenDumped = 0;
// Used only for validating the code.
shared long allCollectedByThreads;
synchronized class Dumper {
private:
shared(Collector)*[] collectors;
void register(shared(Collector) * collector) shared {
writeln("registering ", collector);
collectors ~= collector;
}
// Dumps current results.
void dump(File output) shared {
long[] data;
foreach (collector; collectors) {
collector.aggregate(data);
}
const allData = data.sum;
if (allData != 0) {
stdout.writefln!"Just collected:%-(\n %,s%)"(data);
allThatHasBeenDumped += allData;
}
}
}
shared(Dumper) dumper;
shared static this() {
writeln("Making a Dumper");
dumper = new Dumper();
} >
shared(Collector) * collector;
static this() {
writeln("Making a Collector");
collector = new shared(Collector)();
dumper.register(cast(shared)collector);
}
// Main thread function
void doWork() {
try {
doWorkImpl();
} catch (Throwable exc) {
stderr.writeln("Caught Throwable: ", exc.msg);
}
}
// The implementation of each thread.
void doWorkImpl() {
auto sw = StopWatch();
sw.start();
long i = 0;
while (sw.peek < threadRunTime) {
(cast(shared)collector).add(i);
++i;
}
--i;
auto total = i * (i + 1) / 2;
writefln("Thread collected %,s items equaling %,s with %s",
i, total, collector);
atomicOp!"+="(allCollectedByThreads, total);
}
void main() {
writeln("main started");
iota(workerCount).each!(_ => spawn(&doWork));
auto sw = StopWatch();
sw.start();
while (sw.peek < mainRunTime) {
dumper.dump(stdout);
Thread.sleep(100.msecs);
}
// One final collection (and dump):
dumper.dump(stdout);
assert(allThatHasBeenDumped == allCollectedByThreads);
}
Hi Ali,
thanks a lot for that, I will first have to digest that.
Just one first question: Our discussion with using TLS for the
collectors proposed to not need any lock on the add method for
collector, because its thread local and with that thread safe?
Kind regards,
Christian
Re: Tracing/Profiling D Applications
On 2022-05-26 22:19, Ali Çehreli wrote: On 5/26/22 12:54, Christian Köstlin wrote: > I want to be able to dump > tracings even while the program is still running. Then I would have to > collect the tls data of all still running threads. I am not sure without testing but I am under the impression that mutexes can be very slow especially if the data collection is happening very frequently. The threads may not have breathing room to perform their tasks. Since dumping would be much slower compared to the performance penalty of grabbing a mutex that it can be measured as zero-cost in the dumping time. :) So, you may still want to collect per-thread and combine incrementally as you dump? Perhaps a premature optimization... Please test. :) You are certainly right.. It really depends on the usecase. I also think my current approach is not good to e.g. do syscall tracing of something like read. I wonder how I can synchronize the "dumping" and the collection of the threads. Would be cool to have an efficient lockless implementation of appender ... If you want to combine before dumping, then collector registration may be needed so that the dumper can contact each collector and say "please add your data". > __gshared does not put > the shared type attribute the the profiler. Correct. thanks! > If I use shared I would also > have to make all methods I call on the shared profiler shared and also > would have to have a shared appender (or cast shared away)? Possibly. :/ Ali --- Christian
Re: Tracing/Profiling D Applications
On 2022-05-25 23:56, Ali Çehreli wrote: On 5/25/22 14:35, Christian Köstlin wrote: > 1. I went for a singleton for storing tracing/logging information that > needs to be initialized manually. Is __gshared the right way to do that? I think this is where thread-local storage comes in handy. As the D runtime does for dmd's -profile command line switch, you can have a module-level Appender, which collects data for that thread without worrying about race conditions. Nice idea. Similar to the fast singleton here: https://p0nce.github.io/d-idioms/#Leveraging-TLS-for-a-fast-thread-safe-singleton Not sure if this will work for me though, as I want to be able to dump tracings even while the program is still running. Then I would have to collect the tls data of all still running threads. > to unlock the mutex? You need the mutex only when each thread is exiting to combine their results. Each static ~this() could grab the mutex and add its result to the one global Appender. And that "global" Appender would be 'shared' or __gshared; doesn't matter. Thanks for the explanation. From my understanding __gshared does not put the shared type attribute the the profiler. If I use shared I would also have to make all methods I call on the shared profiler shared and also would have to have a shared appender (or cast shared away)? Kind regards, Christian
Re: Tracing/Profiling D Applications
On 2022-05-26 01:05, frame wrote: On Wednesday, 25 May 2022 at 21:35:07 UTC, Christian Köstlin wrote: Is there also a way to get the "real" threadid? I'm using that functions inside threads: core.sys.windows.winbase.GetCurrentThreadId on Windows core.sys.posix.pthread.pthread_self on Unix (implied pthreads are used)Thanks, I will give those a try!
Tracing/Profiling D Applications
I experimented with application level tracing/profiling of d applications similar to what is described in https://dlang.org/blog/2020/03/13/tracing-d-applications/ as the "writef-based approach". Only difference is, that I am emitting json (https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/edit#heading=h.lpfof2aylapb) that is compatible with https://ui.perfetto.dev/ which gives nice interactive insights of the data. The code is available at https://github.com/gizmomogwai/profiled and can be used as dub package by registering it locally with dub add-local. The github repository also contains one screenshot produced with a patched unit-threaded library to emit the tracing information on a small project of mine. I do have a few question on how to do things in dlang though: 1. I went for a singleton for storing tracing/logging information that needs to be initialized manually. Is __gshared the right way to do that? 2. The data is stored in a std.array.appender protected by a mutex and only later printed as json. Any way to do this more elegantly than with the scope(exit) constructs to unlock the mutex? 3. For the tracing/profiling events I went with subclasses of an abstract event class (I only implemented one kind of event, whereas the tracing json format supports a lot more scenarios). When one is interested in when something happens and how long it takes, one can call `public Unique!CompleteEventProcess start(string name)` on the profiler which an uniquepointer that is then automatically deleted if it goes out of scope (and then takes the time). Is this a reasonable way todo it? E.g. it would be an error if the unique ptr is passed to another thread, because the CompleteEvents of the tracing format expect to be starting and stopping on the same thread. 4. Would something like that be interesting for others as a dub package (ATM its so small, that having it as an external dependency is not worth it IMHO)? 5. The complete event normally expects the real process and thread id as used by the os. Looks like https://dlang.org/phobos/std_process.html#.Pid.processID delivers the process id. Is there also a way to get the "real" threadid? Thanks in advance, Christian
Re: Help with DynamicArray of Emsi Containers
On 2022-05-01 09:12, vit wrote: DynamicArray has disabled postblit (is not copyable). Package autoptr is deprecated (internaly redirected to btl:atuoptr), all functionality is moved to package [BTL](https://code.dlang.org/packages/btl) (subpackage btl:autoptr). This library contains subpackage btl:vector with functionality like DynamicArray with support for copying. I use IntrusivePtr inside Vector/SmallVector and i have no problems. Thanks a lot for those pointers, I changed/upgraded my dependencies, and it already looks better.
Help with DynamicArray of Emsi Containers
I am struggling with initializing an Emsi Containers DynamicArray in a
nice way. Some background information of my usecase:
I experimenting in porting some old 3d engine code of mine from c++ to
dlang. In the engine I want to exactly control when resources are freed
and not rely on the garbage collector. For that I am using reference
counters courtesy of autoptr.
e.g.
```d
alias Texture = IntrusivePtr!TextureData; // dub package autoptr
class TextureData
{
IFImage image; // dub package imagefmt
}
```
another class of mine now is supposed to hold several of those textures
for multitexturing and should be initialized in its constructor:
```d
class Appearance
{
Texture[] textures;
this(Texture[] textures)
{
this.textures = textures;
}
}
```
Unfortunately this does not work properly together with autoptr's (see
https://github.com/submada/autoptr/issues/5#issuecomment-997683868).
Because of that I am using now DynamicArrays from Emsi Containers, but I
am having trouble initializing them:
```d
class Apperance
{
DynamicArray!Texture textures;
this(DynamicArray!Texture textures)
{
this.textures = textures;
}
}
```
does not compile.
What would be the best way to initialize the variable?
Kind regards,
Christian
Re: Offline D documentation/tutorial
On 2022-02-13 01:16, LorenDB wrote: Is there a way to download tour.dlang.org, the D spec, and/or the Phobos spec as an offline HTML site? I like the ability of cppreference.com to be saved as an offline HTML archive and I'd like to have that for D as well. In addition to the already mentioned solutions there are at least two more: 1. I was pointed by Seb to https://devdocs.io/d/ which offers an offline mode via html5. 2. Another nice thing for dlang would be to support dash docsets (https://kapeli.com/dash). There is a free opensource reader for those as well (https://zealdocs.org/). https://github.com/trishume/d-phobos-docset tried to create such docsets with a small script, but thats not up2date. All in all I think there are almost to many half baked solutions to the offline documentation problem in dlang. Perhaps this could be something for the steering committee. Kind regards, Christian
Re: std.concurrency and const
On 2022-01-06 02:55, frame wrote: On Wednesday, 5 January 2022 at 22:22:19 UTC, Christian Köstlin wrote: Hi all, I really like std.concurrency but I now stumbled upon the following. When receiving messages as const, they also need to be sent as const (otherwise they are not matched). Comparing this to normal function calls I would expect a different behavior. They do. You have an error: `Message` doesn't match with `(Tid sender, const(Message) m)`. You don't get the argument `sender` here. Duh ... you are right, my mistake!!! Thanks! Christian
std.concurrency and const
Hi all,
I really like std.concurrency but I now stumbled upon the following.
When receiving messages as const, they also need to be sent as const
(otherwise they are not matched). Comparing this to normal function
calls I would expect a different behavior.
```d
import std.concurrency;
import std.stdio;
struct Message
{
}
void fun(const(Message) m) {
writeln("fun(const(Message))");
}
/*
void fun(Message m) {
writeln("fun(Message)");
}
*/
void receiver()
{
receive(
(Tid sender, const(Message) m)
{
writeln("received const(Message)");
},
(const(Variant) v)
{
writeln("Received const(Variant)", v);
},
(Variant v)
{
writeln("Received variant", v);
},
);
writeln("done");
}
int main(string[] args)
{
auto blubTid = spawnLinked(&receiver);
blubTid.send(Message());
receive(
(LinkTerminated t)
{
}
);
fun(Message());
fun(const(Message)());
return 0;
}
```
output is something like:
```
Received variantMessage()
done
fun(const(Message))
fun(const(Message))
```
whereas I would expect
```
received const(Message)
done
fun(const(Message))
fun(const(Message))
```
Looking forward for the explanation of that.
Kind regards and happy 2022!
Christian
Re: Is it possible to exchange the linker with a dub project to use mold
On 2021-12-22 16:28, Paul Backus wrote: On Wednesday, 22 December 2021 at 15:20:15 UTC, Christian Köstlin wrote: https://github.com/rui314/mold Kind regards, Christian This was recently discussed in the "General" forum: https://forum.dlang.org/thread/fiyfgqykhdmglqypx...@forum.dlang.org Thanks for the pointer. Very interesting reads in the bugreports! Kind regards, Christian
Is it possible to exchange the linker with a dub project to use mold
https://github.com/rui314/mold Kind regards, Christian
Re: Display a random image with vibe.d
On 2021-06-20 17:14, vnr wrote:
On Sunday, 20 June 2021 at 14:28:26 UTC, jfondren wrote:
On Sunday, 20 June 2021 at 13:58:22 UTC, vnr wrote:
Thanks for the answers, I understand better what is going on.
So, what should I do to make my server respond with a random image,
and not the random image page? I'm fairly new to vibe.d, so I don't
yet know the intricacies of how to handle this style of thing and I
couldn't find how to do it in the documentation.
Thank you.
Responding with a random image is an option, but what you have is
a good start, you just need to also serve images on request.
I'm very distracted right now or I would've included more in my
earlier post, but Vibe's online documentation has what you need.
Try, especially, https://vibed.org/docs#http-routing
You want a route for the random-image page, and you want to serve
static files under images/
Great, thanks a lot, it works as expected! Here is the code used
(app.d), for those who are interested:
```
import vibe.vibe;
void main()
{
auto router = new URLRouter;
router.get("/", &index);
router.get("*", serveStaticFiles("public"));
auto settings = new HTTPServerSettings;
settings.bindAddresses = ["::1", "127.0.0.1"];
settings.port = 8080;
auto listener = listenHTTP(settings, router);
scope (exit) listener.stopListening();
logInfo("Please open http://127.0.0.1:8080/ in your browser.");
runApplication();
}
/// The index page
void index(HTTPServerRequest req, HTTPServerResponse res)
{
import std.random;
import std.format;
auto rnd = Random(unpredictableSeed);
const auto rndimg = format("/images/rndimg/img%d.jpg",
uniform(1, 27, rnd));
res.render!("index.dt", req, rndimg);
}
```
The Random object should only be created once, but here its created for
every request.
1. It is relatively slow to reinitialize it
2. If you really want to have uniform distribution its probably better
to not throw the UniformRandomNumberGenerator generator away.
You can simplify that by using just uniform(1, 27) (it should fallback
to the default rndGen that is initialized per thread
(https://dlang.org/library/std/random/rnd_gen.html).
here a benchmark of both functions:
```
import std;
import std.datetime.stopwatch;
auto onlyUniform() {
return uniform(1, 27);
}
auto withNewRandom() {
auto rnd = Random(unpredictableSeed);
return uniform(1, 27, rnd);
}
void main() {
100_000.benchmark!(onlyUniform, withNewRandom).writeln;
}
```
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
On 2021-05-31 18:50, Christian Köstlin wrote:
On 2021-05-31 13:40, CandG wrote:
On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:
On 5/27/21 10:13 AM, Christian Köstlin wrote:
P.S.: I still do not get how to post formatted snippets with
thunderbird to the newsgroup/forum :/
It's not possible currently.
I no longer use thunderbird, but:
-
https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac09bf40a933b851e5R338
- IIRC thunderbird allows changing headers: try adding
"Content-Type" to the comma-separated list "mail.compose.other.header"
- Then in the composition window make sure Content-Type is set to
something like "text/plain; markup=markdown"
Thanks for the tip, lets see if it works:
```D
void main(string[] args) {
writeln("Hello World");
}
```
Kind regards,
Christian
another try.
```D
void main(string[] args) {
writeln("Hello World");
}
```
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
On 2021-05-31 13:40, CandG wrote:
On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:
On 5/27/21 10:13 AM, Christian Köstlin wrote:
P.S.: I still do not get how to post formatted snippets with
thunderbird to the newsgroup/forum :/
It's not possible currently.
I no longer use thunderbird, but:
-
https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac09bf40a933b851e5R338
- IIRC thunderbird allows changing headers: try adding "Content-Type"
to the comma-separated list "mail.compose.other.header"
- Then in the composition window make sure Content-Type is set to
something like "text/plain; markup=markdown"
Thanks for the tip, lets see if it works:
```D
void main(string[] args) {
writeln("Hello World");
}
```
Kind regards,
Christian
Re: where do I find the complete phobos function list names ?
On 2021-05-26 01:46, Paul Backus wrote: On Tuesday, 25 May 2021 at 22:05:16 UTC, someone wrote: I was unsuccessfully searching the site for them in the form of a master index to begin with. I need them, in plain text, in order to add them to a VIM custom syntax highlight plugin I already made which I am already using but is lacking phobos support. Can anyone point me to the right place please ? There is no global index in the online documentation; the best you can get is an index of each module. If you really want this, your best bet is probably to run a source-code indexer like `ctags` on the Phobos source tree, and do some scripting to transform the results into something usable in your Vim plugin. e.g. I found this file https://dlang.org/library/symbols.js which is used e.g. by https://dlang.org/library/std/algorithm/sorting/sort.html to implement the search. Perhaps that helps. Kind regards, Christian
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
On 2021-05-27 18:56, Ali Çehreli wrote:
On 5/27/21 9:19 AM, Ali Çehreli wrote:
auto result = new string[users.length];
users.enumerate.parallel.each!(en => result[en.index] =
servers.doSomething(en.value));
writeln(result);
I still like the foreach version more:
auto result = new string[users.length];
foreach (i, user; users.parallel) {
result[i] = servers.doSomething(user);
}
writeln(result);
Ali
Hi Ali,
both of those variants do work for me, thanks a lot!
Still not sure which I prefer (almost too many options now :) ).
I am so happy that I asked in this forum, help is much appreciated!
Christian
Re: where do I find the complete phobos function list names ?
On 2021-05-26 01:46, Paul Backus wrote: On Tuesday, 25 May 2021 at 22:05:16 UTC, someone wrote: I was unsuccessfully searching the site for them in the form of a master index to begin with. I need them, in plain text, in order to add them to a VIM custom syntax highlight plugin I already made which I am already using but is lacking phobos support. Can anyone point me to the right place please ? There is no global index in the online documentation; the best you can get is an index of each module. If you really want this, your best bet is probably to run a source-code indexer like `ctags` on the Phobos source tree, and do some scripting to transform the results into something usable in your Vim plugin. Where is the index for the search functionality on dlang.org located? Could that be used? Kind regards, Christian
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
On 2021-05-27 15:00, sighoya wrote:
On Thursday, 27 May 2021 at 12:58:28 UTC, Christian Köstlin wrote:
That looks nice, but unfortunately my data for servers and users in
the real world is not static but comes from a config file.
Okay, but then parametrizing the static lambda with runtime parameters
should work. The important fact is that the closure needs to be static.
Ah thanks, now I understand.
So what I came up with now is a combination of the things mentioned:
```D
import std;
string doSomething(string[] servers, string user) {
return user ~ servers[0];
}
struct UserWithServers {
string user;
string[] servers;
}
void main(string[] args) {
auto servers = args;
auto users = ["u1", "u2", "u3"];
auto usersWithServers = users.map!(user => UserWithServers(user,
servers)).array;
static fn = function(UserWithServers user) =>
user.servers.doSomething(user.user);
writeln(taskPool.amap!(fn)(usersWithServers));
}
```
Making also the example a little bit more "realistic" by using dynamic
data for servers.
I would like to use auto fn, but somehow saying that its a function is
not enough for dmd. From my understanding a function would never need a
context?!?
Thanks a lot!
Christian
P.S.: I still do not get how to post formatted snippets with thunderbird
to the newsgroup/forum :/
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
On 2021-05-27 14:48, sighoya wrote:
On Thursday, 27 May 2021 at 12:17:36 UTC, Christian Köstlin wrote:
Can you explain me, where here a double context is needed? Because all
data now should be passed as arguments to amap?
Kind regards,
Christian
I believe D's type system isn't smart enough to see independence
between context and closure, otherwise your original example would also
work as users and servers are context independent.
What about:
```D
string doSomething(string[] servers, string user) {
return user ~ servers[0];
}
void main() {
static servers = ["s1", "s2", "s3"];
static users = ["u1", "u2", "u3"];
static lambda = (string user) => servers.doSomething(user);
writeln(map!(user => servers.doSomething(user))(users));
writeln(taskPool.amap!(lambda)(users));
}
```
That looks nice, but unfortunately my data for servers and users in the
real world is not static but comes from a config file.
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
On 2021-05-27 13:11, sighoya wrote:
On Thursday, 27 May 2021 at 09:58:40 UTC, Christian Köstlin wrote:
I have this small program here
test.d:
```
import std;
string doSomething(string[] servers, string user) {
return user ~ servers[0];
}
void main() {
auto servers = ["s1", "s2", "s3"];
auto users = ["u1", "u2", "u3"];
writeln(map!(user => servers.doSomething(user))(users));
writeln(taskPool.amap!(user => servers.doSomething(user))(users));
}
```
I think it relates to https://issues.dlang.org/show_bug.cgi?id=5710
The reason is that amap requires a this pointer of type TaskPool and a
context pointer to the closure which belongs to main, at least because
it requires servers. Having both isn't possible due to problems in non
DMD compilers.
If you rewrite it more statically:
```D
string doSomething(string[] servers, string user) {
return user ~ servers[0];
}
string closure(string user)
{
return servers.doSomething(user);
}
auto servers = ["s1", "s2", "s3"];
int main()
{
auto users = ["u1", "u2", "u3"];
writeln(map!(user => servers.doSomething(user))(users));
writeln(taskPool.amap!(closure)(users));
return 0;
}
```
PS: Just enable markdown if you want to highlight D code
On a second not I needed to make server __gshared in my real program, as
otherwise its a thread local variable (in the small demo program, this
did not occur, I guess because the parallel operations we're too fast).
Kind regards,
Christian
Re: How to work around the infamous dual-context when using delegates together with std.parallelism
Thanks for the proposed solution. It also works in my slightly bigger
program (although I do not like to make servers more global).
I tried also the following (which unfortunately also does not work as
intended):
```D
import std;
string doSomething(string[] servers, string user) {
return user ~ servers[0];
}
int main()
{
auto users = ["u1", "u2", "u3"];
auto servers = ["s1", "s2", "s3"];
auto usersWithServers = users.map!(user => tuple!("user",
"servers")(user, servers)).array;
writeln(map!(userWithServers =>
userWithServers.servers.doSomething(userWithServers.user))(usersWithServers));
writeln(taskPool.amap!(userWithServers =>
userWithServers.servers.doSomething(userWithServers.user))(usersWithServers));
return 0;
}
```
Here I try to put the data I need together into one tuple ("manually")
and then pass it all to amap. Can you explain me, where here a double
context is needed? Because all data now should be passed as arguments to
amap?
Kind regards,
Christian
