Re: Simplest multithreading example
On Friday, 1 September 2017 at 20:02:23 UTC, ag0aep6g wrote:
On 09/01/2017 07:27 AM, Brian wrote:
double [] hugeCalc(int i){
// Code that takes a long time
}
so if I do
double[][int] _hugeCalcCache;
foreach(i ; I)
_hugeCalcCache[i] = hugeCalc(i);
of course the required time is I.length * (a long time), so I
wanted to shorten this by splitting to different threads :
foreach(i ; parallel(I) )
_hugeCalcCache[i] = hugeCalc(i);
but as you can guess, it doesn't work that easily.
Works pretty well for me:
double [] hugeCalc(int i)
{
// Code that takes a long time
import core.thread: Thread;
import std.datetime: seconds;
Thread.sleep(1.seconds);
return [i];
}
void main()
{
static import std.range;
import std.parallelism: parallel;
auto I = std.range.iota(0, 10);
double[][int] _hugeCalcCache;
foreach(i ; parallel(I))
_hugeCalcCache[i] = hugeCalc(i);
}
That runs in about 3 seconds here. The serial version would of
course take about 10 seconds. So, parallelism achieved!
Though I don't know if it's safe to access an associative array
concurrently like that. I'd use a normal dynamic array instead
and initialize it before going parallel:
auto _hugeCalcCache = new double[][](10);
Thanks very much for your help, I finally had time to try your
suggestions. The initial example you showed does indeed have the
same problem of not iterating over all values :
double [] hugeCalc(int i){
// Code that takes a long time
import core.thread: Thread;
import std.datetime: seconds;
Thread.sleep(1.seconds);
return [i];
}
static import std.range;
import std.parallelism: parallel;
auto I = std.range.iota(0, 100);
double[][int] _hugeCalcCache;
foreach(i ; parallel(I))
_hugeCalcCache[i] = hugeCalc(i);
writeln( _hugeCalcCache.keys ); // this is some random
subset of (0,100)
but this does seem to work using your other method of
initialization :
auto _hugeCalcCache = new double[][](100);
foreach(i ; parallel(I))
_hugeCalcCache[i] = hugeCalc(i);
foreach( double[] x ; _hugeCalcCache)
writeln( x ); // this now contains all values
so I guess initializing the whole array at compile time makes it
thread safe ?
(The second case runs in 16 seconds on my computer.)
Anyways it seems to work, thanks again !
Re: SIMD under LDC
On Monday, 4 September 2017 at 23:06:27 UTC, Nicholas Wilson
wrote:
On Monday, 4 September 2017 at 20:39:11 UTC, Igor wrote:
I found that I can't use __simd function from core.simd under
LDC
Correct LDC does not support the core.simd interface.
and that it has ldc.simd but I couldn't find how to implement
equivalent to this with it:
ubyte16* masks = ...;
foreach (ref c; pixels) {
c = __simd(XMM.PSHUFB, c, *masks);
}
I see it has shufflevector function but it only accepts
constant masks and I am using a variable one. Is this possible
under LDC?
You have several options:
* write a regular for loop and let LDC's optimiser take care of
the rest.
alias mask_t = ReturnType!(equalMask!ubyte16);
pragma(LDC_intrinsic, "llvm.masked.load.v16i8.p0v16i8")
ubyte16 llvm_masked_load(ubyte16* val,int align, mask_t
mask, ubyte16 fallthru);
ubyte16* masks = ...;
foreach (ref c; pixels) {
auto mask = equalMask!ubyte16(*masks, [-1,-1,-1, ...]);
c = llvm_masked_load(,16,mask, [0,0,0,0 ... ]);
}
The second one might not work, because of type differences in
llvm, but should serve as a guide to hacking the `cmpMask` IR
code in ldc.simd to do what you want it to.
BTW. Shuffling channels within pixels using DMD simd is about
5 times faster than with normal code on my machine :)
Don't underestimate ldc's optimiser ;)
I seen cases where the compiler fail to optimized for smid.
Re: SIMD under LDC
On Monday, 4 September 2017 at 20:39:11 UTC, Igor wrote:
I found that I can't use __simd function from core.simd under
LDC
Correct LDC does not support the core.simd interface.
and that it has ldc.simd but I couldn't find how to implement
equivalent to this with it:
ubyte16* masks = ...;
foreach (ref c; pixels) {
c = __simd(XMM.PSHUFB, c, *masks);
}
I see it has shufflevector function but it only accepts
constant masks and I am using a variable one. Is this possible
under LDC?
You have several options:
* write a regular for loop and let LDC's optimiser take care of
the rest.
alias mask_t = ReturnType!(equalMask!ubyte16);
pragma(LDC_intrinsic, "llvm.masked.load.v16i8.p0v16i8")
ubyte16 llvm_masked_load(ubyte16* val,int align, mask_t mask,
ubyte16 fallthru);
ubyte16* masks = ...;
foreach (ref c; pixels) {
auto mask = equalMask!ubyte16(*masks, [-1,-1,-1, ...]);
c = llvm_masked_load(,16,mask, [0,0,0,0 ... ]);
}
The second one might not work, because of type differences in
llvm, but should serve as a guide to hacking the `cmpMask` IR
code in ldc.simd to do what you want it to.
BTW. Shuffling channels within pixels using DMD simd is about 5
times faster than with normal code on my machine :)
Don't underestimate ldc's optimiser ;)
Re: replace switch for mapping
On Monday, 4 September 2017 at 09:23:24 UTC, Andrea Fontana wrote: On Thursday, 31 August 2017 at 23:17:52 UTC, EntangledQuanta wrote: Generally one has to use a switch to map dynamic components. Given a set X and Y one can form a switch to map X to Y: [...] Does this work for you? https://dpaste.dzfl.pl/e2669b595539 Andrea No, do you realize you are passing those enums at compile time? It won't work if they are "runtime" variables, which is the whole point of doing all this. You've essentially make a simple problem complicated. Why not just overload foo properly?
SIMD under LDC
I found that I can't use __simd function from core.simd under LDC
and that it has ldc.simd but I couldn't find how to implement
equivalent to this with it:
ubyte16* masks = ...;
foreach (ref c; pixels) {
c = __simd(XMM.PSHUFB, c, *masks);
}
I see it has shufflevector function but it only accepts constant
masks and I am using a variable one. Is this possible under LDC?
BTW. Shuffling channels within pixels using DMD simd is about 5
times faster than with normal code on my machine :)
Re: Making a repo of downloaded dub package
On Monday, 4 September 2017 at 14:35:47 UTC, Dukc wrote: Bump Search for word "local" here: https://code.dlang.org/docs/commandline. Maybe some of those can help you. If not you could make a pull request for dub that would support such a thing :)
Re: Using closure causes GC allocation
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote: In order to resolve the issue "Using closure causes GC allocation" it was stated that we need to use delegates Alternatively you can drop the functional style and use a foreach loop that doesn't require delegates, but you'd still need the GC to store the result in an array. And even then you could still use Array (from std.container).
Re: Making a repo of downloaded dub package
Bump
Re: Returning multiple values from a function
On Monday, 4 September 2017 at 09:22:25 UTC, Vino.B wrote:
Output :
1
2
["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
Required Output:
Test1 = 1
Test2 = 2
Path = ["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
From,
Vino.B
If you just need it to be displayed then you can add it to the
writeln call : writeln("Test 1 = ", Params[0]);
But if you want to do it dynamically then maybe you should be
using an associative array or a tuple of tuples :
import std.stdio;
import std.array;
import std.typecons;
alias Result = Tuple!(
Tuple!(string, int),
Tuple!(string, int),
Tuple!(string, string[])
);
Result Params () {
int Test1;
int Test2;
string[] File1;
string[] File2;
auto Path = appender!(string[]); //string[] path;
path.reserve(2) ?
Test1 = 1;
Test2 = 2;
File1 = ["C:\\Temp\\TEAM1\\BACKUP"];
File2 = ["C:\\Temp\\TEAM2\\ARCHIVE"];
Path ~= File1;
Path ~= File2;
return tuple(
tuple("Test1", Test1),
tuple("Test2", Test2),
tuple("Path", Path.data)
);
}
void main (){
auto res = Params();
writeln(res[0][0], " = ", res[0][1]);
writeln(res[1][0], " = ", res[1][1]);
writeln(res[2][0], " = ", res[2][1]);
}
You can also loop through res :
void main (){
auto res = Params();
foreach(r; res)
writeln(r[0], " = ", r[1]);
}
Re: Returning multiple values from a function
On Monday, 4 September 2017 at 09:22:25 UTC, Vino.B wrote:
Thank you very much, i have used your idea and was able to
resolve, and i need one more favor. the below code outputs the
value but i need the name of the variable + value as below.
Output :
1
2
["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
Required Output:
Test1 = 1
Test2 = 2
Path = ["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
For fixed names case you can hardcode it:
writeln("Test1 = ", Params[0]);
writeln("Test2 = ", Params[1]);
writeln("Path = ", Params[2]);
You can't print the actual name of the variable used in tuple
constructing because tuple doesn't store it. More of this, the
tuple can be constructed from expression without a name, so it's
impossible in common case.
Re: replace switch for mapping
On Thursday, 31 August 2017 at 23:17:52 UTC, EntangledQuanta wrote: Generally one has to use a switch to map dynamic components. Given a set X and Y one can form a switch to map X to Y: [...] Does this work for you? https://dpaste.dzfl.pl/e2669b595539 Andrea
Re: Returning multiple values from a function
On Monday, 4 September 2017 at 07:40:23 UTC, crimaniak wrote:
On Monday, 4 September 2017 at 07:27:12 UTC, Vino.B wrote:
Hi,
Can you help me in how to return multiple values from a
function, the below code is throwing an error as below
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
Tuple!(int, string[]) Params () {
return tuple(1, ["C:\\Temp\\TEAM1\\BACKUP",
"C:\\Temp\\TEAM2\\ARCHIVE"]);
}
void main (){
Params.writeln;
}
Hi,
Thank you very much, i have used your idea and was able to
resolve, and i need one more favor. the below code outputs the
value but i need the name of the variable + value as below.
Output :
1
2
["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
Required Output:
Test1 = 1
Test2 = 2
Path = ["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"]
Program:
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
import std.array: appender;
Tuple!(int,int, string[]) Params () {
int Test1;
int Test2;
string[] File1;
string[] File2;
auto Path = appender!(string[]);
Test1 = 1;
Test2 = 2;
File1 = ["C:\\Temp\\TEAM1\\BACKUP"];
File2 = ["C:\\Temp\\TEAM2\\ARCHIVE"];
Path ~= File1;
Path ~= File2;
return tuple (Test1, Test2, Path.data);
}
void main (){
writeln(Params[0]);
writeln(Params[1]);
writeln(Params[2]);
}
return tuple (Test1, Test1Test2, Path.data);
}
void main (){
writeln(Params[0]);
writeln(Params[1]);
}
From,
Vino.B
Re: Bug in D!!!
On Wednesday, 30 August 2017 at 20:47:12 UTC, EntangledQuanta
wrote:
interface I
{
void Go(T)(S!T s);
static final I New()
{
return new C();
}
}
abstract class A : I
{
}
class C : A
{
void Go(T)(S!T s)
{
}
}
This is a blocker for me! Can someone open a ticket?
Judging by the length of the thread that I did not read, the
real problem was not spotted, otherwise, it would be shorter. The
problem is called "virtual method in the interface" anti-pattern.
Just never do that, and life will be easier. In this case, I
recommend to move Go to A and make it just dispatcher for
specialized private non-templated virtual functions. You don't
need all this mess with string templates for it.
Re: Dub documentation with an initial ddoc file
On Sunday, 3 September 2017 at 23:14:15 UTC, Conor O'Brien wrote:
I've been trying to figure out how to generate documentation
for my project using dub. I have found this link[1] which told
me how I could use dub to generate docs:
dub build --build=docs
However, I wish to have a set of macros that are present on
every documentation file, that would define how the resultant
HTML document is rendered. I tried:
dub build --build=docs html.ddoc
But I got the following error:
Expected one or zero arguments.
Run "dub build -h" for more information about the "build"
command.
How might I include `html.ddoc` with every file that has
documentation?
Add this to your dub.json:
"configurations": [
{
"name": "docs",
"buildOptions": ["syntaxOnly"],
"dflags": ["-Dddocs"],
"sourceFiles": ["html.ddoc"]
}
]
Or if you use dub.sdl:
configuration "docs" {
buildOptions "syntaxOnly"
dflags "-Dddocs"
sourceFiles "html.ddoc"
}
This adds a new configuration named "docs", which can be used
like this:
$ dub -c docs
Re: Returning multiple values from a function
On Monday, 4 September 2017 at 07:27:12 UTC, Vino.B wrote:
Hi,
Can you help me in how to return multiple values from a
function, the below code is throwing an error as below
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
Tuple!(int, string[]) Params () {
return tuple(1, ["C:\\Temp\\TEAM1\\BACKUP",
"C:\\Temp\\TEAM2\\ARCHIVE"]);
}
void main (){
Params.writeln;
}
Re: Using closure causes GC allocation
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
On Saturday, 2 September 2017 at 20:54:03 UTC, Vino.B wrote:
On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner
wrote:
On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
[...]
Cannot reproduce under Linux with dmd 2.076.0 (with commented
out Windows-only check). I'll try to see what happens on
Windows once I have a VM setup.
[...]
You changed the type of dFiles, which you return from
cleanFiles, without changing the return type of cleanFiles.
Change the return type of cleanFiles to the type the compiler
error above tells you it should be (`Tuple!(string,
string)[]` instead of `string[][]`), or let the compiler
infer it via auto (`auto cleanFiles(...`).
Hi,
Thank you very much, was able to resolve the second code
issue by changing the return type of the function.
Hi,
In order to resolve the issue "Using closure causes GC
allocation" it was stated that we need to use delegates, can
you please help me on how to as i have not gone that much far
in D programming.
import std.stdio: File,writeln;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir,
remove;
import std.typecons: tuple, Tuple;
import std.algorithm: filter, map, each;
import std.array: array;
Tuple!(string)[] logClean (string[] Lglst, int LogAge) {
if (!Lglst[0].exists) { mkdir(Lglst[0]); }
auto ct1 = Clock.currTime();
auto st1 = ct1 + days(-LogAge);
auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a
=> a.exists && a.isFile && a.timeCreated < st1).map!(a =>
tuple(a.name)).array;
dFiles.each!(a => a[0].remove);
return dFiles;
}
void main () {
string[] LogDir =
["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"];
int LogAge = 1;
logClean(LogDir,LogAge);
}
From,
Vino.B
If you are getting Using closure causes GC allocation from the
above, it will be because you reference a local in the template
delegate argument to filter:
filter!(a => a.exists &&
a.isFile &&
a.timeCreated < st1)
^^^
because of the local (st1) it needs a closure. I can't remember
how to transform delegates to take parameters and then call with
the parameter set to the local, someone else will have to fill
you in on that one.
Returning multiple values from a function
Hi,
Can you help me in how to return multiple values from a
function, the below code is throwing an error as below
Program:
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
Tuple!(int, string[]) Params () {
int Test1;
string[] Path;
Test1 = 1;
Path = ["C:\\Temp\\TEAM1\\BACKUP", "C:\\Temp\\TEAM2\\ARCHIVE"];
return Test1;
return Path;
}
void main (){
int Test1;
string[] Path;
Params;
writeln(Test1);
writeln(Path);
}
Error:
TEx1.d(9): Error: cannot implicitly convert expression Test1 of
type int to Tuple!(int, string[])
TEx1.d(10): Error: cannot implicitly convert expression Path of
type string[] to Tuple!(int, string[])
From,
Vino.B
