Re: std.array.array for immutable data types
On Monday, 19 February 2018 at 07:08:49 UTC, Fra Mecca wrote: Is there a way to avoid using to! conversion here? immutable string[] dst = to!(immutable string[])(array(pipe.readEnd.byLineCopy)); assumeUnique. immutable string[] dst = pipe.readEnd.byLineCopy.array.assumeUnique;
Re: std.array.array for immutable data types
On Monday, 19 February 2018 at 07:08:49 UTC, Fra Mecca wrote: Is there a way to avoid using to! conversion here? immutable string[] dst = to!(immutable string[])(array(pipe.readEnd.byLineCopy)); Are you looking for something like assumeUnique [1]? ``` pipe.readEnd.byLineCopy.array.assumeUnique; ``` It's just a nice wrapper for cast(immutable string[]). [1] https://dlang.org/phobos/std_exception.html#assumeUnique
std.array.array for immutable data types
Is there a way to avoid using to! conversion here? immutable string[] dst = to!(immutable string[])(array(pipe.readEnd.byLineCopy));
Re: multithread/concurrency/parallel methods and performance
On Sunday, 18 February 2018 at 17:54:58 UTC, SrMordred wrote:
I´m experimenting with threads and related recently.
(i´m just started so may be some terrrible mistakes here)
With this base work:
foreach(i ; 0 .. SIZE)
{
results[i] = values1[i] * values2[i];
}
and then with this 3 others methods: parallel, spawn and
Threads.
this was my results:
_base : 456 ms and 479 us
_parallel : 331 ms, 324 us, and 4 hnsecs
_concurrency : 367 ms, 348 us, and 2 hnsecs
_thread : 369 ms, 565 us, and 3 hnsecs
(code here : https://run.dlang.io/is/2pdmmk )
All methods have minor speedup gains. I was expecting a lot
more.
Since I have 7 cores I expected like below 100ms.
The operation is trivial and dataset is rather small. In such
cases SIMD with eg array ops is the way to go:
result[] = values[] * values2[];
Parallelism is gets more interesting with more expensive
operations. You may also try bigger sizes or both.
I´m not seeing false sharing in this case. or i'm wrong?
If someone can expand on this, i'll be grateful.
Thanks!
Re: multithread/concurrency/parallel methods and performance
On Sunday, 18 February 2018 at 17:54:58 UTC, SrMordred wrote:
I´m experimenting with threads and related recently.
(i´m just started so may be some terrrible mistakes here)
With this base work:
foreach(i ; 0 .. SIZE)
{
results[i] = values1[i] * values2[i];
}
and then with this 3 others methods: parallel, spawn and
Threads.
this was my results:
_base : 456 ms and 479 us
_parallel : 331 ms, 324 us, and 4 hnsecs
_concurrency : 367 ms, 348 us, and 2 hnsecs
_thread : 369 ms, 565 us, and 3 hnsecs
(code here : https://run.dlang.io/is/2pdmmk )
All methods have minor speedup gains. I was expecting a lot
more.
Since I have 7 cores I expected like below 100ms.
I´m not seeing false sharing in this case. or i'm wrong?
If someone can expand on this, i'll be grateful.
Thanks!
As SIZE=1024*1024 (i.e. not much, possibly well within L2 cache
for 32bit) it may be that dealing with the concurrency overhead
adds a significant amount of overhead. Also the run.dlang.io link
has no -O flag and thus no optimisations
without -O i get
_base : 323 ms, 92 μs, and 6 hnsecs
_parallel : 276 ms, 649 μs, and 3 hnsecs
_concurrency : 221 ms, 931 μs, and 7 hnsecs
_thread : 212 ms, 277 μs, and 3 hnsecs
with it I get
_base : 150 ms, 728 μs, and 5 hnsecs
_parallel : 120 ms, 78 μs, and 5 hnsecs
_concurrency : 134 ms, 787 μs, and 4 hnsecs
_thread : 129 ms, 476 μs, and 2 hnsecs
with SIZE= 16*1024*1024 without -O i get
_base : 5 secs, 835 ms, 240 μs, and 9 hnsecs
_parallel : 4 secs, 802 ms, 279 μs, and 8 hnsecs
_concurrency : 2 secs, 133 ms, 685 μs, and 3 hnsecs
_thread : 2 secs, 108 ms, 860 μs, and 9 hnsecs
with SIZE= 16*1024*1024 with -O i get
_base : 2 secs, 502 ms, 523 μs, and 4 hnsecs
_parallel : 1 sec, 769 ms, 945 μs, and 3 hnsecs
_concurrency : 1 sec, 362 ms, 747 μs, and 1 hnsec
_thread : 1 sec, 335 ms, 720 μs, and 1 hn
Re: countUntil to print all the index of a given string.
On Sunday, 18 February 2018 at 15:23:14 UTC, Cym13 wrote: On Sunday, 18 February 2018 at 14:48:59 UTC, Cym13 wrote: [...] Just thought of a much better/simpler solution for that last case that also doesn't force you to read all data (which might be impossible when dealing with infinite ranges): import std.range; import std.algorithm; a[] .enumerate // get tuples (index, value) .filter!(t => t[1] == "Test2") // keep only if value == "Test2" .map!(t => t[0]) // keep only the index part .writeln; Completely lazy. A nice example of how D's multiparadigm programming model allows the utilisation of algorithms that can scale to process massive datasets. There is an article in there somewhere ;-)
Re: Trying to forward unwrapped opDispatch names to alias this
On Monday, 19 February 2018 at 00:42:05 UTC, aliak wrote:
struct B(T) {
T t;
A a;
alias a this;
auto opDispatch(string name)() if (hasMember!(T, name)) {
return mixin("t." ~ name);
Did you perhaps mean `A` instead of `T` here? cuz in your code T
is int, not the struct.
Trying to forward unwrapped opDispatch names to alias this
I have a scenario where I'm wrapping functionality for a type,
but only if the contained type has a member. I want those to take
precedence. If the member is not there, then I want to delegate
to an aliases type via alias this.
I get an error here when I call b.p. Even though property p is in
type A and there's an alias a this in B.
Anyway to do this?
struct A {
@property string p() {
return "A";
}
@property void p(string str) {}
}
struct B(T) {
T t;
A a;
alias a this;
auto opDispatch(string name)() if (hasMember!(T, name)) {
return mixin("t." ~ name);
}
}
void main() {
B!int b;
b.max.writeln;
b.p.writeln; // Error: no property 'p' for type 'B!int'
}
Cheers,
- Ali
Re: isFuture
On Sunday, February 18, 2018 19:10:02 Tofu Ninja via Digitalmars-d-learn wrote: > What is __traits(isFuture)? The language documents says it tests > for @future which doesn't really help as @future is undocumented. https://github.com/dlang/DIPs/blob/master/DIPs/DIP1007.md - Jonathan M Davis
isFuture
What is __traits(isFuture)? The language documents says it tests for @future which doesn't really help as @future is undocumented.
Re: multithread/concurrency/parallel methods and performance
On Sunday, 18 February 2018 at 17:54:58 UTC, SrMordred wrote:
I´m experimenting with threads and related recently.
(i´m just started so may be some terrrible mistakes here)
With this base work:
foreach(i ; 0 .. SIZE)
{
results[i] = values1[i] * values2[i];
}
and then with this 3 others methods: parallel, spawn and
Threads.
this was my results:
_base : 456 ms and 479 us
_parallel : 331 ms, 324 us, and 4 hnsecs
_concurrency : 367 ms, 348 us, and 2 hnsecs
_thread : 369 ms, 565 us, and 3 hnsecs
(code here : https://run.dlang.io/is/2pdmmk )
All methods have minor speedup gains. I was expecting a lot
more.
Since I have 7 cores I expected like below 100ms.
I´m not seeing false sharing in this case. or i'm wrong?
If someone can expand on this, i'll be grateful.
Thanks!
It may be due to thread local storage:
https://tour.dlang.org/tour/en/multithreading/thread-local-storage
https://dlang.org/articles/migrate-to-shared.html
I'm not sure though, as I don't know how your results array
initialised.
Jordan
multithread/concurrency/parallel methods and performance
I´m experimenting with threads and related recently.
(i´m just started so may be some terrrible mistakes here)
With this base work:
foreach(i ; 0 .. SIZE)
{
results[i] = values1[i] * values2[i];
}
and then with this 3 others methods: parallel, spawn and Threads.
this was my results:
_base : 456 ms and 479 us
_parallel : 331 ms, 324 us, and 4 hnsecs
_concurrency : 367 ms, 348 us, and 2 hnsecs
_thread : 369 ms, 565 us, and 3 hnsecs
(code here : https://run.dlang.io/is/2pdmmk )
All methods have minor speedup gains. I was expecting a lot more.
Since I have 7 cores I expected like below 100ms.
I´m not seeing false sharing in this case. or i'm wrong?
If someone can expand on this, i'll be grateful.
Thanks!
Re: confused with data types
Thanks for all the insights :)
Re: countUntil to print all the index of a given string.
On Sunday, 18 February 2018 at 14:48:59 UTC, Cym13 wrote: [...] Just thought of a much better/simpler solution for that last case that also doesn't force you to read all data (which might be impossible when dealing with infinite ranges): import std.range; import std.algorithm; a[] .enumerate // get tuples (index, value) .filter!(t => t[1] == "Test2") // keep only if value == "Test2" .map!(t => t[0]) // keep only the index part .writeln; Completely lazy.
Re: std.traits.isBoolean
On Sunday, 18 February 2018 at 14:52:37 UTC, Tony wrote:
At
https://dlang.org/library/std/traits/is_boolean.html
it has:
enum isBoolean(T) = is(BooleanTypeOf!T) && !isAggregateType!T;
per:
https://dlang.org/library/std/traits/is_aggregate_type.html
isAggregateType is true for [struct, union, class, interface].
So BooleanTypeOf!T is true for structs, unions, classes and
interfaces? And if yes, why is that so?
Generally, no. But with alias this, it can be:
=
import std.traits : BooleanTypeOf;
import std.stdio : writeln;
struct NoBool {
int x;
}
struct AliasThisBool {
bool b;
alias b this;
}
void main()
{
static if(is(BooleanTypeOf!NoBool)) writeln("NoBool");
static if(is(BooleanTypeOf!AliasThisBool))
writeln("AliasThisBool");
}
=
std.traits.isBoolean
At https://dlang.org/library/std/traits/is_boolean.html it has: enum isBoolean(T) = is(BooleanTypeOf!T) && !isAggregateType!T; per: https://dlang.org/library/std/traits/is_aggregate_type.html isAggregateType is true for [struct, union, class, interface]. So BooleanTypeOf!T is true for structs, unions, classes and interfaces? And if yes, why is that so?
Re: confused with data types
On Sunday, 18 February 2018 at 13:08:09 UTC, thorstein wrote:
On Sunday, 18 February 2018 at 12:51:04 UTC, thorstein wrote:
// Solution 1
foreach(row; arr)
{ foreach(col; row)
{ col[] *= skalar;
}
}
return arr;
// Solution 2
import std.array;
return array(arr.map!(b => array(b[].map!(c =>
array(c[].map!(d => d * skalar));
// Solution 3
import std.algorithm;
arr.each!(a => a[].each!(b => b[] *= skalar));
return arr;
Q#1:
Does the compiler optimizes all solutions equally strong or
does it prefer implementations like solution 1?
Q#2:
Solution 2 is a 1-liner but a bit harder to read. Why reducing
solution 3 to:
return arr.each!(a => a[].each!(b => b[] *= skalar));
gives a compile error? I do writeln() the function result.
Q#3:
If I can:
static import std.array;
return std.array.array(arr.map!(b =>
std.array.array(b[].map!(c =>...
How would I apply a similar version with 'static import
std.algorithm' to solution 3?
static import std.algorithm;
std.algorithm.arr.each!(a => a[]... //does obviously not work
Thanks, thorstein
Sorry, Solution 2 should be:
import std.array;
return array(arr.map!(b => array(b[].map!(c => c[] *=
skalar;
and is as readable as Solution 3. However, Q#2 still remains.
Q#1: i would guess solution 1 would be the fastest and solution 2
the slowest.
Q#2: you get a error because each returns void.
Q#3: you cant use ufcs with static imports, you have to call the
function normaly:
static import algo = std.algorithm;
algo.each!((a) => algo.each!((b) => b[] *= skalar)(a))(arr);
Re: countUntil to print all the index of a given string.
On Sunday, 18 February 2018 at 11:55:37 UTC, Vino wrote:
Hi All,
Request your help on printing the all index of an array
element , eg; the below code prints the index of the string
"Test2" as [1], but the string "Test2" is present 2 times at
index 1 and 4, so how do I print all the index of a given
string.
import std.stdio;
import std.container;
import std.algorithm;
void main () {
auto a = Array!string("Test1", "Test2", "Test3", "Test1",
"Test2");
writeln(SList!int(a[].countUntil("Test2"))[]);
}
Output
[1]
Expected
[1, 4]
From,
Vino.B
countUntil is good when you want to avoid having to look at all
your data, but in this case I don't think it's the best solution.
You could do a loop storing each index and then restart your
countUntil from there, but quite frankly it would be easier to
just loop over the array at that point:
ulong[] result;
for (ulong index=0 ; indexHowever, if you want a more streamlined, functionnal solution,
you can go all the way and avoid all explicit loops and
intermediate variables using fold:
import std.range: enumerate;
import std.algorithm: fold;
a[]
.enumerate
.fold!((a, b) => b[1] == "Test2" ? a ~ b[0] :
a)(cast(ulong[])[])
.writeln;
Re: Return value in BetterC mode.
On Saturday, 17 February 2018 at 13:47:28 UTC, meppl wrote: On Saturday, 17 February 2018 at 07:58:40 UTC, ANtlord wrote: ... ... sadly I have no good idea how to name the title of that issue :/ I looked at it again and came up with a title name: https://issues.dlang.org/show_bug.cgi?id=18457
Re: confused with data types
On Sunday, 18 February 2018 at 12:51:04 UTC, thorstein wrote:
Thank you for the very informative answers showing different
gears in D!
However, there are still some details I'm struggling with:
Assume some calculations on a very big numeric array
'double[][][] arr'.
Now we could choose 1 out of 3 different implementations:
// Solution 1
foreach(row; arr)
{ foreach(col; row)
{ col[] *= skalar;
}
}
return arr;
// Solution 2
import std.array;
return array(arr.map!(b => array(b[].map!(c => array(c[].map!(d
=> d * skalar));
// Solution 3
import std.algorithm;
arr.each!(a => a[].each!(b => b[] *= skalar));
return arr;
Q#1:
Does the compiler optimizes all solutions equally strong or
does it prefer implementations like solution 1?
Q#2:
Solution 2 is a 1-liner but a bit harder to read. Why reducing
solution 3 to:
return arr.each!(a => a[].each!(b => b[] *= skalar));
gives a compile error? I do writeln() the function result.
Q#3:
If I can:
static import std.array;
return std.array.array(arr.map!(b => std.array.array(b[].map!(c
=>...
How would I apply a similar version with 'static import
std.algorithm' to solution 3?
static import std.algorithm;
std.algorithm.arr.each!(a => a[]... //does obviously not work
Thanks, thorstein
Sorry, Solution 2 should be:
import std.array;
return array(arr.map!(b => array(b[].map!(c => c[] *= skalar;
and is as readable as Solution 3. However, Q#2 still remains.
Re: confused with data types
Thank you for the very informative answers showing different
gears in D!
However, there are still some details I'm struggling with:
Assume some calculations on a very big numeric array
'double[][][] arr'.
Now we could choose 1 out of 3 different implementations:
// Solution 1
foreach(row; arr)
{ foreach(col; row)
{ col[] *= skalar;
}
}
return arr;
// Solution 2
import std.array;
return array(arr.map!(b => array(b[].map!(c => array(c[].map!(d
=> d * skalar));
// Solution 3
import std.algorithm;
arr.each!(a => a[].each!(b => b[] *= skalar));
return arr;
Q#1:
Does the compiler optimizes all solutions equally strong or does
it prefer implementations like solution 1?
Q#2:
Solution 2 is a 1-liner but a bit harder to read. Why reducing
solution 3 to:
return arr.each!(a => a[].each!(b => b[] *= skalar));
gives a compile error? I do writeln() the function result.
Q#3:
If I can:
static import std.array;
return std.array.array(arr.map!(b => std.array.array(b[].map!(c
=>...
How would I apply a similar version with 'static import
std.algorithm' to solution 3?
static import std.algorithm;
std.algorithm.arr.each!(a => a[]... //does obviously not work
Thanks, thorstein
countUntil to print all the index of a given string.
Hi All,
Request your help on printing the all index of an array element
, eg; the below code prints the index of the string "Test2" as
[1], but the string "Test2" is present 2 times at index 1 and
4, so how do I print all the index of a given string.
import std.stdio;
import std.container;
import std.algorithm;
void main () {
auto a = Array!string("Test1", "Test2", "Test3", "Test1",
"Test2");
writeln(SList!int(a[].countUntil("Test2"))[]);
}
Output
[1]
Expected
[1, 4]
From,
Vino.B
