Re: Improving dot product for standard multidimensional D arrays
On Sunday, 1 March 2020 at 20:58:42 UTC, p.shkadzko wrote: Hello again, Thanks to previous thread on multidimensional arrays, I managed to play around with pure D matrix representations and even benchmark a little against numpy: [...] Matrix multiplication is about cache-friendly blocking. https://www.cs.utexas.edu/users/pingali/CS378/2008sp/papers/gotoPaper.pdf `mir-blas` package can be used for matrix operations for ndslice. `cblas` - if you want to work with your own matrix type .
Re: Using LDC2 on ARM
On Monday, 2 March 2020 at 17:45:26 UTC, Severin Teona wrote: Hello, I am working on a project that uses a Raspberry Pi (armv7l) and the latest LDC version I found for this architecture is 1.13.0. Can you help me install the latest version(1.20.0)? Also, I'm having problems using the DPP package with the 1.13.0 LDC version, most likely because the latest version of DPP(0.4.1) is using a newer version of Phobos. Is there any reason why the LDC team stopped releasing pre-built binaries for arm? I'm looking forward to your help. Teona. please refer to https://wiki.dlang.org/Programming_in_D_tutorial_on_Embedded_Linux_ARM_devices you can build your own LDC toolchain always using the latest.
Re: in not working for arrays is silly, change my view
On Mon, Mar 02, 2020 at 07:51:34PM -0500, Steven Schveighoffer via Digitalmars-d-learn wrote: [...] > On 3/2/20 6:46 PM, H. S. Teoh wrote: > > To prevent the optimizer from eliding "useless" code, you need to do > > something with the return value that isn't trivial (assigning to a > > variable that doesn't get used afterwards is "trivial", so that's > > not enough). The easiest way is to print the result: the optimizer > > cannot elide I/O. > > Yeah, well, that means you are also benchmarking the i/o (which would > dwarf the other pieces being tested). Not necessarily. Create a global variable whose sole purpose is to accumulate the return values of the functions being tested, then return its value as the return value of main(). The optimizer is bound by semantics not to elide anything then. > I think assigning the result to a global fits the bill pretty well, > but obviously only works when you're not inside a pure function. [...] A sufficiently-advanced optimizer would notice the global isn't referred to anywhere else, and therefore of no effect, and elide it anyway. Not saying it actually would, 'cos I think you're probably right, but I'm leaving nothing to chance when the LDC optimizer is in question. :-P T -- It only takes one twig to burn down a forest.
Re: in not working for arrays is silly, change my view
On 3/2/20 7:32 PM, aliak wrote:
On Monday, 2 March 2020 at 23:27:22 UTC, Steven Schveighoffer wrote:
What I think is happening is that it determines nobody is using the
result, and the function is pure, so it doesn't bother calling that
function (probably not even the lambda, and then probably removes the
loop completely).
I'm assuming for some reason, the binary search is not flagged pure,
so it's not being skipped.
Apparently you're right:
https://github.com/dlang/phobos/blob/5e13653a6eb55c1188396ae064717a1a03fd7483/std/range/package.d#L11107
That's not definitive. Note that a template member or member of a struct
template can be *inferred* to be pure.
It's also entirely possible for the function to be pure, but the
compiler decides for another reason not to elide the whole thing.
Optimization isn't ever guaranteed.
If I change to this to ensure side effects:
bool makeImpure; // TLS variable outside of main
...
auto results = benchmark!(
() => makeImpure = r1.canFind(max),
() => makeImpure = r2.contains(max),
() => makeImpure = r3.canFind(max),
)(5_000);
writefln("%(%s\n%)", results); // modified to help with the comma
confusion
I now get:
4 secs, 428 ms, and 3 hnsecs
221 μs and 9 hnsecs
4 secs, 49 ms, 982 μs, and 5 hnsecs
More like what I expected!
A damn! And here I was thinking that branch prediction made a HUGE
difference! Ok, I'm taking my tail and slowly moving away now :) Let us
never speak of this again.
LOL, I'm sure this will come up again ;) The forums are full of
confusing benchmarks where LDC has elided the whole thing being tested.
It's amazing at optimizing. Sometimes, too amazing.
On 3/2/20 6:46 PM, H. S. Teoh wrote:
> To prevent the optimizer from eliding "useless" code, you need to do
> something with the return value that isn't trivial (assigning to a
> variable that doesn't get used afterwards is "trivial", so that's not
> enough). The easiest way is to print the result: the optimizer cannot
> elide I/O.
Yeah, well, that means you are also benchmarking the i/o (which would
dwarf the other pieces being tested).
I think assigning the result to a global fits the bill pretty well, but
obviously only works when you're not inside a pure function.
-Steve
Re: in not working for arrays is silly, change my view
On Monday, 2 March 2020 at 23:27:22 UTC, Steven Schveighoffer
wrote:
What I think is happening is that it determines nobody is using
the result, and the function is pure, so it doesn't bother
calling that function (probably not even the lambda, and then
probably removes the loop completely).
I'm assuming for some reason, the binary search is not flagged
pure, so it's not being skipped.
Apparently you're right:
https://github.com/dlang/phobos/blob/5e13653a6eb55c1188396ae064717a1a03fd7483/std/range/package.d#L11107
If I change to this to ensure side effects:
bool makeImpure; // TLS variable outside of main
...
auto results = benchmark!(
() => makeImpure = r1.canFind(max),
() => makeImpure = r2.contains(max),
() => makeImpure = r3.canFind(max),
)(5_000);
writefln("%(%s\n%)", results); // modified to help with the
comma confusion
I now get:
4 secs, 428 ms, and 3 hnsecs
221 μs and 9 hnsecs
4 secs, 49 ms, 982 μs, and 5 hnsecs
More like what I expected!
A damn! And here I was thinking that branch prediction made a
HUGE difference! Ok, I'm taking my tail and slowly moving away
now :) Let us never speak of this again.
-Steve
Re: in not working for arrays is silly, change my view
On Mon, Mar 02, 2020 at 06:27:22PM -0500, Steven Schveighoffer via Digitalmars-d-learn wrote: [...] > Yeah, this looked very fishy to me. ldc can do some nasty "helpful" > things to save you time! When I posted my results, I was using DMD. > > I used run.dlang.io with ldc, and verified I get the same (similar) > result as you. But searching through 5 billion integers can't be > instantaneous, on any modern hardware. [...] Beware that LDC's over-zealous optimizer can sometimes elide an entire function call tree if it determines that the return value is not used anywhere. I've also observed that it can sometimes execute the entire function call tree at compile-time and emit just a single instruction that loads the final result. So, always check the assembly output so that you're sure the benchmark is actually measuring what you think it's measuring. To prevent the optimizer from eliding "useless" code, you need to do something with the return value that isn't trivial (assigning to a variable that doesn't get used afterwards is "trivial", so that's not enough). The easiest way is to print the result: the optimizer cannot elide I/O. T -- Freedom: (n.) Man's self-given right to be enslaved by his own depravity.
Re: in not working for arrays is silly, change my view
On 3/2/20 5:21 PM, aliak wrote:
On Monday, 2 March 2020 at 21:33:37 UTC, Steven Schveighoffer wrote:
On 3/2/20 3:52 PM, aliak wrote:
On Monday, 2 March 2020 at 15:47:26 UTC, Steven Schveighoffer wrote:
On 3/2/20 6:52 AM, Andrea Fontana wrote:
On Saturday, 29 February 2020 at 20:11:24 UTC, Steven Schveighoffer
wrote:
1. in is supposed to be O(lg(n)) or better. Generic code may
depend on this property. Searching an array is O(n).
Probably it should work if we're using a "SortedRange".
int[] a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
auto p = assumeSorted(a);
assert(3 in p);
That could work. Currently, you need to use p.contains(3). opIn
could be added as a shortcut.
It only makes sense if you have it as a literal though, as
p.contains(3) isn't that bad to use:
assert(3 in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].assumeSorted);
There's no guarantee that checking if a value is in a sorted list is
any faster than checking if it's in a non sorted list. It's why sort
usually switches from a binary-esque algorithm to a linear one at a
certain size.
Well of course! A binary search needs Lg(n) comparisons for pretty
much any value, whereas a linear search is going to end early when it
finds it. So there's no guarantee that searching for an element in the
list is going to be faster one way or the other. But Binary search is
going to be faster overall because the complexity is favorable.
Overall tending towards infinity maybe, but not overall on the average
case it would seem. Branch prediction in CPUs changes that in that with
a binary search it is always a miss. Whereas with linear it's always a hit.
The list could potentially need to be _very_ large for p.contains to
make a significant impact over canFind(p) AFAIK.
Here's a small test program, try playing with the numbers and see
what happens:
import std.random;
import std.range;
import std.algorithm;
import std.datetime.stopwatch;
import std.stdio;
void main()
{
auto count = 1_000;
auto max = int.max;
alias randoms = generate!(() => uniform(0, max));
auto r1 = randoms.take(count).array;
auto r2 = r1.dup.sort;
auto elem = r1[uniform(0, count)];
auto elem = r1[$-1]; // try this instead
benchmark!(
() => r1.canFind(elem),
() => r2.contains(elem),
)(1_000).writeln;
}
Use LDC and -O3 of course. I was hard pressed to get the sorted
contains to be any faster than canFind.
This begs the question then: do these requirements on in make any
sense? An algorithm can be log n (ala the sorted search) but still be
a magnitude slower than a linear search... what has the world come to
臘♂️
PS: Why is it named contains if it's on a SortedRange and canFind
otherwise?
A SortedRange uses O(lgn) steps vs. canFind which uses O(n) steps.
canFind is supposed to tell the reader that it's O(n) and contains O(lgn)?
canFind means find will not return empty. find is linear search.
Probably not the clearest distinction, but contains isn't an algorithm,
it's a member. I don't think there's any general naming convention for
members like this, but I would say if contains means O(lgn) then that's
what we should use everywhere to mean that.
I suppose the naming could be improved.
If you change your code to testing 1000 random numbers, instead of a
random number guaranteed to be included, then you will see a
significant improvement with the sorted version. I found it to be
about 10x faster. (most of the time, none of the other random numbers
are included). Even if you randomly select 1000 numbers from the
elements, the binary search will be faster. In my tests, it was about
5x faster.
Hmm... What am I doing wrong with this code? And also how are you
compiling?:
void main()
{
auto count = 1_000_000;
auto max = int.max;
alias randoms = generate!(() => uniform(0, max - 1));
auto r1 = randoms.take(count).array;
auto r2 = r1.dup.sort;
auto r3 = r1.dup.randomShuffle;
auto results = benchmark!(
() => r1.canFind(max),
() => r2.contains(max),
() => r3.canFind(max),
)(5_000);
results.writeln;
}
$ ldc2 -O3 test.d && ./test
[1 hnsec, 84 μs and 7 hnsecs, 0 hnsecs]
Yeah, this looked very fishy to me. ldc can do some nasty "helpful"
things to save you time! When I posted my results, I was using DMD.
I used run.dlang.io with ldc, and verified I get the same (similar)
result as you. But searching through 5 billion integers can't be
instantaneous, on any modern hardware.
So I also tried this.
auto results = benchmark!(
() => false,
() => r2.contains(max),
() => r3.canFind(max),
)(5_000);
[4 μs and 9 hnsecs, 166 μs and 8 hnsecs, 4 μs and 7 hnsecs]
Hey look! returning a boolean is more expensive than a 1 million element
linear search!
What I think is happening is that it determines nobody is using the
result, and the function is pure, so it doesn't bother calling
Re: Improving dot product for standard multidimensional D arrays
On Monday, 2 March 2020 at 20:56:50 UTC, jmh530 wrote:
On Monday, 2 March 2020 at 20:22:55 UTC, p.shkadzko wrote:
[snip]
Interesting growth of processing time. Could it be GC?
+--+-+
| matrixDotProduct | time (sec.) |
+--+-+
| 2x[100 x 100]|0.01 |
| 2x[1000 x 1000] |2.21 |
| 2x[1500 x 1000] | 5.6 |
| 2x[1500 x 1500] |9.28 |
| 2x[2000 x 2000] | 44.59 |
| 2x[2100 x 2100] | 55.13 |
+--+-+
Your matrixDotProduct creates a new Matrix and then returns it.
When you look at the Matrix struct, it is basically building
upon D's GC-backed slices. So yes, you are using the GC here.
You could try creating the output matrices outside of the
matrixDotProduct function and then pass them by pointer or
reference into the function if you want to profile just the
calculation.
I tried using ref (pointer to struct) but it only made things
slower by 0.5 s.
I an not passing the result matrix to "toIdx" anymore, this is
not necessary we just need the column value. This didn't change
anything though.
Here is how the code looks now.
*
pragma(inline) static int toIdx(int matrixCols, in int i, in int
j)
{
return matrixCols * i + j;
}
Matrix!T matrixDotProduct(T)(Matrix!T m1, Matrix!T m2, ref
Matrix!T initM)
in
{
assert(m1.rows == m2.cols);
}
do
{
/// This implementation requires opIndex in Matrix struct.
for (int i; i < m1.rows; ++i)
{
for (int j; j < m2.cols; ++j)
{
for (int k; k < m2.rows; ++k)
{
initM.data[toIdx(initM.cols, i, j)] += m1[i, k] *
m2[k, j];
}
}
}
return initM;
}
void main() {
Matrix!double initMatrix = Matrix!double(m1.rows, m2.cols);
auto e = matrixDotProduct!double(m5, m6, initMatrix).to2D;
}
*
I tried disabling GC via GC.disable; GC.enable; before and after
3 loops in matrixDotProduct to see what happens. But nothing
changed :(
Re: Using LDC2 on ARM
On Monday, 2 March 2020 at 17:45:26 UTC, Severin Teona wrote: Is there any reason why the LDC team stopped releasing pre-built binaries for arm? It's the only package that isn't auto-generated by CI services and requires manual steps in a painfully slow qemu environment, a process that takes hours and which I wasn't willing to keep up any longer.
Re: Using LDC2 on ARM
On Monday, 2 March 2020 at 17:45:26 UTC, Severin Teona wrote: Hello, I am working on a project that uses a Raspberry Pi (armv7l) and the latest LDC version I found for this architecture is 1.13.0. Can you help me install the latest version(1.20.0)? Also, I'm having problems using the DPP package with the 1.13.0 LDC version, most likely because the latest version of DPP(0.4.1) is using a newer version of Phobos. Is there any reason why the LDC team stopped releasing pre-built binaries for arm? I'm looking forward to your help. Teona. Ldc2 version 1.17 is available here https://github.com/ldc-developers/ldc/releases/download/v1.17.0/ldc2-1.17.0-linux-armhf.tar.xz If you have a raspberry 3 you can also switch to aarch64. Aarch64 binaries are available for every new release of ldc2 on the github release page. Kind regards Andre
Re: in not working for arrays is silly, change my view
On Monday, 2 March 2020 at 21:33:37 UTC, Steven Schveighoffer
wrote:
On 3/2/20 3:52 PM, aliak wrote:
On Monday, 2 March 2020 at 15:47:26 UTC, Steven Schveighoffer
wrote:
On 3/2/20 6:52 AM, Andrea Fontana wrote:
On Saturday, 29 February 2020 at 20:11:24 UTC, Steven
Schveighoffer wrote:
1. in is supposed to be O(lg(n)) or better. Generic code
may depend on this property. Searching an array is O(n).
Probably it should work if we're using a "SortedRange".
int[] a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
auto p = assumeSorted(a);
assert(3 in p);
That could work. Currently, you need to use p.contains(3).
opIn could be added as a shortcut.
It only makes sense if you have it as a literal though, as
p.contains(3) isn't that bad to use:
assert(3 in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].assumeSorted);
There's no guarantee that checking if a value is in a sorted
list is any faster than checking if it's in a non sorted list.
It's why sort usually switches from a binary-esque algorithm
to a linear one at a certain size.
Well of course! A binary search needs Lg(n) comparisons for
pretty much any value, whereas a linear search is going to end
early when it finds it. So there's no guarantee that searching
for an element in the list is going to be faster one way or the
other. But Binary search is going to be faster overall because
the complexity is favorable.
Overall tending towards infinity maybe, but not overall on the
average case it would seem. Branch prediction in CPUs changes
that in that with a binary search it is always a miss. Whereas
with linear it's always a hit.
The list could potentially need to be _very_ large for
p.contains to make a significant impact over canFind(p) AFAIK.
Here's a small test program, try playing with the numbers and
see what happens:
import std.random;
import std.range;
import std.algorithm;
import std.datetime.stopwatch;
import std.stdio;
void main()
{
auto count = 1_000;
auto max = int.max;
alias randoms = generate!(() => uniform(0, max));
auto r1 = randoms.take(count).array;
auto r2 = r1.dup.sort;
auto elem = r1[uniform(0, count)];
auto elem = r1[$-1]; // try this instead
benchmark!(
() => r1.canFind(elem),
() => r2.contains(elem),
)(1_000).writeln;
}
Use LDC and -O3 of course. I was hard pressed to get the
sorted contains to be any faster than canFind.
This begs the question then: do these requirements on in make
any sense? An algorithm can be log n (ala the sorted search)
but still be a magnitude slower than a linear search... what
has the world come to 臘♂️
PS: Why is it named contains if it's on a SortedRange and
canFind otherwise?
A SortedRange uses O(lgn) steps vs. canFind which uses O(n)
steps.
canFind is supposed to tell the reader that it's O(n) and
contains O(lgn)?
If you change your code to testing 1000 random numbers, instead
of a random number guaranteed to be included, then you will see
a significant improvement with the sorted version. I found it
to be about 10x faster. (most of the time, none of the other
random numbers are included). Even if you randomly select 1000
numbers from the elements, the binary search will be faster. In
my tests, it was about 5x faster.
Hmm... What am I doing wrong with this code? And also how are you
compiling?:
void main()
{
auto count = 1_000_000;
auto max = int.max;
alias randoms = generate!(() => uniform(0, max - 1));
auto r1 = randoms.take(count).array;
auto r2 = r1.dup.sort;
auto r3 = r1.dup.randomShuffle;
auto results = benchmark!(
() => r1.canFind(max),
() => r2.contains(max),
() => r3.canFind(max),
)(5_000);
results.writeln;
}
$ ldc2 -O3 test.d && ./test
[1 hnsec, 84 μs and 7 hnsecs, 0 hnsecs]
Note that the compiler can do a lot more tricks for linear
searches, and CPUs are REALLY good at searching sequential
data. But complexity is still going to win out eventually over
heuristics. Phobos needs to be a general library, not one that
only caters to certain situations.
General would be the most common case. I don't think extremely
large (for some definition of large) lists are the more common
ones. Or maybe they are. But I'd be surprised. I also don't think
phobos is a very data-driven library. But, that's a whole other
conversation :)
-Steve
Re: in not working for arrays is silly, change my view
On 3/2/20 3:52 PM, aliak wrote:
On Monday, 2 March 2020 at 15:47:26 UTC, Steven Schveighoffer wrote:
On 3/2/20 6:52 AM, Andrea Fontana wrote:
On Saturday, 29 February 2020 at 20:11:24 UTC, Steven Schveighoffer
wrote:
1. in is supposed to be O(lg(n)) or better. Generic code may depend
on this property. Searching an array is O(n).
Probably it should work if we're using a "SortedRange".
int[] a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
auto p = assumeSorted(a);
assert(3 in p);
That could work. Currently, you need to use p.contains(3). opIn could
be added as a shortcut.
It only makes sense if you have it as a literal though, as
p.contains(3) isn't that bad to use:
assert(3 in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].assumeSorted);
There's no guarantee that checking if a value is in a sorted list is any
faster than checking if it's in a non sorted list. It's why sort usually
switches from a binary-esque algorithm to a linear one at a certain
size.
Well of course! A binary search needs Lg(n) comparisons for pretty much
any value, whereas a linear search is going to end early when it finds
it. So there's no guarantee that searching for an element in the list is
going to be faster one way or the other. But Binary search is going to
be faster overall because the complexity is favorable.
The list could potentially need to be _very_ large for p.contains
to make a significant impact over canFind(p) AFAIK.
Here's a small test program, try playing with the numbers and see what
happens:
import std.random;
import std.range;
import std.algorithm;
import std.datetime.stopwatch;
import std.stdio;
void main()
{
auto count = 1_000;
auto max = int.max;
alias randoms = generate!(() => uniform(0, max));
auto r1 = randoms.take(count).array;
auto r2 = r1.dup.sort;
auto elem = r1[uniform(0, count)];
auto elem = r1[$-1]; // try this instead
benchmark!(
() => r1.canFind(elem),
() => r2.contains(elem),
)(1_000).writeln;
}
Use LDC and -O3 of course. I was hard pressed to get the sorted contains
to be any faster than canFind.
This begs the question then: do these requirements on in make any sense?
An algorithm can be log n (ala the sorted search) but still be a
magnitude slower than a linear search... what has the world come to 臘♂️
PS: Why is it named contains if it's on a SortedRange and canFind
otherwise?
A SortedRange uses O(lgn) steps vs. canFind which uses O(n) steps.
If you change your code to testing 1000 random numbers, instead of a
random number guaranteed to be included, then you will see a significant
improvement with the sorted version. I found it to be about 10x faster.
(most of the time, none of the other random numbers are included). Even
if you randomly select 1000 numbers from the elements, the binary search
will be faster. In my tests, it was about 5x faster.
Note that the compiler can do a lot more tricks for linear searches, and
CPUs are REALLY good at searching sequential data. But complexity is
still going to win out eventually over heuristics. Phobos needs to be a
general library, not one that only caters to certain situations.
-Steve
Re: Improving dot product for standard multidimensional D arrays
On Monday, 2 March 2020 at 20:22:55 UTC, p.shkadzko wrote: [snip] Interesting growth of processing time. Could it be GC? +--+-+ | matrixDotProduct | time (sec.) | +--+-+ | 2x[100 x 100]|0.01 | | 2x[1000 x 1000] |2.21 | | 2x[1500 x 1000] | 5.6 | | 2x[1500 x 1500] |9.28 | | 2x[2000 x 2000] | 44.59 | | 2x[2100 x 2100] | 55.13 | +--+-+ Your matrixDotProduct creates a new Matrix and then returns it. When you look at the Matrix struct, it is basically building upon D's GC-backed slices. So yes, you are using the GC here. You could try creating the output matrices outside of the matrixDotProduct function and then pass them by pointer or reference into the function if you want to profile just the calculation.
Re: in not working for arrays is silly, change my view
On Monday, 2 March 2020 at 15:47:26 UTC, Steven Schveighoffer
wrote:
On 3/2/20 6:52 AM, Andrea Fontana wrote:
On Saturday, 29 February 2020 at 20:11:24 UTC, Steven
Schveighoffer wrote:
1. in is supposed to be O(lg(n)) or better. Generic code may
depend on this property. Searching an array is O(n).
Probably it should work if we're using a "SortedRange".
int[] a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
auto p = assumeSorted(a);
assert(3 in p);
That could work. Currently, you need to use p.contains(3). opIn
could be added as a shortcut.
It only makes sense if you have it as a literal though, as
p.contains(3) isn't that bad to use:
assert(3 in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].assumeSorted);
-Steve
There's no guarantee that checking if a value is in a sorted list
is any faster than checking if it's in a non sorted list. It's
why sort usually switches from a binary-esque algorithm to a
linear one at a certain size. The list could potentially need to
be _very_ large for p.contains to make a significant impact over
canFind(p) AFAIK.
Here's a small test program, try playing with the numbers and see
what happens:
import std.random;
import std.range;
import std.algorithm;
import std.datetime.stopwatch;
import std.stdio;
void main()
{
auto count = 1_000;
auto max = int.max;
alias randoms = generate!(() => uniform(0, max));
auto r1 = randoms.take(count).array;
auto r2 = r1.dup.sort;
auto elem = r1[uniform(0, count)];
benchmark!(
() => r1.canFind(elem),
() => r2.contains(elem),
)(1_000).writeln;
}
Use LDC and -O3 of course. I was hard pressed to get the sorted
contains to be any faster than canFind.
This begs the question then: do these requirements on in make any
sense? An algorithm can be log n (ala the sorted search) but
still be a magnitude slower than a linear search... what has the
world come to 臘♂️
PS: Why is it named contains if it's on a SortedRange and canFind
otherwise?
Re: in not working for arrays is silly, change my view
On Monday, 2 March 2020 at 15:50:08 UTC, Steven Schveighoffer wrote: On 3/2/20 6:39 AM, JN wrote: On Saturday, 29 February 2020 at 21:56:51 UTC, Ali Çehreli wrote: Because you mentioned canFind, I think you want the semantics to be "is there an element with this value." If so, it would be confusing to use the same operator for two different things: For associative arrays, it means "is there an element accessible with this key." Does it? I always viewed it as "is this value in list of keys" Array keys are the element index.. So essentially: int[int] c1; int[] c2 = new int[4]; c1[3] = 10; c2[3] = 10; assert(3 in c1); // true assert(3 in c2); // what should this do? -Steve If in were to mean "is this value in list of keys" then to be consistent: 3 in c2 == 3 < c2.length
Re: Improving dot product for standard multidimensional D arrays
On Sunday, 1 March 2020 at 20:58:42 UTC, p.shkadzko wrote: Hello again, Thanks to previous thread on multidimensional arrays, I managed to play around with pure D matrix representations and even benchmark a little against numpy: [...] Interesting growth of processing time. Could it be GC? +--+-+ | matrixDotProduct | time (sec.) | +--+-+ | 2x[100 x 100]|0.01 | | 2x[1000 x 1000] |2.21 | | 2x[1500 x 1000] | 5.6 | | 2x[1500 x 1500] |9.28 | | 2x[2000 x 2000] | 44.59 | | 2x[2100 x 2100] | 55.13 | +--+-+
Re: GtkD - Add images to IconView with Pixbuf
On Monday, 2 March 2020 at 16:37:46 UTC, cfcd14f496326e429ce03c48650b7966 wrote: Hello. :-) I found this guide : https://www.kksou.com/php-gtk2/sample-codes/display-a-list-of-thumbnail-images-using-GtkIconView.php This guide used 'Pixbuf' for use fromFile method, but GtkD don't have it. : https://api.gtkd.org/gdkpixbuf.Pixbuf.Pixbuf.html Is there any other way? Thanks. Looks like gtkd follows a more d idiomatic way. Use constructor Pixbuf(filename). this(string filename);
Re: Improving dot product for standard multidimensional D arrays
On Monday, 2 March 2020 at 18:17:05 UTC, p.shkadzko wrote: [snip] I tested @fastmath and @optmath for toIdx function and that didn't change anyting. @optmath is from mir, correct? I believe it implies @fastmath. The latest code in mir doesn't have it doing anything else at least.
Re: Using LDC2 on ARM
On Mon, Mar 2, 2020 at 6:50 PM Severin Teona via Digitalmars-d-learn wrote: > > Hello, > > I am working on a project that uses a Raspberry Pi (armv7l) and > the latest LDC version I found for this architecture is 1.13.0. > Can you help me install the latest version(1.20.0)? > > Also, I'm having problems using the DPP package with the 1.13.0 > LDC version, most likely because the latest version of DPP(0.4.1) > is using a newer version of Phobos. > > Is there any reason why the LDC team stopped releasing pre-built > binaries for arm? > > I'm looking forward to your help. > Teona. Do you really need to build you app on arm, could not you use crosscompiling?
Re: Using LDC2 on ARM
On Mon, Mar 2, 2020 at 7:40 PM Daniel Kozak wrote: > > On Mon, Mar 2, 2020 at 6:50 PM Severin Teona via Digitalmars-d-learn > Do you really need to build you app on arm, could not you use crosscompiling? https://wiki.dlang.org/Cross-compiling_with_LDC
Re: Using LDC2 on ARM
On Mon, Mar 2, 2020 at 7:40 PM Daniel Kozak wrote: > > On Mon, Mar 2, 2020 at 6:50 PM Severin Teona via Digitalmars-d-learn > wrote: > > > > Hello, > > > > I am working on a project that uses a Raspberry Pi (armv7l) and > > the latest LDC version I found for this architecture is 1.13.0. > > Can you help me install the latest version(1.20.0)? > > > > Also, I'm having problems using the DPP package with the 1.13.0 > > LDC version, most likely because the latest version of DPP(0.4.1) > > is using a newer version of Phobos. > > > > Is there any reason why the LDC team stopped releasing pre-built > > binaries for arm? > > > > I'm looking forward to your help. > > Teona. > > Do you really need to build you app on arm, could not you use crosscompiling? But if you really wan to compile it on RPI you could try use archlinuxarm and ldc package: https://archlinuxarm.org/packages/armv7h/ldc
Re: Improving dot product for standard multidimensional D arrays
On Monday, 2 March 2020 at 15:00:56 UTC, jmh530 wrote: On Monday, 2 March 2020 at 13:35:15 UTC, p.shkadzko wrote: [snip] Thanks. I don't have time right now to review this thoroughly. My recollection is that the dot product of two matrices is actually matrix multiplication, correct? It generally makes sense to defer to other people's implementation of this. I recommend trying lubeck's version against numpy. It uses a blas/lapack implementation. mir-glas, I believe, also has a version. Also, I'm not sure if the fastmath attribute would do anything here, but something worth looking into. Yes, this it is a sum of multiplications between elements of two matrices or a scalar product in case of vectors. This is not simple element-wise multiplication that I did in earlier benchmarks. I tested @fastmath and @optmath for toIdx function and that didn't change anyting.
Using LDC2 on ARM
Hello, I am working on a project that uses a Raspberry Pi (armv7l) and the latest LDC version I found for this architecture is 1.13.0. Can you help me install the latest version(1.20.0)? Also, I'm having problems using the DPP package with the 1.13.0 LDC version, most likely because the latest version of DPP(0.4.1) is using a newer version of Phobos. Is there any reason why the LDC team stopped releasing pre-built binaries for arm? I'm looking forward to your help. Teona.
GtkD - Add images to IconView with Pixbuf
Hello. :-) I found this guide : https://www.kksou.com/php-gtk2/sample-codes/display-a-list-of-thumbnail-images-using-GtkIconView.php This guide used 'Pixbuf' for use fromFile method, but GtkD don't have it. : https://api.gtkd.org/gdkpixbuf.Pixbuf.Pixbuf.html Is there any other way? Thanks.
Re: in not working for arrays is silly, change my view
On 3/2/20 6:39 AM, JN wrote: On Saturday, 29 February 2020 at 21:56:51 UTC, Ali Çehreli wrote: Because you mentioned canFind, I think you want the semantics to be "is there an element with this value." If so, it would be confusing to use the same operator for two different things: For associative arrays, it means "is there an element accessible with this key." Does it? I always viewed it as "is this value in list of keys" Array keys are the element index.. So essentially: int[int] c1; int[] c2 = new int[4]; c1[3] = 10; c2[3] = 10; assert(3 in c1); // true assert(3 in c2); // what should this do? -Steve
Re: in not working for arrays is silly, change my view
On 3/2/20 6:52 AM, Andrea Fontana wrote: On Saturday, 29 February 2020 at 20:11:24 UTC, Steven Schveighoffer wrote: 1. in is supposed to be O(lg(n)) or better. Generic code may depend on this property. Searching an array is O(n). Probably it should work if we're using a "SortedRange". int[] a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]; auto p = assumeSorted(a); assert(3 in p); That could work. Currently, you need to use p.contains(3). opIn could be added as a shortcut. It only makes sense if you have it as a literal though, as p.contains(3) isn't that bad to use: assert(3 in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].assumeSorted); -Steve
Re: Improving dot product for standard multidimensional D arrays
On Monday, 2 March 2020 at 13:35:15 UTC, p.shkadzko wrote: [snip] Thanks. I don't have time right now to review this thoroughly. My recollection is that the dot product of two matrices is actually matrix multiplication, correct? It generally makes sense to defer to other people's implementation of this. I recommend trying lubeck's version against numpy. It uses a blas/lapack implementation. mir-glas, I believe, also has a version. Also, I'm not sure if the fastmath attribute would do anything here, but something worth looking into.
Re: Improving dot product for standard multidimensional D arrays
On Monday, 2 March 2020 at 11:33:25 UTC, jmh530 wrote:
On Sunday, 1 March 2020 at 20:58:42 UTC, p.shkadzko wrote:
Hello again,
[snip]
What compiler did you use and what flags?
Ah yes, sorry. I used latest ldc2 (1.20.0-x64) for Windows.
Dflags -mcpu=native and "inline", "optimize", "releaseMode".
Here is a dub.json of the project:
{
"name": "app",
"targetType": "executable",
"dependencies": {
"mir": "~>3.2.0"
},
"dflags-ldc": ["-mcpu=native"],
"buildTypes": {
"release": {
"buildOptions": ["releaseMode", "inline", "optimize"],
"dflags": ["-boundscheck=off"]
},
"tests": {
"buildOptions": ["unittests"]
}
}
}
Re: in not working for arrays is silly, change my view
On Saturday, 29 February 2020 at 20:11:24 UTC, Steven Schveighoffer wrote: 1. in is supposed to be O(lg(n)) or better. Generic code may depend on this property. Searching an array is O(n). Probably it should work if we're using a "SortedRange". int[] a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]; auto p = assumeSorted(a); assert(3 in p);
Re: in not working for arrays is silly, change my view
On Saturday, 29 February 2020 at 21:56:51 UTC, Ali Çehreli wrote: Because you mentioned canFind, I think you want the semantics to be "is there an element with this value." If so, it would be confusing to use the same operator for two different things: For associative arrays, it means "is there an element accessible with this key." Does it? I always viewed it as "is this value in list of keys" Unless 'in' works with arrays to mean "is this index valid", then I don't see the benefit. If we had it, I think more people would ask "why does 'in' work differently for arrays?" Are there other languages that support this semantic? Checking... Ok, Python has it, highly likely because they don't have arrays to begin with. Well, Python lists are for most purposes equivalent to arrays and it hasn't really been confusing for people.
Re: Improving dot product for standard multidimensional D arrays
On Sunday, 1 March 2020 at 20:58:42 UTC, p.shkadzko wrote: Hello again, [snip] What compiler did you use and what flags?
