Re: Implementing Haskell's Type-Level Quicksort in D
Philippe Sigaud: So yes, D does not have Haskell nice syntax for pattern matching. I'd like some of such syntax for templates (and a little different syntax added to match structs inside switch statements: https://d.puremagic.com/issues/show_bug.cgi?id=596 ). Bye, bearophile
Re: how to iterate over an AA by key-value pair (tuple)?
Timothee Cour:
auto byKeyValue(){...}
It's probably better to call the method byPair that is shorter.
If I/someone does it, will it be merged in?
Some people want those pairs to be tuples. But D lacks built-in
tuples, so you need to use typecons ones. But to use the typecons
ones you need to import half Phobos. It seems there is no good
solution.
Bye,
bearophile
Re: how to iterate over an AA by key-value pair (tuple)?
On Friday, 14 February 2014 at 07:35:34 UTC, Timothee Cour wrote: That seems like a worthy enhancement. If I/someone does it, will it be merged in? I really want it to happen, but I also want it to happen right. See the relevant pull request[1]. [1] https://github.com/D-Programming-Language/druntime/pull/574
Re: Ranges, constantly frustrating
On Fri, 14 Feb 2014 02:48:51 -, Jesse Phillips jesse.k.phillip...@gmail.com wrote: On Thursday, 13 February 2014 at 14:30:41 UTC, Regan Heath wrote: Don't get me wrong, counting the elements as you iterate over them is useful, but it isn't the index into the range you're likely after. Nope, not what I am after. If I was, I'd iterate over the original range instead or keep a line count manually. Maybe a better way to phrase this is, while counting may be what you're implementation needs, it is not immediately obvious what 'i' should be. Someone who desires an index into the original array will expect 'i' to be that; even though it can be explained that .take() is not the same range as the original. Thus it is better to be explicit with the .enumerate function. FWIW I disagree. I think it's immediately and intuitively obvious what 'i' should be when you're foreaching over X items taken from another range, even if you do not know take returns another range. Compare it to calling a function on a range and foreaching on the result, you would intuitively and immediately expect 'i' to relate to the result, not the input. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Ranges, constantly frustrating
On Friday, 14 February 2014 at 12:10:51 UTC, Regan Heath wrote: FWIW I disagree. I think it's immediately and intuitively obvious what 'i' should be when you're foreaching over X items taken from another range, even if you do not know take returns another range. Compare it to calling a function on a range and foreaching on the result, you would intuitively and immediately expect 'i' to relate to the result, not the input. R How should it behave on ranges without length, such as infinite ranges? Also, `enumerate` has the advantage of the `start` parameter, which usefulness is demonstrated in `enumerate`'s example as well as in an additional example in the bug report. I'm not yet sure whether I think it should be implemented at the language or library level, but I think the library approach has some advantages.
inverse of escapeShellCommand?
is there a function to get the inverse of escapeShellCommand? ie: assert(escapeShellCommandInverse(` foo 'hello world' `)==[`foo`, `hello world`]);
Re: Ranges, constantly frustrating
Regan Heath:
FWIW I disagree. I think it's immediately and intuitively
obvious what 'i' should be when you're foreaching over X items
taken from another range, even if you do not know take returns
another range. Compare it to calling a function on a range and
foreaching on the result, you would intuitively and immediately
expect 'i' to relate to the result, not the input.
Using enumerate has several advantages. It gives a bit longer
code, but it keeps as much complexity as possible out of the
language. So the language gets simpler to implement and its
compiler is smaller and simpler to debug.
Also, using enumerate is more explicit, if you have an
associative array you can iterate it in many ways:
foreach (v; AA) {}
foreach (k, v; AA) {}
foreach (k; AA.byKeys) {}
foreach (i, k; AA.byKeys.enumerate) {}
foreach (i, v; AA.byValues.enumerate) {}
foreach (k, v; AA.byPairs) {}
foreach (i, k, v; AA.byPairs.enumerate) {}
If you want all those schemes built in a language (and to use
them without adding .enumerate) you risk making a mess. In this
case explicit is better than implicit.
Python does the same with its enumerate function and keeps the
for loop simple:
for k in my_dict: pass
for i, v in enumerate(my_dict.itervalues()): pass
etc.
In D we have a mess because tuples are not built-in. Instead of
having a built-in functionality similar to what enumerate does,
it's WAY better to have built-in tuples. Finding what's important
and what is not important to have as built-ins in a language is
an essential and subtle design problem.
Bye,
bearophile
Re: Implementing Haskell's Type-Level Quicksort in D
On Friday, 14 February 2014 at 06:05:08 UTC, Philippe Sigaud
wrote:
`alias` is just a bit of syntax sugar, it does not (at least for
2.064) have the same power than fully defining a template and
the `is(...)` expression.
Right. What I was saying, however, is it is strange to me that
this code will compile:
alias numPred(N: Succ!a, a) = a;
struct Number(a)
if (is(a == Zero) || is(a == Succ!x, x))
{}
enum numValue(N: Number!Zero) = 0;
enum numValue(N: Number!x, x) = numValue!(Number!(numPred!x)) + 1;
assert(numValue!(Number!Zero) == 0);
assert(numValue!(Number!Four) == 4);
While this code won't:
struct Nil{}
struct Cons(a, b) {}
alias list1 = Cons!(Three, Cons!(Two, Cons!(Four, Cons!(One,
Nil;
alias numlHead(L: Cons!(a, b), a, b) = a;
alias numlTail(L: Cons!(a, b), a, b) = b;
assert(is(numlHead!list1 == Three));
Since list1 is a roughly similar construction to
Number!(Succ!(Succ!(Succ!(Succ!Zero, it is surprising to me
that the template matching system can correctly destructure the
latter, while not the former. This is obviously a bug, I'm just
surprised that it exists.
So yes, D does not have Haskell nice syntax for pattern
matching. You can do some pattern matching using templates,
but it tends to be a bit heavier than ML/F#/Haskell.
While it is heavier than Haskell's syntax, I have been
consistently and pleasantly surprised by how powerful D's
template pattern matching is (bugs notwithstanding). I wonder how
well-known this is outside this mailing list...
(Some would say that at least D does not force you to encode
numbers as succ/zero list, since you can use numbers directly
as template args, but that's another story).
Sure, but I want to stay as close to the original Haskell as
possible (and Peano Numbers are pretty damn cool anyway).
Re: Implementing Haskell's Type-Level Quicksort in D
Meta: While it is heavier than Haskell's syntax, I have been consistently and pleasantly surprised by how powerful D's template pattern matching is (bugs notwithstanding). I wonder how well-known this is outside this mailing list... I keep reading blog posts that use Haskell and present supposed marvels, using very complex things, that can be routinely done in D, and with less complexity for the brain of the programer, often leading to faster code and equally safe :-) So while I like several parts of Haskell, it's sometimes over-hyped (while D is nearly invisible). Bye, bearophile
Optimize my code =)
Hiho,
I am fairly new to the D programming and still reading throught
the awesome online book at http://ddili.org/ders/d.en/index.html.
However, I think it is missing some things or I am missing
glasses and overlooked these parts in the book.^^
Currently I am trying to write a very simple Matrix library for
some minor linear algebra operations such as calculating
determine, some simple compositions etc. I am aware that this
probably has been done already and I am mainly focusing learning
D with this project. =)
I already wrote a similar Matrix library for Java and C++ with
more or less the same algorithms in order to benchmark these
languages.
As I am very new to D I instantly ran into certain optimizing
issues. E.g. the simple matrix multiplication based in my java
implementation requires about 1.5 secs for multiplying two
1000x1000 matrices, however my D implementation requires 39
seconds without compiler optimizations and about 14 seconds with
-inline, -O, -noboundscheck and -release. So I am sure that I
just missed some possible optimization routines for D and that I
could miss entire patters used in D to write more optimized code
- especially when looking at the const and immutability concepts.
I won't paste the whole code, just the important parts.
class Matrix(T = double) {
private T[] data;
private Dimension dim;
}
This is my class with its templated data as a one dimensional
array (as I don't like jagged-arrays) and a dimension (which is a
struct). The dimension object has some util functionality such as
getting the total size or mapping (row, col) to a one-dimensional
index besides the getter for rows and columns.
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
This is one of the constructor. Its task is just to create a new
Matrix instance filled with what is 0 for the templated value -
don't know if there is a better approach for this. I experienced
that floating point values are sadly initialized with nan which
isn't what I wanted - double.init = nan.
I am using opIndex and opIndexAssign in order to access and
assign the matrix values:
T opIndex(size_t row, size_t col) const {
immutable size_t i = this.dim.offset(row, col);
if (i = this.dim.size) {
// TODO - have to learn exception handling in D first. :P
}
return this.data[i];
}
Which calls:
size_t size() @property const {
return this.rows * this.cols;
}
I think and hope that this is getting optimized via inlining. =)
This works similar for opIndexAssign.
The function I am mainly benchmarking is the simple matrix
multiplication where one of the multiplied matrices is tranposed
first in order to improve cache hit ratio.
Matrix opMul(ref const(Matrix) other) {
if (this.dim.rows != other.dim.cols || this.dim.cols !=
ther.dim.rows) {
// TODO - still have to learn exception handling first ...
}
auto m = new Matrix(this.dim.rows, other.dim.cols);
auto s = new Matrix(other).transposeAssign();
size_t r1, r2, c;
T sum;
for (r1 = 0; r1 this.dim.rows; ++r1) {
for (r2 = 0; r2 other.dim.rows; ++r2) {
sum = to!T(0);
for (c = 0; c this.dim.cols; ++c) {
sum += this[r1, c] * other[r2, c];
}
m[r1, r2] = sum;
}
}
return m;
}
I am aware that there are faster algorithms for matrix
multiplication but I am mainly learning how to write efficient D
code in general and not based on any algorithm.
This is more or less the same algorithm I am using with java and
c++. Both require about 1.5 secs (after java warm-up) to perform
this task for two 1000x1000 matrices. D compiled with DMD takes
about 14 seconds with all (known-to-me) optimize flag activated.
(listed above)
I wanted to make s an immutable matrix in order to hopefully
improve performance via this change, however I wasn't technically
able to do this to be honest.
Besides that I can't find a way how to make a use of move
semantics like in C++. For example a rvalue-move-constructor or a
move-assign would be a very nice thing for many tasks.
I am not quite sure also if it is normal in D to make an idup
function as slices have which creates an immutable copy of your
object. And are there important things I have to do while
implementing an idup method for this matrix class?
Another nice thing to know would be if it is possible to
initialize an array before it is default initialized with T.init
where T is the type of the array's fields. In C++ e.g. there is
no default initialization which is nice if you have to initialize
every single field anyway. E.g. in a Matrix.random()
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:00:09 UTC, Robin wrote:
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
I am no expert at optimizing D code, so this is a bit of a shot
in the dark, but does your speed improve at all if you replace:
this.data = new T[this.dim.size];
with:
this.data.length = this.dim.size
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:40:31 UTC, Craig Dillabaugh
wrote:
On Friday, 14 February 2014 at 16:00:09 UTC, Robin wrote:
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
I am no expert at optimizing D code, so this is a bit of a shot
in the dark, but does your speed improve at all if you replace:
this.data = new T[this.dim.size];
with:
this.data.length = this.dim.size
Why would that make a difference here? They're (almost) identical
are they not? Certainly the body of the work, the allocation
itself, is the same.
Re: Optimize my code =)
Robin:
class Matrix(T = double) {
private T[] data;
private Dimension dim;
}
Also try final class or struct in your benchmark. And try to
use ldc2 compiler for performance benchmarks.
Perhaps dim is better const, unless you want to change the shape
of the matrix.
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
Better:
this(in size_t rows, in size_t cols) pure nothrow {
this.dim = Dimension(rows, cols);
this.data =
minimallyInitializedArray!(typeof(data))(this.dim.size);
this.data[] = to!T(0);
}
I experienced that floating point values are sadly initialized
with nan which isn't what I wanted - double.init = nan.
This is actually a good feature of D language :-)
T opIndex(size_t row, size_t col) const {
immutable size_t i = this.dim.offset(row, col);
if (i = this.dim.size) {
// TODO - have to learn exception handling in D first. :P
}
Look in D docs for the contract programming.
Also add the pure nothrow attributes.
Which calls:
size_t size() @property const {
return this.rows * this.cols;
}
I think and hope that this is getting optimized via inlining. =)
Currently class methods are virtual, and currently D compilers
are not inlining virtual calls much. The solution is to use a
final class, final methods, etc.
This works similar for opIndexAssign.
The function I am mainly benchmarking is the simple matrix
multiplication where one of the multiplied matrices is
tranposed first in order to improve cache hit ratio.
Transposing the whole matrix before the multiplication is not
efficient. Better to do it more lazily.
auto m = new Matrix(this.dim.rows, other.dim.cols);
auto s = new Matrix(other).transposeAssign();
size_t r1, r2, c;
T sum;
for (r1 = 0; r1 this.dim.rows; ++r1) {
for (r2 = 0; r2 other.dim.rows; ++r2) {
Better to define r1, r2 inside the for. Or often even better to
use a foreach with interval.
D compiled with DMD takes about 14 seconds with all
(known-to-me) optimize flag activated. (listed above)
DMD is not efficient with floating point values. Try ldc2 or gdc
compilers (I suggest ldc2).
I wanted to make s an immutable matrix in order to hopefully
improve performance via this change,
Most times immutability worsens performance, unless you are
passing data across threads.
however I wasn't technically able to do this to be honest.
It's not too much hard to use immutability in D.
Besides that I can't find a way how to make a use of move
semantics like in C++. For example a rvalue-move-constructor or
a move-assign would be a very nice thing for many tasks.
That adds too much complexity to .
Another nice thing to know would be if it is possible to
initialize an array before it is default initialized with
T.init where T is the type of the array's fields. In C++ e.g.
there is no default initialization which is nice if you have to
initialize every single field anyway. E.g. in a Matrix.random()
method which creates a matrix with random values. There it is
unnecessary that the (sometimes huge) array is completely
initialized with the type's init value.
It's done by the function that I have used above, from the
std.array module.
Bye,
bearophile
Re: Optimize my code =)
Craig Dillabaugh: this.data = new T[this.dim.size]; with: this.data.length = this.dim.size It's the same thing. Bye, bearophile
Re: Ranges, constantly frustrating
On Fri, 14 Feb 2014 13:14:51 -, bearophile bearophileh...@lycos.com wrote: Regan Heath: FWIW I disagree. I think it's immediately and intuitively obvious what 'i' should be when you're foreaching over X items taken from another range, even if you do not know take returns another range. Compare it to calling a function on a range and foreaching on the result, you would intuitively and immediately expect 'i' to relate to the result, not the input. Using enumerate has several advantages. In my case I didn't need any of these. Simple things should be simple and intuitive to write. Yes, we want enumerate *as well* especially for the more complex cases but we also want the basics to be simple, intuitive and easy. That's all I'm saying here. This seems to me to be very low hanging fruit. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Ranges, constantly frustrating
On Fri, 14 Feb 2014 12:29:49 -, Jakob Ovrum jakobov...@gmail.com wrote: On Friday, 14 February 2014 at 12:10:51 UTC, Regan Heath wrote: FWIW I disagree. I think it's immediately and intuitively obvious what 'i' should be when you're foreaching over X items taken from another range, even if you do not know take returns another range. Compare it to calling a function on a range and foreaching on the result, you would intuitively and immediately expect 'i' to relate to the result, not the input. R How should it behave on ranges without length, such as infinite ranges? In exactly the same way. It just counts up until you break out of the foreach, or the 'i' value wraps around. In fact the behaviour I want is so trivial I think it could be provided by foreach itself, for iterations of anything. In which case whether 'i' was conceptually an index or simply a count would depend on whether the range passed to foreach (after all skip, take, etc) was itself indexable. Also, `enumerate` has the advantage of the `start` parameter, which usefulness is demonstrated in `enumerate`'s example as well as in an additional example in the bug report. Sure, if you need more functionality reach for enumerate. We can have both; sensible default behaviour AND enumerate for more complicated cases. In my case, enumerate w/ start wouldn't have helped (my file was blocks of 6 lines, where I wanted to skip lines 1, 3, and 6 *of each block*) I'm not yet sure whether I think it should be implemented at the language or library level, but I think the library approach has some advantages. Certainly, for the more complex usage. But I reckon we want both enumerate and a simple language solution which would do what I've been trying to describe. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:47:32 UTC, John Colvin wrote:
On Friday, 14 February 2014 at 16:40:31 UTC, Craig Dillabaugh
wrote:
On Friday, 14 February 2014 at 16:00:09 UTC, Robin wrote:
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
I am no expert at optimizing D code, so this is a bit of a
shot in the dark, but does your speed improve at all if you
replace:
this.data = new T[this.dim.size];
with:
this.data.length = this.dim.size
Why would that make a difference here? They're (almost)
identical are they not? Certainly the body of the work, the
allocation itself, is the same.
Well, in my defense I did start with a disclaimer. For some
reason I thought using .length was the proper way to size arrays.
It is likely faster if you are resizing an existing array
(especially downward), but in this case new memory is being
allocated anyway.
In fact I ran some tests (just allocating a matrix over and over)
and it seems using new is consistently faster than using .length
(by about a second for 5 matrices). Shows how much I know.
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:00:09 UTC, Robin wrote:
This is my class with its templated data as a one dimensional
array (as I don't like jagged-arrays) and a dimension (which is
a struct). The dimension object has some util functionality
such as getting the total size or mapping (row, col) to a
one-dimensional index besides the getter for rows and columns.
Are you sure you need a class here? If you're not using
inheritance, structs can be much faster.
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
You don't need to use std.conv.to here (ditto for when you
initialize sum later). This should work and is clearer:
foreach(ref T element; this.data) element = 0;
(You can do `double item = 0;` and it knows enough to store the
double equivalent of 0)
In the case of initializing sum, I'm not sure the compiler is
folding `sum = to!T(0);` thus you could be getting a lot of
overhead from that. Using std.conv.to is fine but it does
actually do checks in the conversion process, so it can be
expensive in tight loops. Since you're just using a constant 0
here, there's no reason to use std.conv.to.
The function I am mainly benchmarking is the simple matrix
multiplication where one of the multiplied matrices is
tranposed first in order to improve cache hit ratio.
Did you benchmark first to make sure this actually improves
performance? It seems to me like doing the transpose would
require the same cache-missing that you'd get in the actual use.
If you were caching it and using it multiple times, this would
probably be more beneficial. General tip for optimizing:
benchmark before and after every optimization you do. I've been
surprised to find some of my ideas for optimizations slowed
things down before.
Another nice thing to know would be if it is possible to
initialize an array before it is default initialized with
T.init where T is the type of the array's fields.
http://dlang.org/phobos/std_array.html#.uninitializedArray
Re: Ranges, constantly frustrating
Regan Heath: In my case I didn't need any of these. I don't understand. Bye, bearophile
Re: Ranges, constantly frustrating
Isn't this discussion about adding an index to a range? If it is, then I have shown why adding it in the language is a bad idea. Bye, bearophile
Re: Optimize my code =)
Chris Cain: http://dlang.org/phobos/std_array.html#.uninitializedArray minimallyInitializedArray should be used, because it's safer. Bye, bearophile
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:00:09 UTC, Robin wrote:
class Matrix(T = double) {
T opIndex(size_t row, size_t col) const {
First of all make sure you it's not virtual, otherwise each
element access will cost you enough to make it 10x slower than
Java.
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:56:29 UTC, bearophile wrote: Craig Dillabaugh: this.data = new T[this.dim.size]; with: this.data.length = this.dim.size It's the same thing. Bye, bearophile Not quite. Setting length will copy over the existing contents of the array. Using new simply sets every element to .init. Granted, this.data is empty meaning there's nothing to copy over, so there's a negligible overhead which may be optimized out by the compiler anyways. There's also the uninitializedArray function: http://dlang.org/phobos/std_array.html#.uninitializedArray
Re: Optimize my code =)
On Friday, 14 February 2014 at 16:00:09 UTC, Robin wrote:
As I am very new to D I instantly ran into certain optimizing
issues. E.g. the simple matrix multiplication based in my java
implementation requires about 1.5 secs for multiplying two
1000x1000 matrices, however my D implementation requires 39
seconds without compiler optimizations and about 14 seconds
with -inline, -O, -noboundscheck and -release. So I am sure
that I just missed some possible optimization routines for D
and that I could miss entire patters used in D to write more
optimized code - especially when looking at the const and
immutability concepts.
Try with and without -noboundscheck. Sometimes using it seems to
make things slower, which is odd.
Also, compiling for 64-bit may help significantly. When you
compile for 32-bit, you won't get benefits of SIMD instructions
as it's not guaranteed to be supported by the CPU. Since Java
runs a JIT, it would use these SIMD instructions.
Also, you'll get significantly faster code if you use LDC or GDC.
class Matrix(T = double) {
private T[] data;
private Dimension dim;
}
This should be final class or struct, you don't need virtual
methods.
This is my class with its templated data as a one dimensional
array (as I don't like jagged-arrays) and a dimension (which is
a struct). The dimension object has some util functionality
such as getting the total size or mapping (row, col) to a
one-dimensional index besides the getter for rows and columns.
this(size_t rows, size_t cols) {
this.dim = Dimension(rows, cols);
this.data = new T[this.dim.size];
enum nil = to!T(0);
foreach(ref T element; this.data) element = nil;
}
You don't need to!T for setting to 0, just use plain 0. Using
this to will probably result in overhead. Note that this may
evaluated every time you use 'nil', so using a constant will help.
T opIndex(size_t row, size_t col) const {
immutable size_t i = this.dim.offset(row, col);
if (i = this.dim.size) {
// TODO - have to learn exception handling in D first. :P
}
return this.data[i];
}
You're using -noboundscheck then manually implementing bounds
checks, and probably less efficiently than the compiler does.
Either change this to an assert, or remove it. In either case,
the compiler will do it's own built in bounds checking.
This call is especially expensive since you're doing entirely
virtual methods because you specified class instead of final
class or using a struct. None of your Matrix calls will be
inlined.
Which calls:
size_t size() @property const {
return this.rows * this.cols;
}
I think and hope that this is getting optimized via inlining. =)
This works similar for opIndexAssign.
Not unless you final class / struct.
The function I am mainly benchmarking is the simple matrix
multiplication where one of the multiplied matrices is
tranposed first in order to improve cache hit ratio.
Matrix opMul(ref const(Matrix) other) {
if (this.dim.rows != other.dim.cols || this.dim.cols !=
ther.dim.rows) {
// TODO - still have to learn exception handling first ...
}
auto m = new Matrix(this.dim.rows, other.dim.cols);
auto s = new Matrix(other).transposeAssign();
size_t r1, r2, c;
T sum;
for (r1 = 0; r1 this.dim.rows; ++r1) {
for (r2 = 0; r2 other.dim.rows; ++r2) {
sum = to!T(0);
for (c = 0; c this.dim.cols; ++c) {
sum += this[r1, c] * other[r2, c];
}
m[r1, r2] = sum;
}
}
return m;
}
These allocations may potentially hurt.
Also, again, the virtual calls here are a huge hit.
Using to!T(0) is also a waste of performance, as youc an just do
straight 0.
I am aware that there are faster algorithms for matrix
multiplication but I am mainly learning how to write efficient
D code in general and not based on any algorithm.
Using SIMD instructions manually would of course be much, much,
faster, but I understand that it defeats the point somewhat and
is much more ugly.
This is more or less the same algorithm I am using with java
and c++. Both require about 1.5 secs (after java warm-up) to
perform this task for two 1000x1000 matrices. D compiled with
DMD takes about 14 seconds with all (known-to-me) optimize flag
activated. (listed above)
If you used LDC or GDC 64-bit with the changes above, I'd guess
it would be similar. I wouldn't expect D to out-perform Java much
for this particular scenario, there's very little going on and
the JIT should be able to optimize it quite well with SIMD stuff.
I wanted to make s an immutable matrix in order to hopefully
improve performance via this change, however I wasn't
technically able to do this to be honest.
I don't think this would improve performance.
Reading input from piped stdin.
I'm new to D, and I find it quite enjoyable so far.
I have however stumbled upon a problem which I can't seem to
figure out.
I am trying to make a program that creates a child process,
writes something to the child process stdin and reading from its
stdout. I am going to use it later for testing out process pair
redundancy.
Appearently the child blocks at s = stdin.readln(). If I remove
all writing to the child, and instead just read its output,
everything works fine. My code is attached below:
import std.process,std.stdio,std.getopt,core.thread;
void main(string[] args){
bool backup = false;
getopt(args, backup, backup);
writeln(Something worked!);
string s = test;
if (backup){
writeln(Backup up running);
while(true){
s = stdin.readln();
writeln(s);
}
}
auto pipes = pipeProcess([./pipetest, --backup],
Redirect.all);
for(int j = 0; j5; j++){
writeln(j);
pipes.stdin.writeln(j);
writeln(pipes.stdout.readln());
Thread.sleep(500.msecs);
}
while(true){}
}
If anyone could spot what rudimentary mistake I have done, I
would greatly appreciate it. Alternatively, suggesting another
way to implement inter-process communication would also be
appreciated :D
Re: Implementing Haskell's Type-Level Quicksort in D
On Fri, Feb 14, 2014 at 3:24 PM, bearophile bearophileh...@lycos.com wrote: Meta: While it is heavier than Haskell's syntax, I have been consistently and pleasantly surprised by how powerful D's template pattern matching is (bugs notwithstanding). I wonder how well-known this is outside this mailing list... Well, I have a tutorial, but I admit it being somewhat incomplete and now partially out of date (2012). I keep reading blog posts that use Haskell and present supposed marvels, using very complex things, that can be routinely done in D, and with less complexity for the brain of the programer, often leading to faster code and equally safe :-) So while I like several parts of Haskell, it's sometimes over-hyped (while D is nearly invisible). Same here, same here! But you have to admit some of their examples are pretty damn elegant :) I remember doing some Haskell - D translation (a bit like what Meta is doing), making it work, doing the happy-happy dance, and then suddenly realizing I could do the same compile-time computation in one line of D ;)
Re: Reading input from piped stdin.
Just a quick look, but I betcha it has to do with buffering. After writing the line to the pipe, call the flush() method on the output pipe and see what happens there. (Pipes buffer differently than regular output so this is a common mixup, especially with IDEs which communicate with stdout via pipes normally!) let me know if it works
Re: Reading input from piped stdin.
On Friday, 14 February 2014 at 19:05:02 UTC, Thomas wrote:
I'm new to D, and I find it quite enjoyable so far.
I have however stumbled upon a problem which I can't seem to
figure out.
I am trying to make a program that creates a child process,
writes something to the child process stdin and reading from its
stdout. I am going to use it later for testing out process pair
redundancy.
Appearently the child blocks at s = stdin.readln(). If I
remove
all writing to the child, and instead just read its output,
everything works fine. My code is attached below:
import std.process,std.stdio,std.getopt,core.thread;
void main(string[] args){
bool backup = false;
getopt(args, backup, backup);
writeln(Something worked!);
string s = test;
if (backup){
writeln(Backup up running);
while(true){
s = stdin.readln();
writeln(s);
}
}
auto pipes = pipeProcess([./pipetest, --backup],
Redirect.all);
for(int j = 0; j5; j++){
writeln(j);
pipes.stdin.writeln(j);
writeln(pipes.stdout.readln());
Thread.sleep(500.msecs);
}
while(true){}
}
If anyone could spot what rudimentary mistake I have done, I
would greatly appreciate it. Alternatively, suggesting another
way to implement inter-process communication would also be
appreciated :D
Maybe try closing stdin pipe after you are done writing to child.
pipes.stdin.close();
Or try flushing after writing to childs stdin, IIRC:
pipes.stdin.flush();
Re: Reading input from piped stdin.
On Fri, 14 Feb 2014 14:05:01 -0500, Thomas sitronv...@gmail.com wrote: I'm new to D, and I find it quite enjoyable so far. I have however stumbled upon a problem which I can't seem to figure out. I am trying to make a program that creates a child process, writes something to the child process stdin and reading from its stdout. I am going to use it later for testing out process pair redundancy. Appearently the child blocks at s = stdin.readln(). If I remove all writing to the child, and instead just read its output, everything works fine. My code is attached below: stdin and stdout are buffered streams. Buffered streams in D only flush on newline when they are attached to a console (terminal window). Otherwise, they wait until the buffer is full before flushing. Buffer is probably about 4096 bytes. What is likely happening is that you are writing, it's going into the buffer, but not flushing, and then you are waiting for the response (which likely is also not flushing from the child process). Try adding flushes after each writeln. BTW, this is not really a D problem, you would have the same issue in C. -Steve
Re: Reading input from piped stdin.
On Friday, 14 February 2014 at 19:09:06 UTC, nazriel wrote:
On Friday, 14 February 2014 at 19:05:02 UTC, Thomas wrote:
I'm new to D, and I find it quite enjoyable so far.
I have however stumbled upon a problem which I can't seem to
figure out.
I am trying to make a program that creates a child process,
writes something to the child process stdin and reading from
its
stdout. I am going to use it later for testing out process pair
redundancy.
Appearently the child blocks at s = stdin.readln(). If I
remove
all writing to the child, and instead just read its output,
everything works fine. My code is attached below:
import std.process,std.stdio,std.getopt,core.thread;
void main(string[] args){
bool backup = false;
getopt(args, backup, backup);
writeln(Something worked!);
string s = test;
if (backup){
writeln(Backup up running);
while(true){
s = stdin.readln();
writeln(s);
}
}
auto pipes = pipeProcess([./pipetest, --backup],
Redirect.all);
for(int j = 0; j5; j++){
writeln(j);
pipes.stdin.writeln(j);
writeln(pipes.stdout.readln());
Thread.sleep(500.msecs);
}
while(true){}
}
If anyone could spot what rudimentary mistake I have done, I
would greatly appreciate it. Alternatively, suggesting another
way to implement inter-process communication would also be
appreciated :D
Maybe try closing stdin pipe after you are done writing to
child.
pipes.stdin.close();
Ok, nvm.
You are reading from child after each write.
So naa, closing pipe won't do it.
So we are back to flushing :)
Or try flushing after writing to childs stdin, IIRC:
pipes.stdin.flush();
Re: Reading input from piped stdin.
On Friday, 14 February 2014 at 19:08:20 UTC, Adam D. Ruppe wrote: Just a quick look, but I betcha it has to do with buffering. After writing the line to the pipe, call the flush() method on the output pipe and see what happens there. (Pipes buffer differently than regular output so this is a common mixup, especially with IDEs which communicate with stdout via pipes normally!) let me know if it works Impressive reply speed! :) However calling pipes.stdin.flush() immediately after writeln did not seem to work.
Re: Reading input from piped stdin.
On Fri, 14 Feb 2014 14:16:23 -0500, Thomas sitronv...@gmail.com wrote: On Friday, 14 February 2014 at 19:08:20 UTC, Adam D. Ruppe wrote: Just a quick look, but I betcha it has to do with buffering. After writing the line to the pipe, call the flush() method on the output pipe and see what happens there. (Pipes buffer differently than regular output so this is a common mixup, especially with IDEs which communicate with stdout via pipes normally!) let me know if it works Impressive reply speed! :) However calling pipes.stdin.flush() immediately after writeln did not seem to work. You must also flush from the child process -- it too is connected to a pipe and not a console. -Steve
Re: Reading input from piped stdin.
On Friday, 14 February 2014 at 19:12:24 UTC, Steven Schveighoffer wrote: On Fri, 14 Feb 2014 14:05:01 -0500, Thomas sitronv...@gmail.com wrote: I'm new to D, and I find it quite enjoyable so far. I have however stumbled upon a problem which I can't seem to figure out. I am trying to make a program that creates a child process, writes something to the child process stdin and reading from its stdout. I am going to use it later for testing out process pair redundancy. Appearently the child blocks at s = stdin.readln(). If I remove all writing to the child, and instead just read its output, everything works fine. My code is attached below: stdin and stdout are buffered streams. Buffered streams in D only flush on newline when they are attached to a console (terminal window). Otherwise, they wait until the buffer is full before flushing. Buffer is probably about 4096 bytes. What is likely happening is that you are writing, it's going into the buffer, but not flushing, and then you are waiting for the response (which likely is also not flushing from the child process). Try adding flushes after each writeln. BTW, this is not really a D problem, you would have the same issue in C. -Steve Ah, had to add after both writelns. Looks like that solved it. Thanks!
DGUI
I recently downloaded and tried to use DGUI but I can't get it to work. Is there any tutorials on how to build an use it? Or can anyone help me and tel me a way on how I can get it to work?
Re: DGUI
On Friday, 14 February 2014 at 20:29:50 UTC, Josh Phillips wrote: I recently downloaded and tried to use DGUI but I can't get it to work. Is there any tutorials on how to build an use it? Or can anyone help me and tel me a way on how I can get it to work? Unless I am mistaken, it looks like the last time it was updated was in 2011. You'd probably have to update a bunch of stuff in the sources in order to get it to work properly. I would recommend gtkD as a gui library.
Re: Ranges, constantly frustrating
On Friday, 14 February 2014 at 17:42:53 UTC, bearophile wrote: Isn't this discussion about adding an index to a range? If it is, then I have shown why adding it in the language is a bad idea. As far as I understand it, it's about adding an index to _foreach_, as is already supported for arrays: foreach(v; [1,2,3,4]) writeln(v); foreach(i, v; [1,2,3,4]) writeln(i, = , v); But for ranges, the second form is not possible: foreach(v; iota(4)) // ok writeln(v); foreach(i, v; iota(4))// Error: cannot infer argument types writeln(i, = , v);
Re: how to round durations to most significant unit ? (5 secs, 889 ms, and 811 μs = 5 secs)
On Thursday, February 13, 2014 23:37:13 Timothee Cour wrote:
Is there a function to do this?
If not and I/someone writes it, is there interest to add it to std.datetime?
Duration t = ...;
t.to!string = 5 secs, 889 ms, and 811 μs
t.round.to!string= 5 secs
t=...;
t.to!string = 889 ms, and 811 μs
t.round.to!string= 889 secs
Use case: shorter logs.
There is no function for that, and no one's ever asked for anything like it,
so I don't know how worthwhile it is to add. However, one possible
implementation would be
auto roundToLargest(Duration d)
{
foreach(units; TypeTuple!(weeks, days, hours, minutes,
seconds, msecs, usecs))
{
immutable value = d.total!units();
if(value != 0)
return dur!units(value);
}
return d;
}
- Jonathan M Davis
Re: Optimize my code =)
On 2/14/2014 11:00 AM, Robin wrote:
class Matrix(T = double) {
private T[] data;
private Dimension dim;
}
A matrix is just plain-old-data, so use a struct, you don't need a class.
A struct will be much more lightweight: A struct doesn't normally
involve memory allocations like a class does, and you'll get better data
locality and less indirection, even compared to a final class.
I am using opIndex and opIndexAssign in order to access and assign the
matrix values:
T opIndex(size_t row, size_t col) const {
immutable size_t i = this.dim.offset(row, col);
if (i = this.dim.size) {
// TODO - have to learn exception handling in D first. :P
}
return this.data[i];
}
No need for the bounds check. D already does bounds checks automatically
(unless you compile with -noboundscheck, but the whole *point* of that
flag is to disable bounds checks.)
But that said, I don't know whether the compiler might already be
optimizing out your bounds check anyway. So try it and profile, see what
happens.
Another nice thing to know would be if it is possible to initialize an
array before it is default initialized with T.init where T is the type
of the array's fields. In C++ e.g. there is no default initialization
which is nice if you have to initialize every single field anyway. E.g.
in a Matrix.random() method which creates a matrix with random values.
There it is unnecessary that the (sometimes huge) array is completely
initialized with the type's init value.
You can opt-out of the default initialization with a void initializer:
http://dlang.org/declaration.html#VoidInitializer
Although to be honest I forget how to do that for arrays, and the
functions other people already suggested for creating/initing your array
probably already do that anyway.
Re: Optimize my code =)
Nick Sabalausky:
T opIndex(size_t row, size_t col) const {
immutable size_t i = this.dim.offset(row, col);
if (i = this.dim.size) {
// TODO - have to learn exception handling in D first.
:P
}
return this.data[i];
}
No need for the bounds check. D already does bounds checks
automatically
But the row-column bounds of the matrix are not the same as the
1D bounds of the 1D array. You can have out-of-column-bounds and
still be inside the 1D bounds. So that test should go in the
precondition and it should test row and col separately:
T opIndex(in size_t row, in size_t col) const pure nothrow
in {
assert(row nRows);
assert(col nCols);
} body {
return data[dim.offset(row, col)];
}
Bye,
bearophile
Re: Ranges, constantly frustrating
Marc Schütz: As far as I understand it, it's about adding an index to _foreach_, as is already supported for arrays: foreach(v; [1,2,3,4]) writeln(v); foreach(i, v; [1,2,3,4]) writeln(i, = , v); But for ranges, the second form is not possible: foreach(v; iota(4)) // ok writeln(v); foreach(i, v; iota(4))// Error: cannot infer argument types writeln(i, = , v); I see. In my post I have explained why this is a bad idea (it's not explicit so it gives confusion, and it complicates the language/compiler). A better design is to remove the auto-indexing feature for arrays too, and use .enumerate in all cases, as in Python. Bye, bearophile
Re: how to round durations to most significant unit ? (5 secs, 889 ms, and 811 μs = 5 secs)
thanks!
On Fri, Feb 14, 2014 at 2:14 PM, Jonathan M Davis jmdavisp...@gmx.comwrote:
On Thursday, February 13, 2014 23:37:13 Timothee Cour wrote:
Is there a function to do this?
If not and I/someone writes it, is there interest to add it to
std.datetime?
Duration t = ...;
t.to!string = 5 secs, 889 ms, and 811 μs
t.round.to!string= 5 secs
t=...;
t.to!string = 889 ms, and 811 μs
t.round.to!string= 889 secs
Use case: shorter logs.
There is no function for that, and no one's ever asked for anything like
it,
so I don't know how worthwhile it is to add. However, one possible
implementation would be
auto roundToLargest(Duration d)
{
foreach(units; TypeTuple!(weeks, days, hours, minutes,
seconds, msecs, usecs))
{
immutable value = d.total!units();
if(value != 0)
return dur!units(value);
}
return d;
}
- Jonathan M Davis
Floating point init/nan
I seem to be having some difficulty with the nan and init properties of floating point types. Can anyone explain why the following assertions all fail: assert(float.init == float.nan); assert(float.nan == float.nan); assert(float.init == float.init); Thanks.
Re: Floating point init/nan
On Saturday, 15 February 2014 at 05:18:51 UTC, Adam S wrote: assert(float.init == float.nan); nan never equals nan, this is in the floating point spec used by D, C and others. Use this instead: http://dlang.org/phobos/std_math.html#isNaN
Re: Floating point init/nan
Adam D. Ruppe: nan never equals nan, Additionally, the init NaN doesn't have the same bitpattern as the other. There are many NaNs. Bye, bearophile
Re: Getting and using class hierarhy information in runtime
Also I have the following code but I get some error. I think it's
because of std.algorithm.sort function that uses mixin to inject
predcate. But it doesn't import symbols passed in predicate and
fails. Is there some way of resolving this problem or I need to
inject full code of function inside predicate?
import std.stdio, std.algorithm;
class A: Exception { this(){ super(null); } }
class B: A {}
class C: B {}
class D: A {}
class E: A {}
class F: E {}
size_t countDerivations(TypeInfo_Class typeinfo)
{
size_t result;
while(typeinfo !is null)
{ //writeln(typeinfo.name);
result ++;
typeinfo = typeinfo.base;
}
return result;
}
void main()
{
auto typeinfo = typeid(C);
//while(typeinfo !is typeid(Throwable))
//{ writeln(typeinfo.name);
// typeinfo = typeinfo.base;
//}
auto list = [ typeid(C), typeid(B), typeid(A), typeid(F) ];
//writeln( countDerivations( typeid(C) ) );
auto sortedList = sort!(countDerivations(a)
countDerivations(b))( list );
writeln(sortedList);
}
Re: Getting and using class hierarhy information in runtime
I solved it myself. I forget that I can use function as predicate
import std.stdio, std.algorithm;
class A: Exception { this(){ super(null); } }
class B: A {}
class C: B {}
class D: A {}
class E: A {}
class F: E {}
size_t countDerivations(TypeInfo_Class typeinfo)
{
size_t result;
while(typeinfo !is null)
{ //writeln(typeinfo.name);
result ++;
typeinfo = typeinfo.base;
}
return result;
}
void main()
{
auto typeinfo = typeid(C);
//while(typeinfo !is typeid(Throwable))
//{ writeln(typeinfo.name);
// typeinfo = typeinfo.base;
//}
auto list = [ typeid(C), typeid(B), typeid(A), typeid(F) ];
//writeln( countDerivations( typeid(C) ) );
auto sortedList = sort!( (a, b) { return countDerivations(a)
countDerivations(b); } )( list );
writeln(sortedList);
}
