On 02/03/2012 01:21 AM, bearophile wrote:
Through Reddit I've found this good and long slides pack, it's about using Java
data structures to increase memory efficiency of programs:
http://domino.research.ibm.com/comm/research_people.nsf/pages/sevitsky.pubs.html/$FILE/oopsla08%20memory-efficient%20java%20slides.pdf
Despite the D situation is different (there are structs as in C#), it will be
good to have weak and soft references in Phobos, and to have better memory
analysis tools outside Phobos.
The slides have reminded me my desire of a column-oriented "struct array" in
Phobos (some time ago someone has written a minimal version for D1).
The usage is simple:
import std.stdio, std.conv;
struct Foo { // an example struct
int x;
float y;
string s;
this(int xx, float yy) {
x = xx;
y = yy;
s = text(x);
}
float sum() {
return x + y;
}
}
void main() {
auto a1 = new Foo[1000]; // normal not parallel array
foreach (ref Foo f; a1)
writeln(f.s, " ", f.sum());
// default usage example of ParallelArray
// 3 Foo fields stored as 3 separated arrays inside a2
ParallelArray!Foo a2; // valid
static assert(a2[0].sizeof == size_t.sizeof * 4); // 3 pointers + 1 length
a2.length = 1000;
foreach (ref Foo f; a2) // A f Foo is built on the fly
writeln(f, " ", f.sum());
a2[10] = Foo(1, 2, "1");
foreach (x; a2.x_array) // x_array is a property slice
writeln(x);
foreach (y; a2.y_array)
writeln(y);
foreach (s; a2.s_array)
writeln(s);
// specialized usage example of ParallelArray
// x,y fields stored as an array, s field as another array
ParallelArray!(Foo, "x y # s") a3; // valid
static assert(a3[0].sizeof == size_t.sizeof * 3); // 2 pointers + 1 length
a3.length = 1000;
foreach (ref Foo f; a3) // A f Foo is built on the fly
writeln(f, " ", f.sum());
a3[10] = Foo(1, 2, "1");
foreach (xy; a3.x_y_array)
writeln(xy.x, " ", xy.y);
foreach (s; a3.s_array)
writeln(s);
// float z0 = a3.x_y_array[10].sum(); // invalid code
ParallelArray!(Foo, "x # y # s") a4; // valid code
// ParallelArray!(Foo, "x y # s x") a5; // invalid, dupe field x
// ParallelArray!(Foo, "x # y") a6; // invalid, s field missing
// so if you give a string with the field names, you need to
// list them all, and only once each. Other designs are possible
// but this is the simplest to use and implement.
float z1 = a3[10].sum(); // a3[10] returns a Foo
// a3(10) doesn't create a Foo, it just fetches what
// .sum() needs, so it's faster if you have to call .sum()
// on many records.
// so the calls to sum() are implemented at compile-time
float z2 = a3(10).sum();
How would you implement such a thing? Introspection is not (yet) that
powerful.
// To keep design simple. ParallelArray can't create 2D arrays
}
Do you like?
I have several usages of such struct in my code.
Bye,
bearophile
I like it and I think it would be a valuable addition to Phobos, as well
as a nice showcase for Ds templates and string mixins. You didn't
specify what the type of eg. a3.x_y_array is, I think it should be
Tuple!(int, "x", float, "y")[]. Furthermore, I'd rather have it named
a3.x_y. There should also be a possibility to manually name the parts,
as well as a way to specify 'anything that is left':
ParallelArray!(Foo, "pos: x y # rest: ...") pa;
static assert(is(typeof(pa.pos) == Tuple!(int, "x", float, "y")[]));
