Re: D idom for removing array elements
On Monday, 30 January 2017 at 12:31:33 UTC, albert-j wrote: OK, got it. Can you do removal without reallocation with std.container.array? Array!int arr; foreach (i; 0..10) arr ~= i; Sorry, sent too early. arr = arr[].remove!(x=> x > 5); //Doesn't work withouth calling .Array!int
Re: D idom for removing array elements
On Monday, 30 January 2017 at 10:45:03 UTC, cym13 wrote:
Meh.
Forget that, bad memory. remove isn't working in-place. However
slapping ".array" is still asking explicitely for reallocation,
so just forget it. Here is a code that works:
import std.conv;
import std.stdio;
import std.format;
import std.algorithm;
void main(string[] args) {
int [] arr = [8, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
auto before = arr.ptr;
arr = arr.remove!(x => x>5);
auto after = arr.ptr;
assert(arr == [0, 1, 2, 3, 4, 5], arr.to!string);
assert(before == after, "%s %s".format(before, after));
}
OK, got it. Can you do removal without reallocation with
std.container.array?
Array!int arr;
foreach (i; 0..10) arr ~= i;
Re: D idom for removing array elements
On Monday, 30 January 2017 at 00:17:51 UTC, ag0aep6g wrote:
Removing works by overwriting the array with only the wanted
values and discarding the rest.
But then why do I get this:
import std.stdio, std.algorithm, std.array;
int[] arr;
foreach (i; 0..10) arr ~= i; // [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
writeln("&arr[0]:", &arr[0]); // prints "7F56FAE93000"
arr = arr.remove!(x => x > 5).array;
writeln("&arr[0]:", &arr[0]); // prints "7F56FAE92020"
writeln("arr:",arr); // prints: "[0, 1, 2, 3, 4, 5]"
It looks like arr.remove allocates new memory and copies the data
there.
Re: D idom for removing array elements
On Monday, 30 January 2017 at 00:17:51 UTC, ag0aep6g wrote: [...] Great explanation, thank you!
Re: D idom for removing array elements
On Sunday, 29 January 2017 at 23:48:40 UTC, Jordan Wilson wrote: You need to do something like this: auto arrMap = arr.filter!(x => x > 5).map!(x => x^^2).array; It's because arrMap is lazy evaluated. So does it mean that I cannot assign FilterResult and MapResult to a variable and safely use it later, because underlying array may change meanwhile? Since I am dealing with large arrays, I thought I'd save some memory by not calling .array on MapResult.
Re: D idom for removing array elements
On Saturday, 28 January 2017 at 11:54:58 UTC, cym13 wrote:
I am trying to wrap my head around lazy evaluation during
filtering/mapping, but there's something I don't understand.
I want to create an array, square some elements, remove some
elements from original array and add the squared ones to the
original array:
import std.stdio, std.algorithm, std.array;
int[] arr;
foreach (i; 0..10)
arr ~= i;
writeln("Original array: ",arr);
// [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] -- OK
auto arrMap = arr.filter!(x => x > 5).map!(x => x^^2);
writeln("arrMap: ", arrMap);
// [36, 49, 64, 81] -- OK
int[] toRemove = [1, 2, 9];
arr = arr.remove!(x => toRemove.canFind(x)).array;
writeln("Original array after removal: ", arr);
// [0, 3, 4, 5, 6, 7, 8] -- OK
arr ~= arrMap.array;
writeln("Original array after removal and concatenation: ",
arr);
// [0, 3, 4, 5, 6, 7, 8, 64, 49, 64, 81] -- what?
The last result is not what I wanted. I would expect [0, 3, 4, 5,
6, 7, 8] and [36, 49, 64, 81] concatenated into [0, 3, 4, 5, 6,
7, 8, 36, 49, 64, 81], but something else is happening here.
It looks like arr = arr.remove! is messing things up, but
why?
Re: D idom for removing array elements
On Friday, 27 January 2017 at 08:15:56 UTC, Dukc wrote:
void main()
{ import std.stdio, std.algorithm, std.range, std.array,
std.datetime;
int[] a = [1, 2, 3, 4, 5, 6, 7, 4].cycle.take(2000).array;
int[] b = [3, 4, 6].cycle.take(2000).array;
void originalMethod()
{ auto c = a.remove!(x => b.canFind(x));
assert(c[0 .. 4] == [1, 2, 5, 7]);
}
void WilsonMethod()
{ auto c = a.filter!(x => !b.canFind(x)).array;
assert(c[0 .. 4] == [1, 2, 5, 7]);
}
void myMethod()
{ auto sortedB = sort(b.dup);
auto c = a
. filter!(i => !sortedB.contains(i))
. array
;
assert(c[0 .. 4] == [1, 2, 5, 7]);
}
auto r = benchmark!(originalMethod, WilsonMethod,
myMethod)(1);
foreach(result; r) result.writeln;
}
How to make it work with std.container.array?
Re: D idom for removing array elements
On Friday, 27 January 2017 at 08:15:56 UTC, Dukc wrote: TickDuration(28085) TickDuration(42868) TickDuration(1509) Thank you, this is very helpful. I am also wondering why the standard library doesn't have convenience functions for this, e.g. like Java's removeAll? Now there's more typing than necessary for a relatively common task.
Re: D idom for removing array elements
On Thursday, 26 January 2017 at 13:21:38 UTC, Dukc wrote: import std.stdio, std.algorithm, std.range, std.array; int[] a = [1, 2, 3, 4, 5, 6, 7, 4]; int[] b = [3, 4, 6]; auto sortedB = sort(b.dup); auto c = a . filter!(i => !sortedB.contains(i)) . array ; assert(c == [1, 2, 5, 7]); If arrays get large, this should be more efficient since it performs O(n * n.log) instead of O(n * n). It does look much faster than my solution. Will it also work correctly and fast for arrays of custom objects? How should opCmp() be defined if objects don't have a meaningful ordering? The order of elements in the original array does not matter.
D idom for removing array elements
What is the D idiom for removing array elements that are present in another array? Is this the right/fastest way? int[] a = [1, 2, 3, 4, 5, 6, 7, 4]; int[] b = [3, 4, 6]; auto c = a.remove!(x => b.canFind(x)); assert(c == [1, 2, 5, 7]);
Re: General performance tips
Without seeing the source there is nothing we can do. Usually performant d-code looks quite diffrent from java code. For example to avoid the gc :) Well it is actually ODE solver from Numerical recipes (originally in C++) that I am trying to do in D. Code translation seems very straightforward. Maybe there's someone around who has done that already? There's not much object creation going on there, mostly loops over arrays, so I assume GC is not an issue.
General performance tips
I have translated some simulation code from Java into D (a few classes, mostly manipulation of double arrays in small methods). D version runs 10-30% slower than Java (ldc2, dub release build). Profiling did not show any obvious bottlenecks. I am wondering whether I missed something, or such performance difference can be expected. I read on this forum that marking methods final and nothrow can help, but it didn't do much in my case. Is there anything else I should be looking for? In general, something like a wiki page with performance tips would be very useful for D newcomers like me! :)
Re: Profiling calls to small functions
I'm not sure if it's what happening in this case but, in code as simple as this, function calls can sometimes be the bottleneck. You should see how compiling with/without -O affects performance, and adding `pragma(inline)` to funcB. I guess my question is whether it is possible to have meaningful profiling results for this case, given a large cost of calling funcB? In release builds funcA and funcB are inlined, so profiler cannot report on them individually (is it correct, or am I misusing the profiler?). Profiling without inlining will show a large cost of calling funcB, but this cost will not be there in a release build, so the profiling results are irrelevant.
Re: Why does multidimensional arrays not allocate properly?
In anycase, what is the correct notation for indexing? x = new int[][](width, height) and x[height][width] or x[width][height]? It's x[width][height], but because indexing is 0-based, largest valid indexes are x[width-1][height-1].
Re: Profiling calls to small functions
I'm not sure if it's what happening in this case but, in code as simple as this, function calls can sometimes be the bottleneck. You should see how compiling with/without -O affects performance, and adding `pragma(inline)` to funcB. When compiled with -inline, the profiler does not report the performance of funcA and funcB individually, and this is what I want to measure.
Profiling calls to small functions
Let's say I want to create an array of random numbers and do some
operations on them:
void main() {
import std.random;
//Generate array of random numbers
int arrSize = 1;
double[] arr = new double[](arrSize);
foreach (i; 0..arrSize)
arr[i] = uniform01();
//Call funcA on array elements
foreach (i; 1..arr.length-1)
funcA(arr,i);
}
void funcA(double[] arr, size_t i) {
arr[i+1] = arr[i-1]+arr[i];
funcB(arr,i);
}
void funcB(double[] arr, size_t i) {
arr[i-1]= arr[i] + arr[i+1];
arr[i] = arr[i-1] + arr[i+1];
arr[i+1]= arr[i-1] + arr[i];
}
Now I dmd -profile it and look at the performance of funcA with
d-profile-viewer. Inside funcA, only 20% of time is spend in
funcB, but the rest 80% is self-time of funcA. How is it
possible, when funcB has three times the calculations of funcA?
It appears that the call to funcB itself is very expensive.
Re: Referring to array element by descriptive name
Thank you for all your answers. I was concerned because I'm dealing with a small function that is called many times and where the bulk of the calculations in the simulation takes place. So even 5% performance difference would be significant for me. But it is good to know that compilers are smart enough to optimize this.
Referring to array element by descriptive name
Is it possible to refer to an array element by a descriptive
name, just for code clarity, without performance overhead? E.g.
void aFunction(double[] arr) {
double importantElement = arr[3];
... use importantElement ...
}
But the above, I suppose, introduces an extra copy operation?
Re: Using tango with dub
Have you seen https://github.com/economicmodeling/containers it has a HashSet http://economicmodeling.github.io/containers/containers/hashset.HashSet.html Just curious, how is it different from Tango's implementation?
Re: Using tango with dub
Try an older version. Before resorting to that, I am also trying to "dub build --compiler=gdc". Getting different types of errors: ../../../.dub/packages/tango-1.0.3_2.068/tango/tango/math/IEEE.d:614:17: error: instead of C-style syntax, use D-style syntax 'real[3][] vals' [-Werror] static real vals[][3] = // x,frexp,exp ... ^ cc1d: all warnings being treated as errors I tried to suppress the errors by adding buildRequirements:["allowWarnings", "silenceWarnings"] to dub.json, but no luck. Is it possible to get around them?
Re: Using tango with dub
I thought Tango was obsolete a long time ago. Is there a specific reason you need to use Tango and can't use Phobos? I need a Set implementation and from what I understand there isn't one in Phobos right now?
Re: Using tango with dub
Since I just do "dub build", I assume it invokes dmd? I have v2.072.0.
Using tango with dub
I am trying to use Tango in a dub project because I need a HashSet. I added Tango as a dependency to the dub.json, but now dub gives me a bunch of depreciation warnings and a few errors, like ../../../.dub/packages/tango-1.0.3_2.068/tango/tango/util/log/Log.d(349,51): Error: undefined identifier '__va_argsave', did you mean struct '__va_argsave_t'? What am I missing?
