Re: Python calling D
On Sunday, 2 February 2014 at 15:31:30 UTC, Russel Winder wrote: result is: |> LD_LIBRARY_PATH=. python execute.py Segmentation fault You should call Runtime.initialize() prior to calling any other D functions.
Re: std.parallelism: How to wait all tasks finished?
It seems to me that worker threads will continue as long as the queue isn't empty. So if a task adds another task to the pool, some worker will process the newly enqueued task. No. After taskPool.finish() no way to add new tasks to the queue. taskPool.put will not add new tasks. Then perhaps you need to create a new TaskPool (and make sure that workers add their tasks to the correct task pool), so that you can wait on the first task pool, then wait on the second task pool, etc. auto phase1 = new TaskPool(); //make sure all new tasks are added to phase1 phase1.finish(true); auto phase2 = new TaskPool(); //make sure all new tasks are added to phase2 phase2.finish(true);
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 10:25:19 UTC, Chris wrote: Is there a way I can make the return type in getAttribute generic? null does not work with numbers. MyStruct(T) { T[T] attributes; // public auto getAttribute(T attr) { if (!(attr in attributes)) { return null; // Doesn't work for numbers! } return attributes[attr]; } } void main() { auto myStr = MyStruct!int(0); // Error } Whenever i am faced with this situation i do one (or more then one) of the following things. struct MyStruct(T) { T[T] attributes; //(1) Forward the underlying access method Eg: auto opBinaryRight(string s : "in")(T attrib) { return attrib in attributes; } //(2) make a try method. bool tryAttrib(T attrib, out T outAttrib) { auto p = attrib in attributes; if(p) outAttrib = *p; return p !is null; } //(3) Give user option to set default value. T attribOrDefault(T attrib, T default) { auto p = attrib im attributes; return p is null ? default : attrib; } //(4) Use Nullable!T (I prefer #5 over this one) Nullable!T attribOrNull(T attrib) { Nullable!T result; auto p = attrib ib attributes; if(p) result = *p; return result; } //(5) Use a pointer but not forward in operator. T* attribPtr(T attrib) { return attrib in attributes; } //(6) Throw exception (I only do this in combination with one of the above) T attribEx(T attrib) { return *enforce!AttribNotFoundEx(attrib in attributes); } } My personal preference using #2 and #3 in combination. #2 covers the basic case "Is this thing avalible?" and #3 covers the case "Give it to me if it is avalible or use this default value" I think it gives a clear image of what your code is doing at the callsite. Only using #2 or #3 limits you in this sence. For #1, #4 and #5 i personally stay away from them. They force the caller to either use an if or potentially trigger a null pointer derecerence. (Btw what is the benefit of #4? I have never used it since it seems pointless) I very rarly use attribEx. I don't think code shuld just spew exceptions all over the place. They should be reserved for really bad stuff, like bounds checks. One exception i make to this rule is if i'm dealing with ranges. Since the other methods don't lend themselfs for UFCS-chaing.
Re: Performant method for reading huge text files
On Monday, 3 February 2014 at 23:50:54 UTC, bearophile wrote: Rene Zwanenburg: The problem is speed. I'm using LockingTextReader in std.stdio, but it't not nearly fast enough. On my system it only reads about 3 MB/s with one core spending all it's time in IO calls. Are you reading the text by lines? In Bugzilla there is a byLineFast: https://d.puremagic.com/issues/show_bug.cgi?id=11810 Bye, bearophile Nope, I'm feeding it to csvReader which uses an input range of characters. Come to think of it.. Well this is embarassing, I've been sloppy with my profiling :). It appears the time is actually spent converting strings to doubles, done by csvReader to read a row into my Record struct. No way to speed that up I suppose. Still I find it surprising that parsing doubles is so slow.
Re: Performant method for reading huge text files
Rene Zwanenburg: The problem is speed. I'm using LockingTextReader in std.stdio, but it't not nearly fast enough. On my system it only reads about 3 MB/s with one core spending all it's time in IO calls. Are you reading the text by lines? In Bugzilla there is a byLineFast: https://d.puremagic.com/issues/show_bug.cgi?id=11810 Bye, bearophile
Performant method for reading huge text files
I'm running into a problem I've come across before but never found a satisfactory solution for. There's a pretty large ascii file I need to process, currently about 3GB but size will increase in the future. D's ranges in combination with std.algorithm are simply perfect for what I'm doing, and it's trivial to write nice code which doesn't load the entire file into memory. The problem is speed. I'm using LockingTextReader in std.stdio, but it't not nearly fast enough. On my system it only reads about 3 MB/s with one core spending all it's time in IO calls. Sadly I need to support 32 bit, so memory mapped files aren't an option. Does someone know of a way to increase throughput while still allowing me to use a range API?
Re: 3d vector struct
On Monday, 3 February 2014 at 20:10:59 UTC, Brenton wrote: 4) Is it advisable for the cross method to return by value? In C++, I would declare this method as inline and in a header file. Can I trust D to inline away this inefficiency? Perhaps I should pass in the result as a "ref" or "out" parameter (although I don't require the vector to be initialized here)? Is there a more efficient way to do this? Seeing as previous responses skipped over this point: Yes, return by value. The compiler will optimize that for you by moving (not copying) the result. Return-by-value (and optimizations involved) is one of the stronger things in D that IIRC was there even before e.g. C++11 with its move semantics. Performing a move means that it is absolutely possible for clever compiler to even construct the value in-place, but I'm not sure if any of existing D compilers do that as of yet. Return-by-value being optimized as a move might be one more reason why you would like to use slices instead of variables to store coordinates (since that would mean just moving a pointer and a size_t), but that might have to wait until custom allocators finally arrive.
Re: 3d vector struct
On Monday, 3 February 2014 at 20:10:59 UTC, Brenton wrote: double dot(in Vector3d other) inout { return x * other.x + y * other.y + z * other.z; } Vector3d cross(in Vector3d other) inout { const Vector3d result = { y * other.z - z * other.y, z * other.x - x * other.z, x * other.y - y * other.x }; return result; } } Shouldn't these functions be non-member: double dot(in Vector3d one, in Vector3d other) {} Vector3d cross(in Vector3d one, in Vector3d other) {} No one Vector3d is more special in these functions, so treat them equal.
Re: 3d vector struct
5) I notice that a lot of other people online prefer using fixed arrays not structs for Vectors in D, why? It does make some calculations more straightforward. For example I have code that calculates distance between points as follows: double euclideanDistance( double[] pt1, double[] pt2 ) in { assert( pt1.length == pt2.length ); } body { return sqrt( 0.0.reduce!( (sum,pair) => sum + (pair[0]-pair[1])^^2)(zip(pt1, pt2)) ); } Now a point is not a vector, but they are similar in many respects. That fact that I use an array for my points makes such calculations possible. Furthermore you can always add methods to your struct that let users access the appropriate indices as x, y and z. If you use UFCS (I haven't yet) you could make these appear to user code just as if you had named your variables x, y, and z. Finally, maybe as some point you want to support vectors of varied dimension ... it then becomes easier to port your struct. 6) Any other comments or suggestions? Once you have your design more or less settled you should make it generic (if not for practical reasons just for fun and experience). You likely want your type to support only floating-point values, so you can see here how types can be restricted to FP (see line 50). https://github.com/craig-dillabaugh/phobos/blob/master/std/complex.d Thats my fork of the Phobos libraries, likely a bit out of date, but I was too lazy to look up the prope URL.
Re: 3d vector struct
Brenton: 1) I initialize the vector to a null vector, not nans Why? 2) The dot and cross are "inout" methods, i.e. available for mutable, const, and immutable objects. There is no reason to declare "inout" methods as being "const". But I suggest to add pure/nothrow. 3) The dot and cross methods take an input "in" argument. This allows the compiler to choose between passing the parameter by const value or const reference. This is not true. In means "const scope", so it's always passed by value. Bye, bearophile
3d vector struct
Hi, I'm just getting to know D and so am hoping that someone more experienced with the language could review this 3d vector struct and my comments below. I'm planning on building a little ray tracer in the next week or so :) struct Vector3d { double x = 0, y = 0, z = 0; void normalize() { double scale = 1.0 / (x * x + y * y + z * z); x *= scale; y *= scale; z *= scale; } double dot(in Vector3d other) inout { return x * other.x + y * other.y + z * other.z; } Vector3d cross(in Vector3d other) inout { const Vector3d result = { y * other.z - z * other.y, z * other.x - x * other.z, x * other.y - y * other.x }; return result; } } 1) I initialize the vector to a null vector, not nans 2) The dot and cross are "inout" methods, i.e. available for mutable, const, and immutable objects. There is no reason to declare "inout" methods as being "const". 3) The dot and cross methods take an input "in" argument. This allows the compiler to choose between passing the parameter by const value or const reference. I read somewhere that "in" and "scope" have not yet been implemented yet and that I should use "const ref" instead? 4) Is it advisable for the cross method to return by value? In C++, I would declare this method as inline and in a header file. Can I trust D to inline away this inefficiency? Perhaps I should pass in the result as a "ref" or "out" parameter (although I don't require the vector to be initialized here)? Is there a more efficient way to do this? 5) I notice that a lot of other people online prefer using fixed arrays not structs for Vectors in D, why? 6) Any other comments or suggestions?
Re: InstanceOf Template during runtime in a variant
On Monday, 3 February 2014 at 19:35:47 UTC, Andre wrote: Btw. having std.decimal in the library would be really nice;) Kind regards André There is a proposal in Phobos review queue (http://wiki.dlang.org/Review_Queue) but its author does not seem to be active anymore so it moves nowhere.
Re: InstanceOf Template during runtime in a variant
Am 03.02.2014 20:09, schrieb Ali Çehreli: On 02/03/2014 10:15 AM, Andre wrote: > > I want to check whether the value stored in > variant v is a type of Decimal during runtime. > Is there a nice way? > > Kind regards > André > > import std.variant; > > struct Decimal(int scale, int precision){ > int _precision = precision; > int _scale = scale; This is unrelated to your question but you don't need those members as the two template parameters 'scale' and 'precision' are available. If you needed 'precision' and 'scale' be variables, then you probably don't want to make Decimal a template but I can't be sure from here. :) > > this(string value){/*...*/} > } > > void main(){ > Variant v = Decimal!(10,2)("123.00"); > } Ali Thanks for the answers. Yes you are correct, the 2 members are superflous. Btw. having std.decimal in the library would be really nice;) Kind regards André
Re: InstanceOf Template during runtime in a variant
On 02/03/2014 10:15 AM, Andre wrote: > > I want to check whether the value stored in > variant v is a type of Decimal during runtime. > Is there a nice way? > > Kind regards > André > > import std.variant; > > struct Decimal(int scale, int precision){ > int _precision = precision; > int _scale = scale; This is unrelated to your question but you don't need those members as the two template parameters 'scale' and 'precision' are available. If you needed 'precision' and 'scale' be variables, then you probably don't want to make Decimal a template but I can't be sure from here. :) > > this(string value){/*...*/} > } > > void main(){ > Variant v = Decimal!(10,2)("123.00"); > } Ali
Re: InstanceOf Template during runtime in a variant
(you can check for specific type via `v.type() == typeid(Decimal!(10,2))` though)
Re: InstanceOf Template during runtime in a variant
No. Variant only stores TypeInfo for its current data and templated struct will have a totally different type for each set of template arguments. Their similarity exists only during compile-time.
Re: How to "scope"?
On Monday, 3 February 2014 at 17:58:43 UTC, Namespace wrote: On Monday, 3 February 2014 at 17:50:33 UTC, Martin wrote: On Monday, 3 February 2014 at 17:43:09 UTC, Namespace wrote: On Monday, 3 February 2014 at 17:32:07 UTC, Martin wrote: Oops, I of course meant: static Test createFromString(string str) { return new Test(str); } You _can_ use scoped but it may allocate way to much and it's ugly to use: http://dlang.org/phobos/std_typecons.html#.scoped AFAIK scope'd classes was only deprecated because it _can_ be solved with a library solution and scope is/was not fully implemented. So it was more easy to depecate it and replace it with a library solution, as to implement scope as it stand in the docs. I'm aware of "scoped", that's why I used this specific example. How do you use scoped on a function that returns a new instance of some object? auto obj = scoped(functionThatReturnsNewObject()); That obviously doesn't work. In this case where your object already exist and is on the geap, you may want to use Unique: import std.stdio; import std.typecons; class Foo { ~this() { writeln("Foo::DTor"); } } Foo createNewFoo() { return new Foo(); } void main() { { writeln("Startt"); Unique!(Foo) obj = Unique!(Foo)(createNewFoo()); writeln("End"); } writeln("end of main"); } That's exactly what I was looking for. Brilliant, thanks!
InstanceOf Template during runtime in a variant
Hi, I want to check whether the value stored in variant v is a type of Decimal during runtime. Is there a nice way? Kind regards André import std.variant; struct Decimal(int scale, int precision){ int _precision = precision; int _scale = scale; this(string value){/*...*/} } void main(){ Variant v = Decimal!(10,2)("123.00"); }
Re: How to "scope"?
On Monday, 3 February 2014 at 17:50:33 UTC, Martin wrote: On Monday, 3 February 2014 at 17:43:09 UTC, Namespace wrote: On Monday, 3 February 2014 at 17:32:07 UTC, Martin wrote: Oops, I of course meant: static Test createFromString(string str) { return new Test(str); } You _can_ use scoped but it may allocate way to much and it's ugly to use: http://dlang.org/phobos/std_typecons.html#.scoped AFAIK scope'd classes was only deprecated because it _can_ be solved with a library solution and scope is/was not fully implemented. So it was more easy to depecate it and replace it with a library solution, as to implement scope as it stand in the docs. I'm aware of "scoped", that's why I used this specific example. How do you use scoped on a function that returns a new instance of some object? auto obj = scoped(functionThatReturnsNewObject()); That obviously doesn't work. In this case where your object already exist and is on the geap, you may want to use Unique: import std.stdio; import std.typecons; class Foo { ~this() { writeln("Foo::DTor"); } } Foo createNewFoo() { return new Foo(); } void main() { { writeln("Startt"); Unique!(Foo) obj = Unique!(Foo)(createNewFoo()); writeln("End"); } writeln("end of main"); }
Re: How to "scope"?
On Monday, 3 February 2014 at 17:43:09 UTC, Namespace wrote: On Monday, 3 February 2014 at 17:32:07 UTC, Martin wrote: Oops, I of course meant: static Test createFromString(string str) { return new Test(str); } You _can_ use scoped but it may allocate way to much and it's ugly to use: http://dlang.org/phobos/std_typecons.html#.scoped AFAIK scope'd classes was only deprecated because it _can_ be solved with a library solution and scope is/was not fully implemented. So it was more easy to depecate it and replace it with a library solution, as to implement scope as it stand in the docs. I'm aware of "scoped", that's why I used this specific example. How do you use scoped on a function that returns a new instance of some object? auto obj = scoped(functionThatReturnsNewObject()); That obviously doesn't work.
Re: How to "scope"?
On Monday, 3 February 2014 at 17:32:07 UTC, Martin wrote: Oops, I of course meant: static Test createFromString(string str) { return new Test(str); } You _can_ use scoped but it may allocate way to much and it's ugly to use: http://dlang.org/phobos/std_typecons.html#.scoped AFAIK scope'd classes was only deprecated because it _can_ be solved with a library solution and scope is/was not fully implemented. So it was more easy to depecate it and replace it with a library solution, as to implement scope as it stand in the docs.
Re: How to "scope"?
Oops, I of course meant: static Test createFromString(string str) { return new Test(str); }
How to "scope"?
Seeing as the scope keyword is (being?) deprecated, how would you handle something like this: class Test { private: string str; this(string str) { this.str = str; } public: static Test createFromString(string str) { return new Test(); } } void main() { // at the end of the scope, test is destroyed and memory is freed scope Test test = Test.createFromString("test"); } The only solution I can think of is using scope(exit) something like this: scope(exit) { destroy(test); GC.free(cast(void*)test); } which seems clumsy. Any other, better solutions?
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 14:10:35 UTC, bearophile wrote: Dicebot: If you need cheap and simple way to figure out that attribute was missing, change API to return value by out parameter and turn normal return value into boolean success flag. It's probably better to start using Nullable. Bye, bearophile Depends. If you control system as a whole, yes, it is better to build it on top of Nullable. If interfacing with external code is intended, it will just add more boilerplate. Also I hate name "Nullable" and use it under name "Optional" :)
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 14:17:11 UTC, Chris wrote: Probably. I tried using Nullable, but it caused some problems when the attribute wasn't defined: core.exception.AssertError@/usr/include/dmd/phobos/std/typecons.d(1233): Called `get' on null Nullable!int. This is intended. The very point of Nullable is to force you to handle `null` state before accessing actual payload.
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 14:21:29 UTC, Dicebot wrote: On Monday, 3 February 2014 at 14:17:11 UTC, Chris wrote: Probably. I tried using Nullable, but it caused some problems when the attribute wasn't defined: core.exception.AssertError@/usr/include/dmd/phobos/std/typecons.d(1233): Called `get' on null Nullable!int. This is intended. The very point of Nullable is to force you to handle `null` state before accessing actual payload. Aha, I see. So in my scenario it is useless.
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 14:21:29 UTC, Dicebot wrote: This is intended. The very point of Nullable is to force you to handle `null` state before accessing actual payload. P.S. for this very reason in my own implementation of Optional I provide only delegate access method: value.get( ()=> { /* handle 'empty' case */ }, value => { /* 'value' is legit data */ } );
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 14:10:35 UTC, bearophile wrote: Dicebot: If you need cheap and simple way to figure out that attribute was missing, change API to return value by out parameter and turn normal return value into boolean success flag. It's probably better to start using Nullable. Bye, bearophile Probably. I tried using Nullable, but it caused some problems when the attribute wasn't defined: core.exception.AssertError@/usr/include/dmd/phobos/std/typecons.d(1233): Called `get' on null Nullable!int.
Re: Templates: generic "return null;"
Dicebot: If you need cheap and simple way to figure out that attribute was missing, change API to return value by out parameter and turn normal return value into boolean success flag. It's probably better to start using Nullable. Bye, bearophile
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 12:25:16 UTC, Dicebot wrote: You have forgot to mention what behavior you are actually trying to achieve ;) Common not-so-meaningful value is simply T.init , but there can be no such thing as generic sentinel. If you need cheap and simple way to figure out that attribute was missing, change API to return value by out parameter and turn normal return value into boolean success flag. Thanks. T.init actually does the trick. The behavior: auto name = myStruct.getAttribute("name"); if (name == "bla") { // do something } else { // do something else } or (theoretically): auto second = myStruct.getAttribute(1.0); if (second > 1.5) { // do something } else { // do something else. } I haven't got a use case for the second example, but it might be handy for data analysis and I wanted to test how far you can go with templates. The reasoning behind it is that string name; try { name = myStruct.getAttribute("name"); } // ... is a bit awkward an OTT. I'd prefer to introduce hasAttribute("name") instead, if I want to be sure it exists. if (myStruct.hasAttribute("name)) name = myStruct.getAttribute("name");
Re: Templates: generic "return null;"
You have forgot to mention what behavior you are actually trying to achieve ;) Common not-so-meaningful value is simply T.init , but there can be no such thing as generic sentinel. If you need cheap and simple way to figure out that attribute was missing, change API to return value by out parameter and turn normal return value into boolean success flag.
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 10:55:23 UTC, Chris wrote: I'm reluctant to (over)use throw, because I think that throw should be the last resort when you cannot easily predict all the things that can go wrong. Simple requests should give simple answers. If the key doesn't exist it returns nothing. D has no concept of "nothing" in the language. What if your dictionary contains objects, and for some key K it *does* contain a null reference? How would you distinguish absence of value from some "invalid" value? Nullable can help with that, of course. But then again, what if your dictionary does contain other Nullables? :) Generally, inability to return value is an exceptional situation. The problem above only exists because of generic types, else it would be either 'null' or for numbers -1 or something. -1 is a number, just like 0 or 10 or 13142313. Same as above, at the call site you won't be able to tell if -1 indicates that there's no such key in the attributes dictionary or just that attribute happens to have a value of -1.
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 10:32:58 UTC, Stanislav Blinov wrote: On Monday, 3 February 2014 at 10:25:19 UTC, Chris wrote: T[T] attributes; // public auto getAttribute(T attr) { if (!(attr in attributes)) { return null; // Doesn't work for numbers! } return attributes[attr]; } } One way would be to use std.typecons.Nullable(T) as a return type. Thanks. I'll try this one. Another would be to retink your design :) Note that for numbers, there's no distinct "does not exist" value (well, ok, we have NaN for floating point). Neither there is for structs. Generally such methods as your getAttribute either throw, or get a second optional parameter as hint on what to return when the element is not found in the dictionoary. I'm reluctant to (over)use throw, because I think that throw should be the last resort when you cannot easily predict all the things that can go wrong. Simple requests should give simple answers. If the key doesn't exist it returns nothing. The problem above only exists because of generic types, else it would be either 'null' or for numbers -1 or something. But if you have a good point I've overlooked, you can convince me of throw. No ideology :)
Re: Templates: generic "return null;"
On Monday, 3 February 2014 at 10:25:19 UTC, Chris wrote: T[T] attributes; // public auto getAttribute(T attr) { if (!(attr in attributes)) { return null; // Doesn't work for numbers! } return attributes[attr]; } } One way would be to use std.typecons.Nullable(T) as a return type. Another would be to retink your design :) Note that for numbers, there's no distinct "does not exist" value (well, ok, we have NaN for floating point). Neither there is for structs. Generally such methods as your getAttribute either throw, or get a second optional parameter as hint on what to return when the element is not found in the dictionoary.
Templates: generic "return null;"
Is there a way I can make the return type in getAttribute generic? null does not work with numbers. MyStruct(T) { T[T] attributes; // public auto getAttribute(T attr) { if (!(attr in attributes)) { return null; // Doesn't work for numbers! } return attributes[attr]; } } void main() { auto myStr = MyStruct!int(0); // Error }