Re: More automated interfacing of D with C codebases
On 2012-10-21 21:45, timotheecour wrote: Manually porting of C/C++ libraries shouldn't be the way to go (a major problem being when said library gets updated, human errors, porting to different architectures etc). SWIG requires some kind of interface files, which I assume one must have to manually write. Then what's the difference? I've had good experience using SWIG to automate the job for me, and used it successfully to port the following libraries to D: * ncurses * opencv * sfml to name a few. I believe this is the way forward, as opposed to maintaining (necessary stale) ports such as Deimos. Swig isn't perfect however, and sometimes will bail out on constructs such as: C++ classes with multiple inheritance, or a few obscure C or C++ constructs. Even in that case, I find it much easier to tweak the swig interface file to get desired output rather than undertake the huge manual conversion task. For sfml, the interface file is just about 40 lines for example. Improving such kind of automated tools is where we should be concentrating efforts, not on maintaining deimos. One such improvement I made was to allow conversion of C++ templates to D templates (works in large majority of cases), which makes the library much easier to use than using name mangling schemes. More improvements would be welcome (eg support for static library generation as opposed to dynamic load libraries), reducing the need for tweaking swig interface files etc. I don't think SWIG is the right approach because: * You have to create these interface files * It seem to not handle all code * No support for Objective-C BTW, how does SWIG handle macros and documentation? -- /Jacob Carlborg
Re: Slices and array appending
On Monday, 22 October 2012 at 04:59:41 UTC, Jonathan M Davis wrote: Blocks of memory are allocated and freed as a whole. When an array is forced to be reallocated, then a new block of memory is allocated for it, and any existing slices continue to refer to the original block of memory. The original block will not be freed until there are no more slices which refer to it. You really should read this article if you want to know how arrays work in D: http://dlang.org/d-array-article.html - Jonathan M Davis Ah thanks, I didn't realize the _entire_ original array would hang around. That makes my current approach really dumb. Cheers
Re: Slices and array appending
On Monday, October 22, 2012 01:35:29 cal wrote: > Just want to make sure I understand this properly: > > I have a large dynamic array (whose elements are immutable) which > only ever grows. I also have a whole lot of small slices into > this array, the slices never change, and they don't span the > entire contents of the array (they are just pieces). > > Now if I append to the large array, and the runtime needs to > re-allocate to accommodate the change in size, none of the pieces > of the original array which are referred to by the slices can be > freed, right? So I end up basically with two copies of the > original large array, however the first version will now have > little pieces missing from it (wherever the slices don't refer). > > Is that correct? Blocks of memory are allocated and freed as a whole. When an array is forced to be reallocated, then a new block of memory is allocated for it, and any existing slices continue to refer to the original block of memory. The original block will not be freed until there are no more slices which refer to it. You really should read this article if you want to know how arrays work in D: http://dlang.org/d-array-article.html - Jonathan M Davis
Re: override toString of Exception
On 10/21/2012 01:51 PM, David wrote: Am 21.10.2012 22:46, schrieb David:> > This behavior makes sense to me because printing the backtrace should > > concern the application, not the exception itself. If the application > > does not want the backtrace printed, it can handle all of the > exceptions. > > > > Ali > > I just want to add color to my exceptions and the easiest way is to > override toString, that is not possible since, the printing-code in > druntime doesn't call exception.toString, but reimplements it's default > behaviour. > > I "fixed" it: > https://github.com/D-Programming-Language/druntime/pull/331 Well, it can be used to way more than only coloring the exception, you can also provide additional information etc. and I think it makes more sense, why is there a toString implemented if it isn't used. I don't see that: toString() does get used when the exception object appears in a string context. That part works. What you are asking is why not the runtime calls only toString() of the object. The backtrace information is helpful as well, so a special exception type should not take away that information from the whole application. What does the little exception know about the application that it decides that the backtrace not be produced? I think the exception type is the wrong place to make that decision. If it is important for the application that the backtrace is not printed, it can simply catch all exceptions in main. Backtrace is printed only for unhandled exceptions. Ali
Re: Slices and array appending
On Sunday, 21 October 2012 at 23:35:30 UTC, cal wrote: Just want to make sure I understand this properly: I have a large dynamic array (whose elements are immutable) which only ever grows. I also have a whole lot of small slices into this array, the slices never change, and they don't span the entire contents of the array (they are just pieces). Now if I append to the large array, and the runtime needs to re-allocate to accommodate the change in size, none of the pieces of the original array which are referred to by the slices can be freed, right? So I end up basically with two copies of the original large array, however the first version will now have little pieces missing from it (wherever the slices don't refer). Is that correct? Sounds right.. But it could be worse. The slices that are used by the original array are ignored in the new array, so unless you have a main array reference to access the whole thing, then you wasted space can only get worse. Now on the other hand, if you were to append to a slice, then that slice would duplicate and then add onto it making a new slice with minimal loss compared to the much larger reference. In this kind of case I can only think of a few ways I'd want to work around it so to help minimize loss. 1) Lowlevel check/test to see if it can be appended, if not start with new array so we don't reallocate and duplicate. (malloc/realloc may be needed) 2) Reserve memory enough for N elements where it's your estimated Max (or a little over) 3) Save beginning/ending locations as ints instead of a slice (although the main/original array would need to be accessible); You then can create temporary slices for functions as needed, long as they don't leave the current scope. 4) Work with small slices only, meaning you get 100 or so, then a new array with another 100 or so.
Reordered class fields?
D classes are free to reorder their fields, so maybe Bar1 should
reorder its fields as Bar3, to save 4 bytes for each instance on
32 bit systems:
class Bar1 {
void Hello() {}
float f;
double d;
}
class Bar2 {
void Hello() {}
align(4) float f;
align(4) double d;
}
class Bar3 {
void Hello() {}
double d;
float f;
}
void main() {
pragma(msg, __traits(classInstanceSize, Bar1)); // 24
pragma(msg, __traits(classInstanceSize, Bar2)); // 20
pragma(msg, __traits(classInstanceSize, Bar3)); // 20
}
This benchmark shows that if you allocate the class instances on
the heap one at a time the total amount of memory used is the
same for the various Bar (maybe because of the GC), so that
optimization is useful for emplace() only and similar in-place
allocations:
class Bar1 {
void Hello() {}
float f;
double d;
}
class Bar2 {
void Hello() {}
align(4) float f;
align(4) double d;
}
class Bar3 {
void Hello() {}
double d;
float f;
}
int main() {
pragma(msg, __traits(classInstanceSize, Bar1)); // 24
pragma(msg, __traits(classInstanceSize, Bar2)); // 20
pragma(msg, __traits(classInstanceSize, Bar3)); // 20
//--
//auto arr = new Bar1[1_000_000]; // 38.2 MB
//auto arr = new Bar2[1_000_000]; // 38.2 MB
auto arr = new Bar3[1_000_000]; // 38.2 MB
foreach (ref a; arr)
a = new typeof(arr[0])();
int count;
foreach (i; 0 .. 500_000_000) count++;
return count;
}
So is such class field reordering worth an enhancement request in
Bugzilla?
Bye,
bearophile
Slices and array appending
Just want to make sure I understand this properly: I have a large dynamic array (whose elements are immutable) which only ever grows. I also have a whole lot of small slices into this array, the slices never change, and they don't span the entire contents of the array (they are just pieces). Now if I append to the large array, and the runtime needs to re-allocate to accommodate the change in size, none of the pieces of the original array which are referred to by the slices can be freed, right? So I end up basically with two copies of the original large array, however the first version will now have little pieces missing from it (wherever the slices don't refer). Is that correct?
Re: Overload resolution (value vs reference)
On Sun, 2012-10-21 at 19:01 +0200, m0rph wrote:
> How does compiler selects the proper function among overloaded
> functions which differ only in the way the argument is passed (by
> reference or by value)? Is there a way, to control this behavior?
>
> Result of execution of the test code:
> passed by value: 8
> passed by value: 3
> passed by value: Foo(10)
>
>
> Test code:
>
> import std.stdio;
>
> struct Foo {
> int value;
> }
>
> void f(T)(T foo)
> {
> writeln("passed by value: ", foo);
> }
>
> void f(T)(const ref T foo)
> {
> writeln("passed by reference: ", foo);
> }
>
> void main()
> {
> // 8 is a r-vlaue, so it's passed by value
> f(8);
>
> // i is a l-value, it's passed by value and it's ok
> int i = 3;
> f(i);
>
> // foo is a l-value, it's passed by value again, but if
> structure will be big enough it'll be ineffective
>
> auto foo = Foo();
> foo.value = 10;
> f(foo);
> }
>
http://dlang.org/function.html#function-overloading
The big thing to remember here is that constness matters more than
refness when overloads are chosen, so if you want refness to matter when
choosing an overload, then the constness of the overloads must match,
and if there's ever a question between const and non-const, it's the
constness of the argument being passed in which wins (e.g. if you have
ref T and const ref T, then which one gets called depends on whether the
argument is const or not).
As your code stands, if it were
void f(T)(T foo)
{
f(foo);
}
void f(T)(const ref T foo)
{
//...
}
you would get an infinite loop when passing an rvalue to f, whereas
void f(T)(const T foo)
{
f(foo);
}
void f(T)(const ref T foo)
{
//...
}
would not.
- Jonathan M Davis
Re: class opBinary overloading. value + null and null + value
On Sunday, 21 October 2012 at 20:54:15 UTC, ref2401 wrote:
On Sunday, 21 October 2012 at 20:42:08 UTC, cal wrote:
On Sunday, 21 October 2012 at 20:34:51 UTC, ref2401 wrote:
what should i do to make my binary operation commutative?
in the following case
auto v5 = v1 + v2;
there will be compile time error:
overloads pure @safe MyClass(MyClass rhs) and pure @safe
MyClass(MyClass lhs) both match argument list for opBinary
Oh right, sorry. Well if all you want to do is catch the null on
either side of the '+', you could use this overload of
opBinaryRight:
MyClass opBinaryRight(string op: "+", T)(T rhs) if (!is(T ==
MyClass))
{
return null;
}
I'm not sure if this is what you are trying to do though...
Re: override toString of Exception
Am 21.10.2012 22:46, schrieb David:> > This behavior makes sense to me because printing the backtrace should > > concern the application, not the exception itself. If the application > > does not want the backtrace printed, it can handle all of the > exceptions. > > > > Ali > > I just want to add color to my exceptions and the easiest way is to > override toString, that is not possible since, the printing-code in > druntime doesn't call exception.toString, but reimplements it's default > behaviour. > > I "fixed" it: > https://github.com/D-Programming-Language/druntime/pull/331 Well, it can be used to way more than only coloring the exception, you can also provide additional information etc. and I think it makes more sense, why is there a toString implemented if it isn't used.
Re: class opBinary overloading. value + null and null + value
On Sunday, 21 October 2012 at 20:42:08 UTC, cal wrote:
On Sunday, 21 October 2012 at 20:34:51 UTC, ref2401 wrote:
what should i do to make my binary operation commutative?
You can overload opBinaryRight:
MyClass opBinaryRight(string op: "+")(MyClass rhs)
{
if (rhs is null)
{
return null;
}
MyClass result = new MyClass(value + rhs.value);
return result;
}
http://dlang.org/operatoroverloading.html
in the following case
auto v5 = v1 + v2;
there will be compile time error:
overloads pure @safe MyClass(MyClass rhs) and pure @safe
MyClass(MyClass lhs) both match argument list for opBinary
Re: override toString of Exception
> This behavior makes sense to me because printing the backtrace should > concern the application, not the exception itself. If the application > does not want the backtrace printed, it can handle all of the exceptions. > > Ali I just want to add color to my exceptions and the easiest way is to override toString, that is not possible since, the printing-code in druntime doesn't call exception.toString, but reimplements it's default behaviour. I "fixed" it: https://github.com/D-Programming-Language/druntime/pull/331
Re: class opBinary overloading. value + null and null + value
On Sunday, 21 October 2012 at 20:34:51 UTC, ref2401 wrote:
what should i do to make my binary operation commutative?
You can overload opBinaryRight:
MyClass opBinaryRight(string op: "+")(MyClass rhs)
{
if (rhs is null)
{
return null;
}
MyClass result = new MyClass(value + rhs.value);
return result;
}
http://dlang.org/operatoroverloading.html
class opBinary overloading. value + null and null + value
public class MyClass
{
private int value;
this(int val)
{
value = val;
}
MyClass opBinary(string op: "+")(MyClass rhs)
{
if (rhs is null)
{
return null;
}
MyClass result = new MyClass(value + rhs.value);
return result;
}
}
void main()
{
MyClass v1 = new MyClass(4);
MyClass v2 = new MyClass(5);
auto v3 = v1 + null; // v3 is null
auto v4 = null + v2; // v4 equals v2
}
in this example v3 is null. i thought v4 would be null too, but i
was wrong.
what should i do to make my binary operation commutative?
Re: More automated interfacing of D with C codebases
On Sunday, 21 October 2012 at 19:45:53 UTC, timotheecour wrote: I've had good experience using SWIG to automate the job for me, and used it successfully to port the following libraries to D: Yes. SWIG should be a lot faster than my tool. Also I don't have any units to handle C++ code. Completely agree. It has to be automated. Swig isn't perfect however, and sometimes will bail out on constructs such as: C++ classes with multiple inheritance, or a few obscure C or C++ constructs. Even in that case, I find it much easier to tweak If I recall, I ran into some obscure C constructs in the Tcl/Tk/X headers that I have never seen before. I believe I had to various aliases to my "interface file" to handle those. Also use of D reserved words had to be handled: substitute new new_ substitute class class_ substitute 'function;' 'function_;' And some things just plain won't work in D: # not valid D code. substitute '_XPrivate .private9, .private10;' \ '_XPrivate * private9, private10;'
Re: More automated interfacing of D with C codebases
Manually porting of C/C++ libraries shouldn't be the way to go (a major problem being when said library gets updated, human errors, porting to different architectures etc). I've had good experience using SWIG to automate the job for me, and used it successfully to port the following libraries to D: * ncurses * opencv * sfml to name a few. I believe this is the way forward, as opposed to maintaining (necessary stale) ports such as Deimos. Swig isn't perfect however, and sometimes will bail out on constructs such as: C++ classes with multiple inheritance, or a few obscure C or C++ constructs. Even in that case, I find it much easier to tweak the swig interface file to get desired output rather than undertake the huge manual conversion task. For sfml, the interface file is just about 40 lines for example. Improving such kind of automated tools is where we should be concentrating efforts, not on maintaining deimos. One such improvement I made was to allow conversion of C++ templates to D templates (works in large majority of cases), which makes the library much easier to use than using name mangling schemes. More improvements would be welcome (eg support for static library generation as opposed to dynamic load libraries), reducing the need for tweaking swig interface files etc.
Re: More automated interfacing of D with C codebases
I don't think this is exactly what you want either, but it's certainly a lot easier than a manual process: http://gentoo.com/di/mkconfig.html (On Freecode) https://freecode.com/projects/mkconfig I just now released a new version. Freecode will be updated within 24 hours. Check the examples/dtcltk/ directory. There's also a simpler example within the D/ subdirectory in: http://gentoo.com/di/ Macros have to have their signature specified. e.g.: cmacro Tcl_InitMemory tcl.h void 'Tcl_Interp *' cmacro ScreenOfDisplay X11/Xlib.h 'Screen *' 'Display' int -- Brad
Overload resolution (value vs reference)
How does compiler selects the proper function among overloaded
functions which differ only in the way the argument is passed (by
reference or by value)? Is there a way, to control this behavior?
Result of execution of the test code:
passed by value: 8
passed by value: 3
passed by value: Foo(10)
Test code:
import std.stdio;
struct Foo {
int value;
}
void f(T)(T foo)
{
writeln("passed by value: ", foo);
}
void f(T)(const ref T foo)
{
writeln("passed by reference: ", foo);
}
void main()
{
// 8 is a r-vlaue, so it's passed by value
f(8);
// i is a l-value, it's passed by value and it's ok
int i = 3;
f(i);
// foo is a l-value, it's passed by value again, but if
structure will be big enough it'll be ineffective
auto foo = Foo();
foo.value = 10;
f(foo);
}
Re: override toString of Exception
On 10/20/2012 09:42 AM, David wrote:
class BraLaException : Exception {
this(string s, string f=__FILE__, size_t l=__LINE__) {
super(s, f, l);
}
override string toString() {
return "bar";
}
}
when throwing this exception, I still get the stacktrace and not "bar",
is it possible to change that behaviour?
The backtrace is not printed if the exception is caught:
class MyException : Exception
{
string s;
this(string s, string f=__FILE__, size_t l=__LINE__) {
super("bar", f, l);
this.s = s;
}
override string toString() const
{
return s;
}
}
void main()
{
try {
throw new MyException("bar");
} catch (Exception e) {
assert(e.toString() == "bar");
}
}
This behavior makes sense to me because printing the backtrace should
concern the application, not the exception itself. If the application
does not want the backtrace printed, it can handle all of the exceptions.
Ali
Re: Deimos organization
Other things I was unsure about: libmysql, libc, libruby, lua don't have any D files at all - some are just READMEs. Are these still under development? libexif goes libexif/libexif/(d files) instead of libexif/deimos/libexif/(d files). Also by design? Let me amend that. I won't do libc. Too much preprocessor crap. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Deimos organization
On Sun, 21 Oct 2012 00:52:02 -0500, Matt Soucy wrote: I've been messing around with Deimos lately, and I was a little bit confused about some of the design. I saw that a couple (ZeroMQ, ncurses) didn't follow the usual structure of having a c/ and a deimos/ folder. Is this by design, or is it something that should be remedied? Other things I was unsure about: libmysql, libc, libruby, lua don't have any D files at all - some are just READMEs. Are these still under development? libexif goes libexif/libexif/(d files) instead of libexif/deimos/libexif/(d files). Also by design? Thank you, Matt ncurses is mine, Just tell me how you'd like it to change, and I'll fix it. I've been meaning to fix that at some point... I'm willing to manually bind one of those empty repos, but I understand that there has been quite a lot of interest in using dstep to automate the binding process completely. Shrugs. It seems to me its languishing in hopes of an automated tool. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: override toString of Exception
when throwing this exception, I still get the stacktrace and not "bar", is it possible to change that behaviour? Perhaps this function can help: http://dlang.org/phobos/core_runtime.html#traceHandler Unfortunatly not, it expects a function which returns TraceInfo, but I simply wanna modify the resulting string (add escape sequences).
Re: Deimos organization
Am 21.10.2012 07:52, schrieb Matt Soucy:> I've been messing around with Deimos lately, and I was a little bit > confused about some of the design. > I saw that a couple (ZeroMQ, ncurses) didn't follow the usual structure > of having a c/ and a deimos/ folder. Is this by design, or is it > something that should be remedied? I think that c/ folder is intended. At least I always included the c-headers. > Other things I was unsure about: > libmysql, libc, libruby, lua don't have any D files at all - some are > just READMEs. Are these still under development? I think Walter maintains the deimos repos, but he only creates them and merges pull-requests, but others do the bindings, so if someone requests a binding XY but doesn't submit a pull request for XY it stays empty. > libexif goes libexif/libexif/(d files) instead of > libexif/deimos/libexif/(d files). Also by design? I think that should be fixed, pull request? > Thank you, > Matt
Re: SFML-D working example
On Sunday, 21 October 2012 at 07:34:29 UTC, Zhenya wrote: On Sunday, 21 October 2012 at 03:37:20 UTC, Ellery Newcomer wrote: On Saturday, 20 October 2012 at 20:25:09 UTC, Zhenya wrote: Hi!I have a little problem with building example.I downloaded SFML-D working example at this adress https://github.com/krzat/SFML-D/downloads.But then I tried to build it myself,I received many errors like that: Symbol Undefined _D2sf8graphics7Texture6__ctorMFPxaPS2sf8graphics7IntRectZC2sf8graphics7Texture I don't understand why compiler isn't satisfied,becouse I include all .lib files,that is in example.Can anybody help my? dmd_build.bat works for me. Are you using it? No,but I created VisualD solution,and configured its import path and lib path. Oh,I'm sorry I made some mistakes. Thank you for your answer)
Re: SFML-D working example
On Sunday, 21 October 2012 at 03:37:20 UTC, Ellery Newcomer wrote: On Saturday, 20 October 2012 at 20:25:09 UTC, Zhenya wrote: Hi!I have a little problem with building example.I downloaded SFML-D working example at this adress https://github.com/krzat/SFML-D/downloads.But then I tried to build it myself,I received many errors like that: Symbol Undefined _D2sf8graphics7Texture6__ctorMFPxaPS2sf8graphics7IntRectZC2sf8graphics7Texture I don't understand why compiler isn't satisfied,becouse I include all .lib files,that is in example.Can anybody help my? dmd_build.bat works for me. Are you using it? No,but I created VisualD solution,and configured its import path and lib path.
