Re: SQLite library on Windows
On Thursday, 27 December 2012 at 01:45:26 UTC, BLM768 wrote: I've been trying various methods to get SQLite working in Windows using the etc.c.sqlite3 bindings, but I can't figure out how to get everything in a form that DMD likes. GCC doesn't seem to output anything that OPTLINK can use, and I can't use the standard DLL build without creating a .LIB file from it, which apparently isn't an easy task. I can't remember if I've tried compiling SQLite with DMC, but I'd rather avoid that option because it would require a custom Makefile. Maybe objcopy has an option that would help... Has anyone managed to get the library working on Windows? Download their windows binaries and use Digital Mars implib tool with /system switch.
Questions about Pull-Requests
Are there some guidelines for doing pull request? Something like: -Guidelines for commit messages -TODO list before doing a pull request (e.g. run tests, untabifiy, etc) -How to reproduce autotester issues -General coding guidlines (what to indent, what not, line breaks etc) Problems I'm currently having (with the latest git version): -Running druntime unittests when druntime is compiled with -debug -g will produce huge masses of console output -Running druntime unittests x86 windows will give me an assertion: core/time.d line 720 seconds -Running druntime unittest x64 windows compiled with -g insteand of -O -release gives me an assertion in gc.gcx line 3200 Why does the autotester not find the unittest failure on x86 windows? Does the autotester only run the tests with release binaries? Because it misses the assertion when compiled with x64 windows debug. Kind Regards Benjamin Thaut
Re: Questions about Pull-Requests
On Thursday, December 27, 2012 11:46:38 Benjamin Thaut wrote: Are there some guidelines for doing pull request? Something like: -Guidelines for commit messages If you're fixing a bug, you need to have something along the lines of fix issue# 777 in it. Beyond that, I don't think that there are any actual guidelines. Just be intelligent about them. -TODO list before doing a pull request (e.g. run tests, untabifiy, etc) Definitely run the unit tests on your local box first. It's kind of pointless to create a pull request that's guaranteed to fail the pull tester. Tabs aren't permitted, so make sure that you remove them if you've added them, but that's part of the general style guide. -How to reproduce autotester issues Build them on the same type of system that's failing? I'm not sure what else you'd be looking for here. -General coding guidlines (what to indent, what not, line breaks etc) This is mostly up-to-date now: http://dlang.org/dstyle.html Most of the guidelines relate to the API rather than code formatting. The main formatting ones are that spaces must be used (never tabs), braces should generally go on their own line (so Allman style, not KR style), and lines have a soft limit of 80 characters and a hard limit of 120. Beyond that, formatting is pretty much just a question of trying to keep the style within a file consistent, but it varies from file to file. Problems I'm currently having (with the latest git version): -Running druntime unittests when druntime is compiled with -debug -g will produce huge masses of console output -Running druntime unittests x86 windows will give me an assertion: core/time.d line 720 seconds -Running druntime unittest x64 windows compiled with -g insteand of -O -release gives me an assertion in gc.gcx line 3200 Why does the autotester not find the unittest failure on x86 windows? That test depends on the the precision of your system clock, though I'm baffled as to how it could be failing with the TickDuration.ticksPerSec = unitsPerSec check in there - especially on seconds. Something about your system is triggering something that I've never seen before, though I suspect that it's more likely to be an issue with the test than the code being tested. Anything involving TickDuration is very hard to test properly, because TickDuration's precision depends on the system clock's precision. Does the autotester only run the tests with release binaries? Because it misses the assertion when compiled with x64 windows debug. It runs them in both debug and release on the POSIX systems, but it may only run them in release on Windows. I vaguely remember something about that, but I'm not sure. - Jonathan M Davis
Re: Questions about Pull-Requests
Jonathan M Davis: This is mostly up-to-date now: http://dlang.org/dstyle.html I didn't know it was updated. I like it! :-) My code already follows all those rules (the only one I don't follow is Braces should be on their own line, but that page it's only required for Phobos). Bye, bearophile
Re: Questions about Pull-Requests
On Thursday, December 27, 2012 12:15:19 bearophile wrote: Jonathan M Davis: This is mostly up-to-date now: http://dlang.org/dstyle.html I didn't know it was updated. I like it! :-) My code already follows all those rules (the only one I don't follow is Braces should be on their own line, but that page it's only required for Phobos). There are probably a few more tweaks that need to be done to it (e.g. I'd love to axe the section on comments, since we don't really follow it, and I think that it's unnecessary), but it's mostly correct now. - Jonathan M Davis
Re: Running out of memory
FG: But I also had to write s.length = 0 instead of either s = [] or s = .dup. Also try: s = null; I wonder why GC didn't reclaim the regions previously used by the array before it an empty one was assigned to s. Some explanations are in the article I've linked previously. Most of the other part of the answer comes from the fact that currently the D GC is not a precise GC. So randomly inbound pointers keep large memory chunks alive. Now, this delete looks like Python's kind of GC cheating. :) Personally, I don't have anything against it, but is it a valid practice? delete is deprecated in D. There is destroy(), and a replacement for delete in the memory/GC module. Bye, bearophile
Re: Running out of memory
On Thursday, 27 December 2012 at 14:15:51 UTC, FG wrote: ... Now, this delete looks like Python's kind of GC cheating. :) Personally, I don't have anything against it, but is it a valid practice? delete itself is considered deprecated in D2, though it was used widely, instead there is destroy function but i still use delete too and never tested that func. also u can call GC.collect() from core.memory module to tell GC to manually free mem but don't accept this as absolutely truth because i'm a bit noob
Re: Running out of memory
On Thursday, December 27, 2012 15:31:45 bearophile wrote: delete is deprecated in D. There is destroy(), and a replacement for delete in the memory/GC module. Indeed. But in this particular case, destroy would be of no help, as it specifically destroys objects _without_ freeing their memory. And at the moment, for strings, I don't believe that it's any different from setting them to null. - Jonathan M Davis
Re: Running out of memory
On 2012-12-27 15:31, bearophile wrote: delete is deprecated in D. There is destroy(), and a replacement for delete in the memory/GC module. destroy() had no effect on program's behavior, even the docs state that it does not initiate a GC cycle or free any GC memory. So I hope delete will still remain an option. Very useful for freeing such large chunks like in this example. What do you mean as the delete replacement?
Re: Questions about Pull-Requests
On 2012-12-27 12:05, Jonathan M Davis wrote: If you're fixing a bug, you need to have something along the lines of fix issue# 777 in it. Beyond that, I don't think that there are any actual guidelines. Just be intelligent about them. Then there's the standard Git guidelines. A commit message should consist of a short description, 50 characters max, followed by, if necessary, a blank line then a full description of the commit. If it's enough to describe the commit in 50 characters that's perfectly fine. -- /Jacob Carlborg
Re: Questions about Pull-Requests
On 2012-12-27 16:15, Jacob Carlborg wrote: Then there's the standard Git guidelines. A commit message should consist of a short description, 50 characters max, followed by, if necessary, a blank line then a full description of the commit. If it's enough to describe the commit in 50 characters that's perfectly fine. Not everyone is following this but it's good to try to follow it. The reason for this is that various git related software uses the first line to display it in a user interface or similar. For example, all git commits are sent to an email list as well. The first line is used as the subject of the email. Hence it's good if it's short. -- /Jacob Carlborg
Re: Running out of memory
On 2012-12-27 16:14, bearophile wrote: FG: What do you mean as the delete replacement? Take a look here (at free()): http://dlang.org/phobos/core_memory.html Oh, I didn't expect this to work on arrays but it does as good as delete: GC.free(cast(void*)s);
Re: Running out of memory
FG: GC.free(cast(void*)s); Don't cast arrays arrays to pointers, it's unclean, because dynamic arrays are two words long. Use the .ptr property: GC.free(s.ptr); Or if the precedent doesn't work: GC.free(cast(void*)s.ptr); Bye, bearophile
Re: SQLite library on Windows
Download their windows binaries and use Digital Mars implib tool with /system switch. Man, I wish I'd known about that tool a while ago... It seems to be working, but it looks like I'll need to recompile the DLL myself; DMD is looking for symbols mangled with a leading underscore, but the DLL has unmangled symbols, so optlink still complains.
Re: SQLite library on Windows
Man, I wish I'd known about that tool a while ago... It seems to be working, but it looks like I'll need to recompile the DLL myself; DMD is looking for symbols mangled with a leading underscore, but the DLL has unmangled symbols, so optlink still complains. Oh; never mind. I forgot the /system switch. Now I feel like an idiot. :)
Re: Using multiple processors
Le 24/12/2012 05:18, thedeemon a écrit : On Sunday, 23 December 2012 at 08:00:56 UTC, n00b wrote: Hello. My program has a great deal of computation to do, so I thought I'd create several threads in order to use multiple processors. It worked fine with a simple test program, but when I try to do this with the real program, only 1 processor is used. How much memory allocation is happening there? High allocation rate and often GCs can kill parallelism since they're happening with a lock. Anyway, too hard to tell anything certain without knowing what your program is doing. I tried to disable GC and reduce memory allocation. It didn't work but I guess allocation rate is still pretty high, I'll try to reduce it even more, and post more details if it still doesn't work. Thanks for your answer.
Re: Running out of memory
On 2012-12-27 16:31, bearophile wrote:
Don't cast arrays arrays to pointers, it's unclean, because dynamic arrays are
two words long. Use the .ptr property:
GC.free(s.ptr);
Things appeared sorted out, but it's not over yet. delete still wins.
Performance-wise free() worked like delete, but there's a nasty glitch.
The following code crashes after a few hundred loops with
core.exception.InvalidMemoryOperationError.
import std.stdio, core.memory;
char[1000] filler = '.';
int fill_times = 1;
void main(string[] args) {
char[] s;
for (int i=0; i 10; i++) {
for (int j=0; j fill_times; j++)
s ~= filler;
writeln(loop , i + 1, , length: , s.length);
stdout.flush();
// delete s;
GC.free(s.ptr);
s = [];
}
}
Surprisingly, when fill_times = 3736, it always crashes after the 341th loop
(relies on the filler having 1000 bytes. Some other numbers also trigger it).
What could be responsible for this odd behavior of free()?
Variadic templates with aliases
I wrote a simple template mixin to hold common operator overloads:
mixin template VectorSpaceOpsMixin(Vector, Scalar, alias a, alias
b)
{
Vector opUnary(string op)() if (op == -) {
return Vector(-a, -b);
}
...
}
This works like this:
struct complex {
double a, b;
mixin VectorSpaceOpsMixin!(complex, double, a, b);
...
}
Now this is nice, but it only works when the vector has 2 parts.
Can the mixin template be made more general? What will then be
its definition? This don't work:
mixin template `VectorSpaceOpsMixin(Vector, Scalar, alias...
parts) { ... }`
Obviously, this kind of functionality is easy with string mixins.
Can it be achieved without them?
Re: Parallelizing code -- design problem in networkish code
On 12/14/2012 01:50 PM, Ali Çehreli wrote:
On 12/13/2012 08:38 PM, Charles Hixson wrote:
Now what I was thinking of involved an array in another class (i.e., not
the Cell class) defined:
Cell[] cells;
and the Cell class, which includes:
public class Cell
{ ...
Idno[] ups;
...
}
where ups is an array of Id#s which are indexes into cells. While
processing a cell, it's likely that ups will be used to index into cells
to adjust the weight of the similar reference down to this cell. (And
yes, there's an array of downs also, which also does an indirection
through cells to adjust weights of cells below it that refer to the
current cell.
That description makes me think that there will be data races on
accesses to weights: Some threads will be reading while others will be
writing. If I understood correctly, that would disqualify std.parallelism.
This process offers several possibilities of collision when done in
parallel, which was why I though that the alterations should probably be
done via creating a new cell and substituting it for the original one.
If this isn't necessary, it would be a real benefit.
I hope others have answers to your questions. Sorry...
Ali
Actually, data races will probably need to be accepted. At least in the
sense of some (external) reads getting stale data. That's no worse than
using immutable data, and creating a new version with every change. (At
that point, any cell holding a pointer to the old version would be
working with stale data.) Any approach involving locking reads of data
will quickly devolve into a HUGE number of pairs of node in deadly embrace.
The thing is, stale data will generally only have a minor effect. And I
think that an optimal solution is probably impossible. What I'm looking
for is a sufficiently good solution. Think more of eventual
consistency than of consistency.
(Note: This reply is over 10 days after the original post, and is only
for the benefit of later readers.)
Re: Variadic templates with aliases
On Thu, Dec 27, 2012 at 9:01 PM, comco void.unsig...@gmail.com wrote:
I wrote a simple template mixin to hold common operator overloads:
mixin template VectorSpaceOpsMixin(Vector, Scalar, alias a, alias b)
{
Vector opUnary(string op)() if (op == -) {
return Vector(-a, -b);
}
...
}
This works like this:
struct complex {
double a, b;
mixin VectorSpaceOpsMixin!(complex, double, a, b);
...
}
Now this is nice, but it only works when the vector has 2 parts. Can the
mixin template be made more general? What will then be its definition? This
don't work:
mixin template `VectorSpaceOpsMixin(Vector, Scalar, alias... parts) { ...
}`
First, the `Vector` part is redundant. You can use `typeof(this)` inside
your mixin to have it 'look around` when it's being mixed in and determine
the type of the local `this`.
So you mixin can become:
mixin VectorSpaceOpsMixin!(double, a, b);
The `double` part could be deduced also, but it depends what kind of layout
you envision for your host structs. I'll let it there.
Now, concerning your question, you can make `parts` a template tuple
parameter:
VectorSpaceOpsMixin(Vector, Scalar, args...) {
The trouble is then that
mixin VectorSpaceOpsMixin!(complex, double, a, b);
will have `args` be the tuple (a,b) (it contains the symbols, not directly
the value themselves). Tuples are not arithmetic values in D (that would
not make sense for most tuples), so you cannot do `-args` directly.
From there, it depends whether you really want the flexibility that passing
a and b around gives you, or if you can make some simplifying assumptions
concerning your host struct.
Solution #1: full flexibility: you want to keep the capability to name the
fields upon which the mixin will act.
That way, even with
struct Vector { double a,b,c; }
you can have
mixin VectorSpaceOpsMixin!(Scalar, a,b); // Look Ma, no c!
You can iterate on `args`. By using `.stringof` on its elements, you get
this.a, this.b, ... From that, you can create the wanted code. Here is
a possibility:
mixin template VectorSpaceOpsMixin(Scalar, args...)
{
typeof(this) opUnary(string op)() if (op == -) {
typeof(this) temp;
// temp.a = -this.a;...
foreach(index, arg; args)
mixin(temp ~ args[index].stringof[4..$] ~ = - ~
args[index].stringof ~ ;);
return temp;
}
}
It creates a temporary, but then a direct return solution would also have
to make one. The only thing I'm not sure is when using floating point
types: `temp.a` and `temp.b` are all initialized with NaN. I don't know if
it's slow to assign another floating point to a NaN. I guess a one line
solution is doable, with a bit of string mixin magic.
Btw, of course, there will be many duplication between the different
arithmetic operators. You can also write another template to, er, write
your mixin for you. But I personally wouldn't bother: it's easier to
maintain explicit code.
Solution # 2: the mixin will act on all fields inside the struct = no need
to pass the names around. In this case, I would even use the first field as
the type of Scalar.
Everything is greatly simplified:
mixin template VectorSpaceOpsMixin2() // no args!
{
typeof(this) opUnary(string op)() if (op == -) {
typeof(this) temp;
foreach(index, ref arg; temp.tupleof)
arg = -this.tupleof[index];
return temp;
}
}
struct Complex {
double a, b;
mixin VectorSpaceOpsMixin2;
}
Don't worry about the foreach: it's all unrolled at compile-time.
Re: Running out of memory
On Thursday, December 27, 2012 16:00:58 FG wrote: On 2012-12-27 15:31, bearophile wrote: delete is deprecated in D. There is destroy(), and a replacement for delete in the memory/GC module. destroy() had no effect on program's behavior, even the docs state that it does not initiate a GC cycle or free any GC memory. So I hope delete will still remain an option. Very useful for freeing such large chunks like in this example. What do you mean as the delete replacement? The idea is that if you want to be manually freeing memory, you shouldn't be using GC memory in the first place. It's unsafe to be freeing it. Let the GC do that. If you want to manually manage memory, then manually manage it with malloc and free. If you really need to, then core.memory has the functions for managing the GC's memory, but you really shouldn't be doing that normally. Manual memory management should become easier once custom allocators are sorted out, since right now, if you want objects, you need to use emplace to turn the malloced memory into a proper object, and it's definitely more of a pain than using new with the GC. But it's still not a good idea in general to try and do manual memory management on GC memory. - Jonathan M Davis
Re: Questions about Pull-Requests
On Thursday, December 27, 2012 16:15:25 Jacob Carlborg wrote: On 2012-12-27 12:05, Jonathan M Davis wrote: If you're fixing a bug, you need to have something along the lines of fix issue# 777 in it. Beyond that, I don't think that there are any actual guidelines. Just be intelligent about them. Then there's the standard Git guidelines. A commit message should consist of a short description, 50 characters max, followed by, if necessary, a blank line then a full description of the commit. If it's enough to describe the commit in 50 characters that's perfectly fine. It's my understanding that limit is 80, not 50. - Jonathan M Davis
