Re: How would you create this construct?
On Friday, 30 March 2018 at 03:14:42 UTC, Mike Parker wrote: On Friday, 30 March 2018 at 02:30:01 UTC, Chris Katko wrote: [...] Something like this? = import std.stdio; [...] This is beautiful. I mean, the struct stuff looks complicated/non-intuitive at first, but it's all boilerplate.
Re: Fast GC allocation of many small objects
On Friday, 30 March 2018 at 23:09:33 UTC, Alexandru Jercaianu
wrote:
On Friday, 30 March 2018 at 20:46:43 UTC, Per Nordlöw wrote:
On Friday, 30 March 2018 at 20:38:35 UTC, rikki cattermole
wrote:
Use a custom allocator (that could be backed by the GC) using
std.experimental.allocators :)
https://dlang.org/phobos/std_experimental_allocator.html
is massive.
I guess I should allocate my nodes using
auto node = theAllocator.make!StrNode("alpha");
Could someone please give a working example of a GC-backed
`theAllocator` suitable for my allocation pattern?
Hello,
You can try the following:
struct Node
{
char[64] arr;
}
enum numNodes = 100_000_000;
void[] buf = GCAllocator.instance.allocate(numNodes *
Node.sizeof);
auto reg = Region!(NullAllocator, 16)(cast(ubyte[])buf);
foreach(i; 0 .. numNodes)
{
Node* n = cast(Node*)(reg.allocate(Node.sizeof).ptr);
// do stuff with the new node
}
I benchmarked this versus
foreach(i; 0 .. numNodes)
{
auto n = new Node;
// do stuff...
}
The first approach was about 30% faster.
Sorry, I forgot to add which imports you need:
import std.experimental.allocator.gc_allocator;
import std.experimental.allocator.building_blocks.region;
import std.experimental.allocator.building_blocks.null_allocator;
Re: Fast GC allocation of many small objects
On Friday, 30 March 2018 at 20:46:43 UTC, Per Nordlöw wrote:
On Friday, 30 March 2018 at 20:38:35 UTC, rikki cattermole
wrote:
Use a custom allocator (that could be backed by the GC) using
std.experimental.allocators :)
https://dlang.org/phobos/std_experimental_allocator.html
is massive.
I guess I should allocate my nodes using
auto node = theAllocator.make!StrNode("alpha");
Could someone please give a working example of a GC-backed
`theAllocator` suitable for my allocation pattern?
Hello,
You can try the following:
struct Node
{
char[64] arr;
}
enum numNodes = 100_000_000;
void[] buf = GCAllocator.instance.allocate(numNodes *
Node.sizeof);
auto reg = Region!(NullAllocator, 16)(cast(ubyte[])buf);
foreach(i; 0 .. numNodes)
{
Node* n = cast(Node*)(reg.allocate(Node.sizeof).ptr);
// do stuff with the new node
}
I benchmarked this versus
foreach(i; 0 .. numNodes)
{
auto n = new Node;
// do stuff...
}
The first approach was about 30% faster.
Re: Fast GC allocation of many small objects
On 31/03/2018 9:46 AM, Per Nordlöw wrote:
On Friday, 30 March 2018 at 20:38:35 UTC, rikki cattermole wrote:
Use a custom allocator (that could be backed by the GC) using
std.experimental.allocators :)
https://dlang.org/phobos/std_experimental_allocator.html
is massive.
I guess I should allocate my nodes using
auto node = theAllocator.make!StrNode("alpha");
Could someone please give a working example of a GC-backed
`theAllocator` suitable for my allocation pattern?
The default theAllocator is the GC :)
Also see[0] and "Sample Assembly"[1] which will really interest you I think.
[0] https://dlang.org/phobos/std_experimental_allocator_gc_allocator.html
[1] https://dlang.org/phobos/std_experimental_allocator_building_blocks.html
Re: Fix transposed ranges
On Friday, 30 March 2018 at 20:43:09 UTC, Cym13 wrote: Hi, I've got the following code that takes a list of files as argument and xor them together (demo example sufficient for that discussion). [...] Forgot to mention but I'm also quite annoyed at the need for that ".array" because "transposed" requires the RoR to be assignable. That kills the laziness. I'm very much open to suggestions regarding that point.
Re: Fast GC allocation of many small objects
On Friday, 30 March 2018 at 20:38:35 UTC, rikki cattermole wrote:
Use a custom allocator (that could be backed by the GC) using
std.experimental.allocators :)
https://dlang.org/phobos/std_experimental_allocator.html
is massive.
I guess I should allocate my nodes using
auto node = theAllocator.make!StrNode("alpha");
Could someone please give a working example of a GC-backed
`theAllocator` suitable for my allocation pattern?
Fix transposed ranges
Hi, I've got the following code that takes a list of files as
argument and xor them together (demo example sufficient for that
discussion).
import std.stdio;
import std.array;
import std.range;
import std.algorithm;
auto rawContent(string path) {
return File(path).byChunk(16384).joiner;
}
void main(string[] args) {
args[1..$]
.map!rawContent
.array
.transposed
.map!(bytes => bytes.fold!((a, b) => a^b))
.writeln;
}
This works but compiles with deprecations:
/usr/include/dlang/dmd/std/algorithm/iteration.d(663):
Deprecation: function `std.range.Transposed!(Result[],
cast(TransverseOptions)0).Transposed.save` is deprecated - This
function is incorrect and will be removed November 2018. See the
docs for more details.
/usr/include/dlang/dmd/std/algorithm/iteration.d(663):
Deprecation: function `std.range.Transposed!(Result[],
cast(TransverseOptions)0).Transposed.save` is deprecated - This
function is incorrect and will be removed November 2018. See the
docs for more details.
What do you think the best course of action would be to make that
code resilient to the modification of transposed? I'd very *very*
much like to keep a straight UFCS line, there's really nothing
complicated enough with this operation to warrant writting
boilerplate, yet the straightforward solution isn't
future-proof...
Any idea?
Re: Fast GC allocation of many small objects
On 31/03/2018 9:31 AM, Per Nordlöw wrote:
I'm working on a graph database with tens of millions of small nodes
containing typically around 8-64 bytes of member data. Is there a faster
way of allocating many small class objects such as
class Node
{
// abstract members
}
class StrNode : Node
{
string value;
}
// more Node-types...
other than
const nodeCount = 10_000_000;
foreach (0 .. n)
{
auto node = new Node(someData);
// connect node...
}
Use a custom allocator (that could be backed by the GC) using
std.experimental.allocators :)
Fast GC allocation of many small objects
I'm working on a graph database with tens of millions of small
nodes containing typically around 8-64 bytes of member data. Is
there a faster way of allocating many small class objects such as
class Node
{
// abstract members
}
class StrNode : Node
{
string value;
}
// more Node-types...
other than
const nodeCount = 10_000_000;
foreach (0 .. n)
{
auto node = new Node(someData);
// connect node...
}
Re: How to customize vibe.data.json
On Friday, 30 March 2018 at 17:58:23 UTC, Seb wrote:
On Friday, 30 March 2018 at 16:47:52 UTC, kerdemdemir wrote:
Hi,
In vibe's web
page(http://vibed.org/api/vibe.data.json/serializeToJson) it
is told that I should implement
[...]
I think you are looking for this -
https://github.com/vibe-d/vibe.d/pull/2088
Feel free to ping the people there ;-)
struct RESTTime {
SysTime time;
alias time this;
static RESTTime fromString(string v) { return
RESTTime(SysTime.fromSimpleString(v)); }
string toString() const { return time.toSimpleString(); }
}
struct TradeData
{
UUID sellOrderID;
UUID buyOrderID;
SysTime buyOrderTime;
SysTime sellOrderTime;
}
void main() {
TradeData t;
Json jsonResult = t.serializeToJson();
writeln(jsonResult.toString());
}
Unfortunately what is shown in the forum is not working with
SysTime.
My program is being terminated in
std.datetime.SysTime.toISOExtString() function by code -11.
I hope it does not sound like spoon feeding but do you know any
solution for SysTime. I am working on it in parallel.
Erdem
Re: How to customize vibe.data.json
On Friday, 30 March 2018 at 17:58:23 UTC, Seb wrote: On Friday, 30 March 2018 at 16:47:52 UTC, kerdemdemir wrote: Hi, In vibe's web page(http://vibed.org/api/vibe.data.json/serializeToJson) it is told that I should implement [...] I think you are looking for this - https://github.com/vibe-d/vibe.d/pull/2088 Feel free to ping the people there ;-) Exactly :)
Re: How to customize vibe.data.json
On Friday, 30 March 2018 at 16:47:52 UTC, kerdemdemir wrote: Hi, In vibe's web page(http://vibed.org/api/vibe.data.json/serializeToJson) it is told that I should implement [...] I think you are looking for this - https://github.com/vibe-d/vibe.d/pull/2088 Feel free to ping the people there ;-)
How to customize vibe.data.json
Hi,
In vibe's web
page(http://vibed.org/api/vibe.data.json/serializeToJson) it is
told that I should implement
Json toJson() const;
static T fromJson(Json src);
string toString() const;
static T fromString(string src);
I think I should implement those as member functions(I am not
sure).
I have a struct like:
struct TradeData
{
import std.uuid : UUID;
import std.datetime : SysTime;
UUID sellOrderID;
UUID buyOrderID;
SysTime buyOrderTime;
SysTime sellOrderTime;
}
What I want is automatically json conversion of UUID and SysTime
classes by returning UUID.toString() and SysTime.toSimpleString()
methods when serializeToJson() is being called. Since these are
std classes I don't know how can I manipulate them.
Can you please tell me how should I use toJson(), fromJson()
functions for customizing user defined objects.
Erdem
Re: dynamically compile and load glue logic
On 31/03/2018 3:38 AM, H. S. Teoh wrote: On Fri, Mar 30, 2018 at 02:24:45PM +, yawniek via Digitalmars-d-learn wrote: in how far is it or would the following be possible: dynamically compile and execute some glue logic that is also written in D under linux? and what happens if that code uses phobos or other dub libs that are available in the host binary? [...] I've written a program where user input drives the generation of a code snippet that then gets passed to an invocation of dmd, compiled, linked and the loaded as a shared library via dlopen(), et al. It works fairly well, and dmd generally is fast enough that the pause is not very noticeable. (This was before dmd-as-a-library was available; I imagine it would be even faster today now that you don't have to spawn a separate dmd process.) Still need to, front end is barely alpha. Can't be reset for example (last I checked, but doubt that has changed).
Re: dynamically compile and load glue logic
On Fri, Mar 30, 2018 at 02:24:45PM +, yawniek via Digitalmars-d-learn wrote: > in how far is it or would the following be possible: > > dynamically compile and execute some glue logic that is also written > in D under linux? > > and what happens if that code uses phobos or other dub libs that are > available in the host binary? [...] I've written a program where user input drives the generation of a code snippet that then gets passed to an invocation of dmd, compiled, linked and the loaded as a shared library via dlopen(), et al. It works fairly well, and dmd generally is fast enough that the pause is not very noticeable. (This was before dmd-as-a-library was available; I imagine it would be even faster today now that you don't have to spawn a separate dmd process.) The generated snippet does import std.math, and it seems to work fine. But then the generated snippets tend to be fairly small, and only use a limited subset of the language, so there may be gotchas that I'm not aware of. T -- Don't throw out the baby with the bathwater. Use your hands...
dynamically compile and load glue logic
in how far is it or would the following be possible: dynamically compile and execute some glue logic that is also written in D under linux? and what happens if that code uses phobos or other dub libs that are available in the host binary? especially the 2nd point is important as i would want to load 100's of those snippets. For the context: the idea is to create a streaming-ETL system where you can dynamically add/remove rules/modules. it would be great if logic could be written directly in D and operate on specific (library provided) objects.
Re: Initializing a class member that is an object
On Friday, 30 March 2018 at 11:14:32 UTC, ketmar wrote: please, make an ER in bugzilla then. 'cause it will be lost here, and with ER we have a chance to eventually do that. Will do.
Re: Initializing a class member that is an object
Laurent Tréguier wrote: On Friday, 30 March 2018 at 11:04:59 UTC, ketmar wrote: p.s.: still, it may be nice to warn user about that. 'cause such runtime initializations are really belong to static ctor. dunno, i'm ok both with warning and without it. I simply think a word about it in the docs would be nice, since this is tricky if you come from another language that doesn't do this. Otherwise I'm fine with it (and it's not exactly hard to fix either) please, make an ER in bugzilla then. 'cause it will be lost here, and with ER we have a chance to eventually do that.
Re: Initializing a class member that is an object
On Friday, 30 March 2018 at 11:04:59 UTC, ketmar wrote: p.s.: still, it may be nice to warn user about that. 'cause such runtime initializations are really belong to static ctor. dunno, i'm ok both with warning and without it. I simply think a word about it in the docs would be nice, since this is tricky if you come from another language that doesn't do this. Otherwise I'm fine with it (and it's not exactly hard to fix either)
Re: Link-time optimisation (LTO)
On Friday, 30 March 2018 at 10:23:15 UTC, Cecil Ward wrote: Say that I use say GDC or LDC. I want to declare a routine as public in one compilation unit (.d src file) and be able to access it from other compilation units. Do I simply declare the routine with the word keyword public before the usual declaration? Or maybe that is the default, like not using the keyword static with function declarations in C? Global functions in a module have default "public" visibility indeed. https://dlang.org/spec/attribute.html#visibility_attributes My principal question: If I successfully do this, with GCC or LDC, will I be able to get the code for the externally defined short routine expanded inline and fully integrated into the generated code that corresponds to the calling source code? (So no ‘call’ instruction is even found.) What you want is "cross-module inlining". As far as I know, DMD and GDC will do cross-module inlining (if inlining is profitable). LDC does not, unless `-enable-cross-module-inlining` is enabled (which is aggressive and may result in linking errors in specific cases, https://github.com/ldc-developers/ldc/pull/1737). LDC with LTO enabled will definitely give you cross-module inlining (also for private functions). You can force inlining with `pragma(inline, true)`, but it is usually better to leave that decision up to the compiler. https://dlang.org/spec/pragma.html#inline -Johan
Re: Initializing a class member that is an object
p.s.: still, it may be nice to warn user about that. 'cause such runtime initializations are really belong to static ctor. dunno, i'm ok both with warning and without it.
Re: Initializing a class member that is an object
Laurent Tréguier wrote: Is this behavior really intentional ? yes. default values should be the same for all objects. it is predictable, and allows to initialize objects to the known state simply by blitting `.init`. that is, default values aren't a syntax sugar for defining implicit ctor actions, they are executed once. this is by design.
Re: Building application with LDC and -flto=thin fails in link stage
On Wednesday, 28 March 2018 at 16:42:23 UTC, Johan Engelen wrote: On Tuesday, 27 March 2018 at 22:10:33 UTC, Per Nordlöw wrote: On Tuesday, 27 March 2018 at 22:00:42 UTC, Johan Engelen wrote: Indeed. Please try to manually link first (without dub) by modifying the command on which dub errors: ``` ldmd2 -flto=thin -of.dub/build/application-release-nobounds-lto-linux.posix-x86_64-ldc_2078-F2904BE3C4DA237C077E5C2B0B23442E/knetquery .dub/build/application-release-nobounds-lto-linux.posix-x86_64-ldc_2078-F2904BE3C4DA237C077E5C2B0B23442E/knetquery.o ../../.dub/packages/gmp-d-master/gmp-d/.dub/build/library-release-nobounds-lto-linux.posix-x86_64-ldc_2078-B287F67CE5FF6145BC229790CFB09607/libgmp-d.a phobos-next/.dub/build/library-release-nobounds-lto-linux.posix-x86_64-ldc_2078-F0F2FDB01B8401C04D657BCC145D46A5/libknet_phobos-next.a -L--no-as-needed -L-lzstd -L-lgmp -L-lc -L-lreadline -L-lz -L-lbz2 ``` -Johan Yes, that works! I'm no dub expert and I don't know how to pass flags to the _compiler_ for the link step. Would be good to figure that out with the dub folks. For a stopgap solution: I think what you are doing is passing `--compiler=/ldmd2` (note: LDMD) to dub when building, and you get separate compilation+linking. If you'd use `--compiler=/ldc2` (note: LDC), then you would not get separate compilation+linking [1] and things work with just setting `dflags`. I've asked about the problem here: https://github.com/dlang/dub/issues/1431 cheers, Johan [1] https://github.com/dlang/dub/issues/809
Initializing a class member that is an object
Coming from a more Java-esque background, I'm used to sometimes
initializing class members outside of the constructor :
class MyClass {
Object member = new Object();
}
I've tried using this in D, but I've come to realize it acts very
differently. In Java, the `new Object()` will be executed every
time a new `MyClass` object is instantiated. In D however, it
seems to be executed once, and members of every `MyClass` object
will then be initialized with a reference to that one unique
`Object`.
Example: https://run.dlang.io/is/Qlx2xY
Is this behavior really intentional ? I don't really see how it
could be useful, and it's really confusing at first to see new
objects with weird values even if none of their members has been
touched yet...
Re: Building application with LDC and -flto=thin fails in link stage
On Thursday, 29 March 2018 at 08:44:21 UTC, Jacob Carlborg wrote: Please read the reply :), although it could be a bit more clear. I'll spell it out for you. Both `dflags` and `lflags` are being used already. With separate compilation and linking, there seems to be no way to pass flags to the compiler during the linking step: dflags is not used and lflags is prefixed with `-L`. -Johan
Link-time optimisation (LTO)
Say that I use say GDC or LDC. I want to declare a routine as public in one compilation unit (.d src file) and be able to access it from other compilation units. Do I simply declare the routine with the word keyword public before the usual declaration? Or maybe that is the default, like not using the keyword static with function declarations in C? My principal question: If I successfully do this, with GCC or LDC, will I be able to get the code for the externally defined short routine expanded inline and fully integrated into the generated code that corresponds to the calling source code? (So no ‘call’ instruction is even found.)
