Re: auto-decoding
On Sunday, 1 April 2018 at 01:19:08 UTC, auto wrote: What is auto decoding and why it is a problem? Auto-decoding is essentially related to UTF representation of Unicode strings. In D, `char[]` and `string` represent UTF8 strings, `wchar[]` and `wstring` represent UTF16 strings and `dchar[]` and `dstring` represent UTF32 strings. You need to know how UFT works in order to understand auto-decoding. Since in practice most code deals with UTF8 I'll explain wrt that. Essentially, the problem comes down to the fact that not all the Unicode characters are representable by 8 bit `char`s (for UTF8). Only the ASCII stuff is represented by the "normal" way. UTF8 uses the fact that the first few buts in a char are never used in ASCII, to tell how many more `char`s ahead that character is encoded in. You can watch this video for a better understanding[0]. By default though, if one were to traverse a `char` looking for characters, they would get unexpected results with Unicode data Auto-decoding tries to solve this by automatically applying the algorithm to decode the characters to Unicode "Code-Points". This is where my knowledge ends though. I'll give you pros and cons of auto-decoding. Pros: * It makes Unicode string handeling much more easier for beginners. * Much less effort in general, it seems to "just work™" Cons: * It makes string handling slow by default * It may be the wrong thing, since you may not want Unicode code-points, but graphemes instead. * Auto-decoding throws exceptions on reaching invalid code-points, so all string handling code in general throws exceptions. If you want to stop auto-decoding, you can use std.string.representation like this: import std.string : representation; auto no_decode = some_string.representation; Now no_decode wont be auto-decoded, and you can use it in place of some_string. You can also use std.utf to decode by graphemes instead. You should also read this blog post: https://jackstouffer.com/blog/d_auto_decoding_and_you.html And this forum post: https://forum.dlang.org/post/eozguhavggchzzruz...@forum.dlang.org [0]: https://www.youtube.com/watch?v=MijmeoH9LT4
auto-decoding
What is auto decoding and why it is a problem?
Re: Fast GC allocation of many small objects
On Saturday, 31 March 2018 at 20:17:26 UTC, Per Nordlöw wrote: auto reg = Region!(NullAllocator, 16)(cast(ubyte[])buf); Thanks! Turns out that Region allocator wasn't fully qualified: https://github.com/dlang/phobos/pull/6400 This will make it possible to allocate with it in pure code :)
Re: Fast GC allocation of many small objects
On Friday, 30 March 2018 at 23:09:33 UTC, Alexandru Jercaianu
wrote:
Hello,
You can try the following:
struct Node
{
char[64] arr;
}
enum numNodes = 100_000_000;
void[] buf = GCAllocator.instance.allocate(numNodes *
Node.sizeof);
auto reg = Region!(NullAllocator, 16)(cast(ubyte[])buf);
Thanks!
Is a `minAlign` of 16 recommended over 8 when allocating classes
or arrays?
Re: Fast GC allocation of many small objects
On Saturday, 31 March 2018 at 19:38:31 UTC, Per Nordlöw wrote: On Saturday, 31 March 2018 at 19:31:37 UTC, Rubn wrote: Be willing to change your code, the allocator can change at any point. What you implement today may not work tomorrow, what you fix to work for tomorrow may not end up working the next day (in terms of releases). That really should be something that is mentioned when you suggest using an experimential feature, there's no guarantees at all. It might not even get put into phobos. If your project is going to be used over the course of a year or more than maybe you shouldn't use it. Ok, thanks for the point. Is the dub package stdx-allocator [1] a better alternative in this regard? [1] https://github.com/dlang-community/stdx-allocator Yep, that's why it was created. It's a snapshot of https://docarchives.dlang.io/v2.077.0/phobos/std_experimental_allocator.html It might be updated later, but with dub it's really easy to lock it to a specific version - which you can't when upgrading the compiler.
Re: Fast GC allocation of many small objects
On Saturday, 31 March 2018 at 19:31:37 UTC, Rubn wrote: Be willing to change your code, the allocator can change at any point. What you implement today may not work tomorrow, what you fix to work for tomorrow may not end up working the next day (in terms of releases). That really should be something that is mentioned when you suggest using an experimential feature, there's no guarantees at all. It might not even get put into phobos. If your project is going to be used over the course of a year or more than maybe you shouldn't use it. Ok, thanks for the point. Is the dub package stdx-allocator [1] a better alternative in this regard? [1] https://github.com/dlang-community/stdx-allocator
Re: Fast GC allocation of many small objects
On Friday, 30 March 2018 at 20:46:43 UTC, Per Nordlöw wrote:
On Friday, 30 March 2018 at 20:38:35 UTC, rikki cattermole
wrote:
Use a custom allocator (that could be backed by the GC) using
std.experimental.allocators :)
https://dlang.org/phobos/std_experimental_allocator.html
is massive.
I guess I should allocate my nodes using
auto node = theAllocator.make!StrNode("alpha");
Could someone please give a working example of a GC-backed
`theAllocator` suitable for my allocation pattern?
Be willing to change your code, the allocator can change at any
point. What you implement today may not work tomorrow, what you
fix to work for tomorrow may not end up working the next day (in
terms of releases). That really should be something that is
mentioned when you suggest using an experimential feature,
there's no guarantees at all. It might not even get put into
phobos. If your project is going to be used over the course of a
year or more than maybe you shouldn't use it.
Re: Fast GC allocation of many small objects
On Saturday, 31 March 2018 at 09:10:13 UTC, Boris-Barboris wrote:
On Friday, 30 March 2018 at 20:31:35 UTC, Per Nordlöw wrote:
Is there a faster way of allocating many small class objects
such as...
maybe something like this:
import std.conv: to;
import std.stdio;
class Node {}
class StrNode : Node
{
string value;
}
void main()
{
writeln(StrNode.classinfo.name);// onlineapp.StrNode
size_t il = StrNode.classinfo.m_init.length;
writeln(il); // 32
void[] backBuf = new void[il * 1000];
StrNode[] nodes = new StrNode[1000];
for (int i = 0; i < 1000; i++)
{
backBuf[i * il .. (i+1) * il] =
StrNode.classinfo.m_init;
nodes[i] = cast(StrNode) [i * il];
nodes[i].value = i.to!string;
}
foreach (n; nodes[995..$])
writeln(n.classinfo.name, " ", n.value);
// prints onlineapp.StrNode 995-999
}
I would have used std.conv.emplace:
import std.stdio;
import std.conv : emplace, to;
class Node {
string value;
this (long n) {
this.value = n.to!string;
}
override string toString () {
return "Node (value: " ~ value ~ ")";
}
}
void main (string [] args) {
/* size_t size = Node.sizeof; */
size_t size = Node.classinfo.m_init.length;
void [] memory = new void [size * 1000];
Node [] nodes = new Node [1000];
foreach (i, node; nodes) {
void [] buf = memory [i * size.. i * size + size];
nodes [i] = emplace!Node (buf, i);
}
nodes [0] .writeln;
nodes [$-1].writeln;
}
Re: Reactive data
On Friday, 23 March 2018 at 12:59:23 UTC, crimaniak wrote:
I want to have reactive variables like in this example:
```
USING_REACTIVE_DOMAIN(D)
// The two words
VarSignalT firstWord = MakeVar(string( "Change" ));
VarSignalT secondWord = MakeVar(string( "me!" ));
// ...
SignalT bothWords = firstWord + string( " " ) +
secondWord;
```
from this page:
http://schlangster.github.io/cpp.react/tutorials/BasicSignals.html
Is this possible to make it in D with
https://github.com/lempiji/rx ? Is there other libraries exists
for this topic?
I think that it can use combineLatest which was recently added.
If using 'alias this' or 'operator overloading', might look like
more original example.
---
import rx;
auto firstWord = new BehaviorSubject!string("Change");
auto secondWord = new BehaviorSubject!string("me!");
auto bothWords = new BehaviorSubject!string("");
combineLatest!((a, b) => a ~ " " ~ b)(firstWord,
secondWord).doSubscribe(bothWords);
writeln(bothWords.value); // Change me!
firstWord.value = "TEST";
writeln(bothWords.value); // TEST me!
---
Re: Link-time optimisation (LTO)
On Friday, 30 March 2018 at 10:23:15 UTC, Cecil Ward wrote: My principal question: If I successfully do this, with GCC or LDC, will I be able to get the code for the externally defined short routine expanded inline and fully integrated into the generated code that corresponds to the calling source code? (So no ‘call’ instruction is even found.) If you compile for lto, ldc compiles code to IR and linker generates machine code from IR with inlining, at the IR level most information about source code is lost, only the actual code remains, declarations are purely source code items. But it only inline IR code, machine code can't be inlined.
Re: Fast GC allocation of many small objects
On Friday, 30 March 2018 at 20:31:35 UTC, Per Nordlöw wrote:
Is there a faster way of allocating many small class objects
such as...
maybe something like this:
import std.conv: to;
import std.stdio;
class Node {}
class StrNode : Node
{
string value;
}
void main()
{
writeln(StrNode.classinfo.name);// onlineapp.StrNode
size_t il = StrNode.classinfo.m_init.length;
writeln(il); // 32
void[] backBuf = new void[il * 1000];
StrNode[] nodes = new StrNode[1000];
for (int i = 0; i < 1000; i++)
{
backBuf[i * il .. (i+1) * il] = StrNode.classinfo.m_init;
nodes[i] = cast(StrNode) [i * il];
nodes[i].value = i.to!string;
}
foreach (n; nodes[995..$])
writeln(n.classinfo.name, " ", n.value);
// prints onlineapp.StrNode 995-999
}
