Re: Assigning global and static associative arrays
On Saturday, 1 September 2012 at 09:16:30 UTC, Jonathan M Davis wrote: [SNIP] so it looks like not only do all instances of the same string enum use the same memory, but another enum with the same string literal shares it as well. So, only one is allocated. And on Linux at least, as I understand it, the string literals go in ROM. But it may be that the above code functions differently in Windows, since it _doesn't_ put string literals in ROM. [SNIP] FYI: I get the exact same behavior in Windows. Not that it matters, but it sounded like you were asking. I'm a bit confused now though: Why would someone want to use an enum when they could use a static immutable instead? If I understood correctly, the enum will *always* be inlined (doesn't create any actual symbols). But if you use a static immutable, then the compiler will create an actual symbol, but probably inline it away if it judges that is a better choice anyways... Is there *any* scenario where one would choose the enum over the static immutable...?
Re: Assigning global and static associative arrays
On 09/02/2012 03:45 PM, monarch_dodra wrote: On Saturday, 1 September 2012 at 09:16:30 UTC, Jonathan M Davis wrote: [SNIP] so it looks like not only do all instances of the same string enum use the same memory, but another enum with the same string literal shares it as well. So, only one is allocated. And on Linux at least, as I understand it, the string literals go in ROM. But it may be that the above code functions differently in Windows, since it _doesn't_ put string literals in ROM. [SNIP] FYI: I get the exact same behavior in Windows. Not that it matters, but it sounded like you were asking. I'm a bit confused now though: Why would someone want to use an enum when they could use a static immutable instead? If I understood correctly, the enum will *always* be inlined (doesn't create any actual symbols). But if you use a static immutable, then the compiler will create an actual symbol, but probably inline it away if it judges that is a better choice anyways... Is there *any* scenario where one would choose the enum over the static immutable...? - If there is no need to access it at run time. - Type deduction.
Re: Assigning global and static associative arrays
On Sunday, 2 September 2012 at 16:20:16 UTC, Timon Gehr wrote: On 09/02/2012 03:45 PM, monarch_dodra wrote: FYI: I get the exact same behavior in Windows. Not that it matters, but it sounded like you were asking. I'm a bit confused now though: Why would someone want to use an enum when they could use a static immutable instead? If I understood correctly, the enum will *always* be inlined (doesn't create any actual symbols). But if you use a static immutable, then the compiler will create an actual symbol, but probably inline it away if it judges that is a better choice anyways... Is there *any* scenario where one would choose the enum over the static immutable...? - If there is no need to access it at run time. - Type deduction. -Type deduction: Not really, I can just declare it as immutable auto. -no need to access it at run time. I guess.
Re: Assigning global and static associative arrays
On 09/02/2012 06:26 PM, monarch_dodra wrote:
On Sunday, 2 September 2012 at 16:20:16 UTC, Timon Gehr wrote:
On 09/02/2012 03:45 PM, monarch_dodra wrote:
FYI: I get the exact same behavior in Windows. Not that it
matters, but it sounded like you were asking.
I'm a bit confused now though: Why would someone want to use an
enum when they could use a static immutable instead?
If I understood correctly, the enum will *always* be inlined
(doesn't create any actual symbols). But if you use a static
immutable, then the compiler will create an actual symbol, but
probably inline it away if it judges that is a better choice
anyways...
Is there *any* scenario where one would choose the enum over the
static immutable...?
- If there is no need to access it at run time.
- Type deduction.
-Type deduction: Not really, I can just declare it as immutable auto.
enum x = 2;
void main(){
auto y = x;
y = 3;
}
-no need to access it at run time. I guess.
Re: Assigning global and static associative arrays
On 09/02/2012 06:45 AM, monarch_dodra wrote: Is there *any* scenario where one would choose the enum over the static immutable...? That's a good question. If Timon Gehr's example is the only difference, I wonder whether the guidelines that I had come up with are still valuable? I would appreciate if someone could review the guidelines under the How to use section here: http://ddili.org/ders/d.en/const_and_immutable.html Ali
Re: Assigning global and static associative arrays
On Sun, Sep 2, 2012 at 8:50 PM, Ali Çehreli acehr...@yahoo.com wrote: On 09/02/2012 06:45 AM, monarch_dodra wrote: Is there *any* scenario where one would choose the enum over the static immutable...? Due to Jonathan advice, I converted a part of my code (enum = static this). At runtime, I got a 40% decrease in runtime, a *very* good news. But then CTFE does not work anymore: some functions tell me the static AA I use is not initialized (or whatever). I tried to use if(__ctfe) but then I lose most of the speedup. All in all, for my Pegged parser generator project, I ended up generating code specifically for compile-time (for CT parsing) and some other function for runtime. So yes, enums are useful for CT computation sometimes.
Re: Assigning global and static associative arrays
On 09/02/2012 12:24 PM, Philippe Sigaud wrote: On Sun, Sep 2, 2012 at 8:50 PM, Ali Çehreliacehr...@yahoo.com wrote: On 09/02/2012 06:45 AM, monarch_dodra wrote: Is there *any* scenario where one would choose the enum over the static immutable...? Due to Jonathan advice, I converted a part of my code (enum = static this). At runtime, I got a 40% decrease in runtime, a *very* good news. But then CTFE does not work anymore: some functions tell me the static AA I use is not initialized (or whatever). I tried to use if(__ctfe) but then I lose most of the speedup. All in all, for my Pegged parser generator project, I ended up generating code specifically for compile-time (for CT parsing) and some other function for runtime. So yes, enums are useful for CT computation sometimes. Thank you. Then I am sticking with the guidelines under the How to use section here: http://ddili.org/ders/d.en/const_and_immutable.html There is some warning there against making arrays and associative arrays enums: quote enum constants bring a hidden cost when they are used for arrays or associative arrays [...] For that reason, it may make more sense to define arrays and associative arrays as immutable variables if they are going to be used more than once in the program. /quote quoteConsider the hidden cost of enum arrays and enum associative arrays. Define them as immutable variables if the arrays are large and they are used more than once in the program./quote Ali
Re: Assigning global and static associative arrays
On Sat, Sep 1, 2012 at 12:21 AM, Jonathan M Davis jmdavisp...@gmx.com wrote: Using an enum is particularly bad for an AA, since it's not exactly a simple data type, and there's definitely some cost to constructing them. Yeah, my (nasty) bad. I'm too used to put 'enum' everywhere. Gosh, I confirm using static this() instead of enum results in a 40% decrease in runtime for me. I'll scourge my code from enum AA...
Re: Assigning global and static associative arrays
On Saturday, September 01, 2012 09:39:04 Philippe Sigaud wrote: On Sat, Sep 1, 2012 at 12:21 AM, Jonathan M Davis jmdavisp...@gmx.com wrote: Using an enum is particularly bad for an AA, since it's not exactly a simple data type, and there's definitely some cost to constructing them. Yeah, my (nasty) bad. I'm too used to put 'enum' everywhere. Gosh, I confirm using static this() instead of enum results in a 40% decrease in runtime for me. I'll scourge my code from enum AA... Using enum can be very useful, but I wouldn't use it for AAs at all, and I'd be leery of using it for much in the way of arrays other than string literals (since the string literals should avoid the memory allocations that other array literals get). It's fine most other stuff, but for those items, you need to be very wary of it or risk lots of unnecessary GC allocations. - Jonathan M Davis
Re: Assigning global and static associative arrays
Using enum can be very useful, but I wouldn't use it for AAs at all, and I'd be leery of using it for much in the way of arrays other than string literals (since the string literals should avoid the memory allocations that other array literals get). It's fine most other stuff, but for those items, you need to be very wary of it or risk lots of unnecessary GC allocations. I guess the rule to follow is not to use enum for anything with lots of allocation. Why do string literals have less memory allocations?
Re: Assigning global and static associative arrays
On Saturday, September 01, 2012 11:07:34 Philippe Sigaud wrote:
Using enum can be very useful, but I wouldn't use it for AAs at all, and
I'd be leery of using it for much in the way of arrays other than string
literals (since the string literals should avoid the memory allocations
that other array literals get). It's fine most other stuff, but for those
items, you need to be very wary of it or risk lots of unnecessary GC
allocations.
I guess the rule to follow is not to use enum for anything with lots
of allocation.
Why do string literals have less memory allocations?
If I understand correctly, you end up with all uses of the same string literal
being the exact same chunk of memory, but I could be wrong. Let's see... Well,
this program seems to print the same thing 4 times
import std.stdio;
enum a = hello;
enum b = hello;
void main()
{
writeln(a.ptr);
writeln(a.ptr);
writeln(b.ptr);
writeln(b.ptr);
}
so it looks like not only do all instances of the same string enum use the
same memory, but another enum with the same string literal shares it as well.
So, only one is allocated. And on Linux at least, as I understand it, the
string literals go in ROM. But it may be that the above code functions
differently in Windows, since it _doesn't_ put string literals in ROM.
Regardless, you can contrast that with this
import std.stdio;
enum a = [1, 2, 3, 4, 5];
enum b = [1, 2, 3, 4, 5];
void main()
{
writeln(a.ptr);
writeln(a.ptr);
writeln(b.ptr);
writeln(b.ptr);
}
which prints 4 different addresses. So clearly, each use of an enum which is an
array literal allocates a new array.
- Jonathan M Davis
Re: Assigning global and static associative arrays
Those still have different addresses when made immutable too, is this something that could be optimized for all immutable enums to behave like strings or are there reasons to keep them unique at each instance?
Re: Assigning global and static associative arrays
On Saturday, September 01, 2012 16:14:29 ixid wrote: Those still have different addresses when made immutable too, is this something that could be optimized for all immutable enums to behave like strings or are there reasons to keep them unique at each instance? The compiler has to got to extra work to make string literals use the same memory like that. While it _could_ do the same with other array literals of basic types, it's not worth it, because they're just not used enough, whereas string literals get used all over the place. - Jonathan M Davis
Re: Assigning global and static associative arrays
On Saturday, September 01, 2012 11:10:17 Jonathan M Davis wrote: On Saturday, September 01, 2012 16:14:29 ixid wrote: Those still have different addresses when made immutable too, is this something that could be optimized for all immutable enums to behave like strings or are there reasons to keep them unique at each instance? The compiler has to got to extra work to make string literals use the same memory like that. While it _could_ do the same with other array literals of basic types, it's not worth it, because they're just not used enough, whereas string literals get used all over the place. The other thing to remember is that for the whole sharing address thing to work, the elements in the array literal must be immutable, and string literals are the only ones which are that way by default. Other array literals are only going to have immutable elements if they're used to directly initialize an array with immutable elements. So, it's probably relatively rare for other array literals to even be used in a way, and unless they're used that way _and_ the same literal is used in multiple places, then you can't make them share the same memory. So, there's not much point in trying to make them share memory. String literals, on the other hand, are immutable and frequently duplicated, so there's some value in sharing them. But even with them, it wouldn't surprise me at all if they only share memory within a single module, since separate compilation could screw with their ability to share. I don't know though. - Jonathan M Davis
Assigning global and static associative arrays
Why does this not work:
int[string] dayNumbers =
[ Monday : 0, Tuesday : 1, Wednesday : 2,
Thursday : 3, Friday : 4, Saturday : 5,
Sunday : 6 ];
void main() {
//Stuff
}
With the error 'non-constant expression'? This error also seems
to prevent me making static associative arrays in functions
elegantly.
Re: Assigning global and static associative arrays
On 8/31/2012 11:38 PM, ixid wrote:
Why does this not work:
int[string] dayNumbers =
[ Monday : 0, Tuesday : 1, Wednesday : 2,
Thursday : 3, Friday : 4, Saturday : 5,
Sunday : 6 ];
void main() {
//Stuff
}
With the error 'non-constant expression'? This error also seems to
prevent me making static associative arrays in functions elegantly.
Non-const variables cannot be initialized at module-scope. If you don't
need to modify the aa, declaring it immutable should allow the
initialization to work. Otherwise, you can initialize it using a static
module constructor.
int[string] dayNumbers;
static this()
{
dayNumbers = ...;
}
void main() {...}
Re: Assigning global and static associative arrays
On 9/1/2012 1:31 AM, Mike Parker wrote:
On 8/31/2012 11:38 PM, ixid wrote:
Why does this not work:
int[string] dayNumbers =
[ Monday : 0, Tuesday : 1, Wednesday : 2,
Thursday : 3, Friday : 4, Saturday : 5,
Sunday : 6 ];
void main() {
//Stuff
}
With the error 'non-constant expression'? This error also seems to
prevent me making static associative arrays in functions elegantly.
Non-const variables cannot be initialized at module-scope. If you don't
need to modify the aa, declaring it immutable should allow the
initialization to work. Otherwise, you can initialize it using a static
module constructor.
int[string] dayNumbers;
static this()
{
dayNumbers = ...;
}
void main() {...}
Nevermind. I spoke too soon. It's the literal [Monday : 0, ...] that's
not constant. You'll have to use the module constructor.
Re: Assigning global and static associative arrays
On Friday, August 31, 2012 16:38:13 ixid wrote:
Why does this not work:
int[string] dayNumbers =
[ Monday : 0, Tuesday : 1, Wednesday : 2,
Thursday : 3, Friday : 4, Saturday : 5,
Sunday : 6 ];
void main() {
//Stuff
}
With the error 'non-constant expression'? This error also seems
to prevent me making static associative arrays in functions
elegantly.
Because they can't be declared at runtime like that. For that to work, they'd
have to be constructable at compile time and then have the structure persist
to runtime, and with all of the pointers and whatnot in an AA, that's far from
trivial. The same occurs with classes. While you can use some of them in CTFE,
you can't use them to directly initialize any variables whose values need to
be known at compile time and persist until runtime.
You have to use static constructor to initialize the variable at runtime.
static this()
{
//initialization code here...
}
- Jonathan M Davis
Re: Assigning global and static associative arrays
Thank you, that certainly makes sense.
Re: Assigning global and static associative arrays
On Fri, Aug 31, 2012 at 7:04 PM, ixid nuacco...@gmail.com wrote:
Thank you, that certainly makes sense.
If you're certain you won't need to modify it, you can make it a
compile-time constant:
enum int[string] dayNumbers =
[ Monday : 0, Tuesday : 1, Wednesday : 2,
Thursday : 3, Friday : 4, Saturday : 5,
Sunday : 6 ];
void main() {
//Stuff
}
Re: Assigning global and static associative arrays
On Friday, August 31, 2012 20:24:27 Philippe Sigaud wrote:
On Fri, Aug 31, 2012 at 7:04 PM, ixid nuacco...@gmail.com wrote:
Thank you, that certainly makes sense.
If you're certain you won't need to modify it, you can make it a
compile-time constant:
enum int[string] dayNumbers =
[ Monday : 0, Tuesday : 1, Wednesday : 2,
Thursday : 3, Friday : 4, Saturday : 5,
Sunday : 6 ];
void main() {
//Stuff
}
Except that that allocates a new AA every time that you use dayNumbers. So,
that's probably a bad idea.
- Jonathan M Davis
Re: Assigning global and static associative arrays
Philippe suggested enum allowing this: enum dayNumbers = [ Monday : 0, Tuesday : 1, Wednesday : 2, Thursday : 3, Friday : 4, Saturday : 5, Sunday : 6 ]; Why does this seem to avoid pointer issues? Is it creating a compile-time associated array or run-time?
Re: Assigning global and static associative arrays
On Fri, Aug 31, 2012 at 9:56 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: Except that that allocates a new AA every time that you use dayNumbers. So, that's probably a bad idea. Oh! I keep forgetting that enums are replaced by their values. Then I think I know where some problem I had came from. Damn, just tested using a static this and my code runs 40% faster! Benchmarking time, thanks Jonathan.
Re: Assigning global and static associative arrays
On Fri, Aug 31, 2012 at 10:34 PM, ixid nuacco...@gmail.com wrote: Philippe suggested enum allowing this: enum dayNumbers = [ Monday : 0, Tuesday : 1, Wednesday : 2, Thursday : 3, Friday : 4, Saturday : 5, Sunday : 6 ]; Why does this seem to avoid pointer issues? Is it creating a compile-time associated array or run-time? It defines an AA literal that's used directly in lieu of dayNumbers every time dayNumbers appears in your code, I think.
Re: Assigning global and static associative arrays
Yep, I am aware of that, for my use it happens to be perfect but I understand that having a bunch of copies all over the place wouldn't be smart.
Re: Assigning global and static associative arrays
Hmm, you mean if you call the same function it creates a new copy every time? I misunderstood you to mean it creates it once at each site in the code it's called.
Re: Assigning global and static associative arrays
On Saturday, September 01, 2012 00:12:06 ixid wrote: Hmm, you mean if you call the same function it creates a new copy every time? I misunderstood you to mean it creates it once at each site in the code it's called. enum values are basically copy-pasted everywhere that they're used. So, if you have something like enum arr = [1, 2, 3, 4 5]; auto a = arr; auto b = arr; auto c = arr; it's effectively identical to auto a = [1, 2, 3, 4, 5]; auto b = [1, 2, 3, 4, 5]; auto c = [1, 2, 3, 4, 5]; as opposed to actual variable such as auto arr = [1, 2, 3, 4, 5]; auto a = arr; auto b = arr; auto c = arr; In this case, each variable is actually a slice of the same array rather than duplicating the value. Using an enum is particularly bad for an AA, since it's not exactly a simple data type, and there's definitely some cost to constructing them. - Jonathan M Davis
Re: Static Associative Arrays
Jonathan M Davis , dans le message (digitalmars.D.learn:34332), a écrit : On Sunday, April 08, 2012 01:24:02 Caligo wrote: On Sat, Apr 7, 2012 at 11:01 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: What do you mean my static associative arrays? Are you asking why you can't initialize a static variable which is an AA at compile time? e.g. - Jonathan M Davis The same way I can create a static array: int[4] = [1, 3, 4, 8]; // has value semantics and dynamic arrays: int[] = [1, 4, 2, 4]; // has reference semantics I want an associative array that has value semantics and it's size doesn't change, just like static arrays. Associative arrays are always on the heap. They always have reference semantics. It would be very expensive to have an AA with value semantics. They contains pointers all over the place. It would be equivalent to calling dup on them every time that you pass them to anything. And trying to put one on the stack would get very messy because all of the pointers involved. AAs are _completely_ different from dynamic and static arrays. Aside from the fact that they both have array in their name and both allow indexing of a sort, there's really no relation between them at all. Yet, one could imagine an associative array with a fixed size and fixed indexes, with no pointers. We could define a simple structure with a static array to store the data and methods to find the element associated with a key. But we could not use this structure in place of a classic associative array (for that we would need pointers obviously).
Re: Static Associative Arrays
On Sat, Apr 7, 2012 at 11:01 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: What do you mean my static associative arrays? Are you asking why you can't initialize a static variable which is an AA at compile time? e.g. - Jonathan M Davis The same way I can create a static array: int[4] = [1, 3, 4, 8]; // has value semantics and dynamic arrays: int[] = [1, 4, 2, 4]; // has reference semantics I want an associative array that has value semantics and it's size doesn't change, just like static arrays. P.S. another point. I was always under the impression that static arrays are allocated on the stack whereas dynamic arrays are allocated on the heap and the GC cleans them up. After today, I'm not so sure if this is true. Are static arrays allocated on the stack? if so, that would be another reason to want to have static associative arrays.
Re: Static Associative Arrays
On Sunday, April 08, 2012 01:24:02 Caligo wrote: On Sat, Apr 7, 2012 at 11:01 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: What do you mean my static associative arrays? Are you asking why you can't initialize a static variable which is an AA at compile time? e.g. - Jonathan M Davis The same way I can create a static array: int[4] = [1, 3, 4, 8]; // has value semantics and dynamic arrays: int[] = [1, 4, 2, 4]; // has reference semantics I want an associative array that has value semantics and it's size doesn't change, just like static arrays. Associative arrays are always on the heap. They always have reference semantics. It would be very expensive to have an AA with value semantics. They contains pointers all over the place. It would be equivalent to calling dup on them every time that you pass them to anything. And trying to put one on the stack would get very messy because all of the pointers involved. AAs are _completely_ different from dynamic and static arrays. Aside from the fact that they both have array in their name and both allow indexing of a sort, there's really no relation between them at all. P.S. another point. I was always under the impression that static arrays are allocated on the stack whereas dynamic arrays are allocated on the heap and the GC cleans them up. After today, I'm not so sure if this is true. Are static arrays allocated on the stack? if so, that would be another reason to want to have static associative arrays. Yes. static arrays go on the stack and are value types, whereas dynamic arrays go no the heap and are reference types which are managed by the GC. If you want more details on how arrays work, read this: http://dlang.org/d-array-article.html - Jonathan M davis
Static Associative Arrays
I'm not questioning the design, but I would like to know the reason: given the fact that associative arrays are built into the language, why don't we have static associative arrays?
Re: Static Associative Arrays
On Saturday, April 07, 2012 18:45:40 Caligo wrote: I'm not questioning the design, but I would like to know the reason: given the fact that associative arrays are built into the language, why don't we have static associative arrays? What do you mean my static associative arrays? Are you asking why you can't initialize a static variable which is an AA at compile time? e.g. static int[string] aa = [hello ; 7, world, 13]; //error That's because that would involve allocating memory at compile time which would then somehow have to be around at runtime. And that doesn't work right now. Classes have the same problem. You have to initialize them at runtime. I believe that dynamic arrays are the only ones that work with that right now, and that's because it's a difficult problem. Obviously, none of the memory allocated at compile time can persist to runtime, so there is no simple solution. At some point, it will probably be possible, but not right now. You _should_ be able to use them in CTFE, but they can't be assigned to anything that will persist until runtime. The solution is to use static constructors to initialize AAs and class references at runtime. Now, if you're talking about something else, I have no idea what you mean by static AAs. - Jonathan M Davis
Re: Static Associative Arrays
On Sat, Apr 07, 2012 at 09:01:40PM -0700, Jonathan M Davis wrote: On Saturday, April 07, 2012 18:45:40 Caligo wrote: I'm not questioning the design, but I would like to know the reason: given the fact that associative arrays are built into the language, why don't we have static associative arrays? What do you mean my static associative arrays? Are you asking why you can't initialize a static variable which is an AA at compile time? e.g. static int[string] aa = [hello ; 7, world, 13]; //error That's because that would involve allocating memory at compile time which would then somehow have to be around at runtime. And that doesn't work right now. Classes have the same problem. You have to initialize them at runtime. I believe that dynamic arrays are the only ones that work with that right now, and that's because it's a difficult problem. Obviously, none of the memory allocated at compile time can persist to runtime, so there is no simple solution. At some point, it will probably be possible, but not right now. You _should_ be able to use them in CTFE, but they can't be assigned to anything that will persist until runtime. [...] I've thought of a way to use CTFE to generate static AA's, but I appear to be running into a CTFE bug or quirk that causes it not to work. Basically, the idea is to use CTFE to generate a series of static struct declarations (each struct being an AA slot) with precomputed references between them. Obviously, the resulting AA must be immutable, but in principle this should allow implementation of static AA's. But I'm running into a weird issue where using a string mixin with a string constant containing the struct definitions work correctly, but if the string is generated piecewise by CTFE code then all references turn into null. I'll have to massage the code a bit into a minimal test case and open an issue on the bugtracker. T -- Windows 95 was a joke, and Windows 98 was the punchline.
