Re: trouble with associative Arrays
On Saturday, 20 January 2024 at 15:16:00 UTC, atzensepp wrote:
The section looks now simpler although I guess that there are
more appropriate mechanisms available (csvreader):
string [] orgids[string];
foreach (line; range)
{
if (!line.empty)
{
auto row = line.split(";");
string word = row[1];
if(word.length>0 && word[0] == '\"')
word= word[1 .. $-1];
orgids[word]=row;
i++;
}
}
Maybe a bit more readable:
```d
import std.string : strip, split;
string [] orgids[string];
foreach (line; range)
{
if (line.empty) continue;
auto row = line.split(";");
auto word = row[1].strip("\"");
orgids[word] = row;
i++;
}
```
Re: trouble with associative Arrays
Thank you T for your hint. This worked perfectly
On Saturday, 20 January 2024 at 14:44:49 UTC, H. S. Teoh wrote:
Because .byLine reuses its line buffer. You want .byLineCopy
instead.
The section looks now simpler although I guess that there are
more appropriate mechanisms available (csvreader):
string [] orgids[string];
foreach (line; range)
{
if (!line.empty)
{
auto row = line.split(";");
string word = row[1];
if(word.length>0 && word[0] == '\"')
word= word[1 .. $-1];
orgids[word]=row;
i++;
}
}
Re: trouble with associative Arrays
On Sat, Jan 20, 2024 at 02:33:24PM +, atzensepp via Digitalmars-d-learn wrote: > Hello, > > I am new with D and want to convert a c program for a csv file manipulation > with exhaustive dynamic memory mechanics to D . > > When reading a CSV-file line by line I would like to create an associative > array to get the row values by the value in the second column. > Although I save the rows in an array (to get different pointers to the > values) the program below always gives the last row. [...] Because .byLine reuses its line buffer. You want .byLineCopy instead. T -- Everybody talks about it, but nobody does anything about it! -- Mark Twain
trouble with associative Arrays
Hello,
I am new with D and want to convert a c program for a csv file
manipulation with exhaustive dynamic memory mechanics to D .
When reading a CSV-file line by line I would like to create an
associative array to get the row values by the value in the
second column.
Although I save the rows in an array (to get different pointers
to the values) the program below always gives the last row.
I am sure someone could help.
thanks
void main( string args[])
{
auto file = File("transpatch2_orig.csv"); // Open for
reading
auto range = file.byLine();
// Print first three lines
foreach (line; range.take(1))
writeln(line);
auto i=0;
char [][] [string] orgids;
char [][][] rows;
foreach (line; range)
{
if (!line.empty)
{
// auto row = line.split(";");
rows ~= (line.split(";"));
string word = rows[$ - 1][1].idup;
if(word.length>0 && word[0] == '\"')
word= word[1 .. $-1];
orgids[word.idup]=rows[$ - 1];
i++;
}
}
writeln( orgids.length);
writeln( args[1],orgids[args[1]]);
writeln( args[2],orgids[args[2]]);
writeln("Lines: ",i);
}
Re: Comparison of multidimensional associative arrays
On Wednesday, 8 February 2023 at 19:04:15 UTC, Alexander Zhirov
wrote:
[...]
I would write a data structure and use struct members to reason
about things, but that's probably just preference.
```
import std;
struct DatabaseEntry
{
int id = -1;
string deleted;
string name;
this(string[string] aa)
{
this.id = aa["id"].to!int;
this.deleted = aa["deleted"];
this.name = aa["name"];
}
auto opEquals(typeof(this) that) const
{
return
(this.id == that.id) &&
(this.deleted == that.deleted) &&
(this.name == that.name);
}
}
auto buildSortedEntryRange(string[string][int] aa)
{
return aa
.byValue
.map!(entry => DatabaseEntry(entry))
.array
.sort!((a,b) => a.id < b.id);
}
void main()
{
auto arrayA =
[
4:["id":"4", "deleted":"f", "name":"6.2"],
3:["id":"3", "deleted":"f", "name":"5.6_hwlister"],
2:["id":"2", "deleted":"t", "name":"6.2"],
1:["id":"1", "deleted":"f", "name":"5.6"]
];
auto arrayB =
[
6:["id":"6", "deleted":"f", "name":"6.2_test"],
5:["id":"5", "deleted":"f", "name":"5.6_test"],
4:["id":"4", "deleted":"f", "name":"6.2_hwlister"],
3:["id":"3", "deleted":"f", "name":"5.6_hwlister"],
2:["id":"2", "deleted":"f", "name":"6.2"],
1:["id":"1", "deleted":"f", "name":"5.6"]
];
auto entriesFromA = buildSortedEntryRange(arrayA);
auto entriesFromB = buildSortedEntryRange(arrayB);
auto range = zip(StoppingPolicy.longest, entriesFromA,
entriesFromB);
foreach (entryA, entryB; range)
{
if (entryA != entryB)
{
writeln(entryB);
// ...
}
}
}
```
Output is:
```
DatabaseEntry(2, "f", "6.2")
DatabaseEntry(4, "f", "6.2_hwlister")
DatabaseEntry(5, "f", "5.6_test")
DatabaseEntry(6, "f", "6.2_test")
```
Re: Comparison of multidimensional associative arrays
On Wednesday, 8 February 2023 at 19:32:22 UTC, Ali Çehreli wrote: This should do it: [...] Yes, it works! I'll try it tomorrow on a large array of data. Thank you very much! This turns out to be a simple loop with a comparison of the existence of a key (whether it is included in an array or not) and a comparison of an existing array as a value with another array by this key. I wonder if can add this to lambda? In one line.
Re: Comparison of multidimensional associative arrays
On 2/8/23 11:04, Alexander Zhirov wrote:
> That is, the result is arrays of table B that are missing OR not equal
> to arrays in table A.
This should do it:
alias MyType = string[string][int];
// 'a' is subtracted from 'b'
MyType difference(MyType b, MyType a) {
MyType result;
foreach (kv; b.byKeyValue) {
auto existing = kv.key in a;
if (!existing || (*existing != kv.value)) {
result[kv.key] = kv.value;
}
}
return result;
}
void main() {
auto a = [
4:["id":"4", "deleted":"f", "name":"6.2"],
3:["id":"3", "deleted":"f", "name":"5.6_hwlister"],
2:["id":"2", "deleted":"t", "name":"6.2"],
1:["id":"1", "deleted":"f", "name":"5.6"]
];
auto b = [
6:["id":"6", "deleted":"f", "name":"6.2_test"],
5:["id":"5", "deleted":"f", "name":"5.6_test"],
4:["id":"4", "deleted":"f", "name":"6.2_hwlister"],
3:["id":"3", "deleted":"f", "name":"5.6_hwlister"],
2:["id":"2", "deleted":"f", "name":"6.2"],
1:["id":"1", "deleted":"f", "name":"5.6"]
];
auto expected = [
6:["id":"6", "deleted":"f", "name":"6.2_test"],
5:["id":"5", "deleted":"f", "name":"5.6_test"],
4:["id":"4", "deleted":"f", "name":"6.2_hwlister"],
2:["id":"2", "deleted":"f", "name":"6.2"]
];
auto diff = difference(b, a);
import std.format;
assert(diff == expected, format!"UNEXPECTED: %s"(diff));
}
Ali
Re: Comparison of multidimensional associative arrays
On Wednesday, 8 February 2023 at 18:57:00 UTC, Anonymouse wrote: Can you explain how you determine how/if two entries are different? I apologize. I have not written, in fact, what I need to get. Array `A` ```d [ 4:["id":"4", "deleted":"f", "name":"6.2"], 3:["id":"3", "deleted":"f", "name":"5.6_hwlister"], 2:["id":"2", "deleted":"t", "name":"6.2"], 1:["id":"1", "deleted":"f", "name":"5.6"] ] ``` Array `B` ```d [ 6:["id":"6", "deleted":"f", "name":"6.2_test"], 5:["id":"5", "deleted":"f", "name":"5.6_test"], 4:["id":"4", "deleted":"f", "name":"6.2_hwlister"], 3:["id":"3", "deleted":"f", "name":"5.6_hwlister"], 2:["id":"2", "deleted":"f", "name":"6.2"], 1:["id":"1", "deleted":"f", "name":"5.6"] ] ``` Diff: ```d [ 6:["id":"6", "deleted":"f", "name":"6.2_test"], 5:["id":"5", "deleted":"f", "name":"5.6_test"], 4:["id":"4", "deleted":"f", "name":"6.2_hwlister"], 2:["id":"2", "deleted":"f", "name":"6.2"] ] ``` That is, the result is arrays of table B that are missing OR not equal to arrays in table A.
Re: Comparison of multidimensional associative arrays
On Wednesday, 8 February 2023 at 17:55:03 UTC, Alexander Zhirov wrote: Not an easy task for me, maybe you can advise your compact solution. There are two associative arrays of type `string[string][int]`. It is necessary to find the differences and return them when comparing: Can you explain how you determine how/if two entries are different?
Re: Comparison of multidimensional associative arrays
On Wednesday, 8 February 2023 at 18:08:40 UTC, Ali Çehreli wrote: Just because this sounds complicated, I hope the data structure can be designed differently to be more friendly to this operation. (?) Ali This is the result of an SQL query. Roughly speaking, I need to compare the result of queries from two different databases and find different ones. The tables are the same, the only difference is in the content.
Re: Comparison of multidimensional associative arrays
On 2/8/23 09:55, Alexander Zhirov wrote: > the differences Is it considered a difference if a key exists but the value is different? Or is that an error case if you encounter that? > return them when comparing: The representation seems difficult as well. When given this: > 6:["id":"6", "deleted":"f", "name":"6.2_test"], and that: > 6:["id":"6", "deleted":"f" ], Should the difference be 6:["name":"6.2_test"] ? And how to represent "missing from inner AA"? Just because this sounds complicated, I hope the data structure can be designed differently to be more friendly to this operation. (?) Ali
Comparison of multidimensional associative arrays
Not an easy task for me, maybe you can advise your compact solution. There are two associative arrays of type `string[string][int]`. It is necessary to find the differences and return them when comparing: ```d [ 6:["id":"6", "deleted":"f", "name":"6.2_test"], 5:["id":"5", "deleted":"f", "name":"5.6_test"], 4:["id":"4", "deleted":"f", "name":"6.2_hwlister"], 3:["id":"3", "deleted":"f", "name":"5.6_hwlister"], 2:["id":"2", "deleted":"f", "name":"6.2"], 1:["id":"1", "deleted":"f", "name":"5.6"] ] [ 4:["id":"4", "deleted":"f", "name":"6.2_hwlister"], 3:["id":"3", "deleted":"f", "name":"5.6_hwlister"], 2:["id":"2", "deleted":"f", "name":"6.2"], 1:["id":"1", "deleted":"f", "name":"5.6"] ] ```
Re: Why dtor are not executed when removing a struct from associative arrays?
On 9/21/21 2:06 AM, Tejas wrote: On Monday, 20 September 2021 at 18:13:53 UTC, Steven Schveighoffer wrote: On 9/20/21 10:22 AM, Tejas wrote: In case you still want to delete stuff deterministically despite what Steve said, I suggest you make your `struct` a reference and use `core.memory.__delete`(not recommended to use this carelessly, btw) Do not call `__delete` here, use `destroy`. `__delete` will attempt to deallocate the block, which likely will fail since the key comes before the value, and GC.free on an interior pointer (I think) fails. But if it succeeded, it would not be good. This leaves a dangling pointer inside the AA. Rehashing the AA likely would result in a memory corruption. If you use destroy, the destructor will be called by the GC as well, but a struct should properly handle destroying the .init value. It doesn't succeed when object is stack allocated, otherwise it works; that's why I suggested changing `S` to `S*`. Oh! I missed that subtle change. I thought you were deleting a pointer to S that lives in a `S[int]`. I still recommend against this because you can easily get a dangling pointer that way. Aside from that, if you are careful enough not to store a retrieved S* from the aa, you could do that. It will incur one extra allocation per element, which is not ideal. Using destroy should be the most effective and safest mechanism. -Steve
Re: Why dtor are not executed when removing a struct from associative arrays?
On Monday, 20 September 2021 at 18:13:53 UTC, Steven
Schveighoffer wrote:
On 9/20/21 10:22 AM, Tejas wrote:
In case you still want to delete stuff deterministically
despite what Steve said, I suggest you make your `struct` a
reference and use `core.memory.__delete`(not recommended to
use this carelessly, btw)
Do not call `__delete` here, use `destroy`. `__delete` will
attempt to deallocate the block, which likely will fail since
the key comes before the value, and GC.free on an interior
pointer (I think) fails.
But if it succeeded, it would not be good. This leaves a
dangling pointer inside the AA. Rehashing the AA likely would
result in a memory corruption.
If you use destroy, the destructor will be called by the GC as
well, but a struct should properly handle destroying the .init
value.
-Steve
It doesn't succeed when object is stack allocated, otherwise it
works; that's why I suggested changing `S` to `S*`.
As for why I didn't use `destroy`, I assumed OP wanted to free
the memory. If they just want the destructor to run, then
`destroy` is better(and safer).
(Also, `hashOf` seems to work just fine even after using
`__delete`, not that it means using `__delete` should ever be
recommended over using `destroy`)
```d
import std.stdio;
import core.memory:__delete;
struct S
{
int a;
this(int param){
a = param;
}
~this()
{
writeln("S(",a,") is being destructed");
}
}
void absoluteRemove(AA, keyType)(AA assArray, keyType key){
//assArray.remove(key);
__delete(assArray[key]);
}
void main()
{
S*[int] aa;
aa[1] = new S(5);
absoluteRemove(aa, 1);
writeln(aa.hashOf);
aa[2] = new S(10);
writeln(aa.hashOf);
absoluteRemove(aa, 2);
writeln(aa.hashOf);
writeln("Why no dtor call on remove?");
}
Output:
S(5) is being destructed
2451737883
5104465521
S(10) is being destructed
5378492086
Why no dtor call on remove?
```
Re: Why dtor are not executed when removing a struct from associative arrays?
On 9/20/21 10:22 AM, Tejas wrote: In case you still want to delete stuff deterministically despite what Steve said, I suggest you make your `struct` a reference and use `core.memory.__delete`(not recommended to use this carelessly, btw) Do not call `__delete` here, use `destroy`. `__delete` will attempt to deallocate the block, which likely will fail since the key comes before the value, and GC.free on an interior pointer (I think) fails. But if it succeeded, it would not be good. This leaves a dangling pointer inside the AA. Rehashing the AA likely would result in a memory corruption. If you use destroy, the destructor will be called by the GC as well, but a struct should properly handle destroying the .init value. -Steve
Re: Why dtor are not executed when removing a struct from associative arrays?
On 9/20/21 5:23 AM, Learner wrote:
I was expecting S instance dtor called
S is being destructed
If you are sure the element can be destroyed, you can call destroy():
import std.stdio;
enum someSpecialInitValue = 777;
struct S
{
int i = someSpecialInitValue;
this(int i)
{
this.i = i;
}
~this()
{
writeln("S with ", i, " is being destructed");
}
}
void destroyAndRemove(AA, Key)(AA aa, Key key) {
auto found = key in aa;
if (found) {
destroy(*found);
aa.remove(key);
}
}
void main()
{
S[int] aa;
aa[1] = S(1);
aa.destroyAndRemove(1);
writeln("Actually, 2 dtor calls! :)");
}
destroy() puts the object to its initial state, which means, it gets
destroyed again. That's why there are two destructor calls below for the
same object. I used a special value to demonstrate the second destructor
is called on the init state:
S with 1 is being destructed
Actually, 2 dtor calls! :)
S with 777 is being destructed
Ali
Re: Why dtor are not executed when removing a struct from associative arrays?
On Monday, 20 September 2021 at 14:03:09 UTC, Tejas wrote:
On Monday, 20 September 2021 at 13:48:01 UTC, Tejas wrote:
On Monday, 20 September 2021 at 12:23:00 UTC, Learner wrote:
[...]
I think it *is* being called:
```d
import std.stdio;
struct S
{
int a;
this(int param){
a = param;
}
~this()
{
writeln("S(",a,") is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S(5);
aa.remove(1);
aa[1] = S(10);
//writeln("Why no dtor call on remove?");
}
Output:
S(5) is being destructed
S(10) is being destructed
```
If the destructors were being executed based on the ending of
scope, `S(10)` would have been destroyed first.
Oh dear, I made a mistake :(
Sorry, shouldn't have commented out the `writeln`
To make up for my mistake:
In case you still want to delete stuff deterministically despite
what Steve said, I suggest you make your `struct` a reference and
use `core.memory.__delete`(not recommended to use this
carelessly, btw)
```d
import std.stdio;
import core.memory:__delete;
struct S
{
int a;
this(int param){
a = param;
}
~this()
{
writeln("S(",a,") is being destructed");
}
}
void absoluteRemove(AA, keyType)(AA assArray, keyType key){
__delete(assArray[key]);
}
void main()
{
S*[int] aa;
aa[1] = new S(5);
absoluteRemove(aa, 1);
aa[2] = new S(10);
absoluteRemove(aa, 2);
writeln("Why no dtor call on remove?");
}
Output:
S(5) is being destructed
S(10) is being destructed
Why no dtor call on remove?
```
Because `__delete` cannot work on stack allocated objects
Re: Why dtor are not executed when removing a struct from associative arrays?
On Monday, 20 September 2021 at 13:48:01 UTC, Tejas wrote:
On Monday, 20 September 2021 at 12:23:00 UTC, Learner wrote:
[...]
I think it *is* being called:
```d
import std.stdio;
struct S
{
int a;
this(int param){
a = param;
}
~this()
{
writeln("S(",a,") is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S(5);
aa.remove(1);
aa[1] = S(10);
//writeln("Why no dtor call on remove?");
}
Output:
S(5) is being destructed
S(10) is being destructed
```
If the destructors were being executed based on the ending of
scope, `S(10)` would have been destroyed first.
Oh dear, I made a mistake :(
Sorry, shouldn't have commented out the `writeln`
Re: Why dtor are not executed when removing a struct from associative arrays?
On 9/20/21 8:23 AM, Learner wrote:
I was expecting something like going out of scope for that
```d
import std.stdio;
struct S
{
~this()
{
writeln("S is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S();
aa.remove(1);
writeln("Why no dtor call on remove?");
}
```
I was expecting S instance dtor called
S is being destructed
AA values are not destroyed on removal. For a simple reason -- someone
might still be referencing it.
```d
struct S
{
int x;
}
void main()
{
S[int] aa;
aa[1] = S(5);
auto sptr = 1 in aa;
sptr.x = 6;
assert(aa[1].x == 6);
aa.remove(1);
assert(sptr.x == 6;
}
```
-Steve
Re: Why dtor are not executed when removing a struct from associative arrays?
On Monday, 20 September 2021 at 12:23:00 UTC, Learner wrote:
I was expecting something like going out of scope for that
```(D)
import std.stdio;
struct S
{
~this()
{
writeln("S is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S();
aa.remove(1);
writeln("Why no dtor call on remove?");
}
I was expecting S instance dtor called
S is being destructed
```
This looks to me like a bug, as
```d
import core.memory : GC;
GC.collect;
```
immediately after the `.remove` will call the struct's destructor.
I only see https://issues.dlang.org/show_bug.cgi?id=20379 as
related, though.
Here's another workaround:
```d
alias purge = (kv, k) { kv[k] = typeof(kv[k]).init; kv.remove(k);
};
```
with the same caveat of the `.init` structs also getting
destructed later, you can use that in place of `.remove`.
Re: Why dtor are not executed when removing a struct from associative arrays?
On Monday, 20 September 2021 at 12:23:00 UTC, Learner wrote:
I was expecting something like going out of scope for that
```(D)
import std.stdio;
struct S
{
~this()
{
writeln("S is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S();
aa.remove(1);
writeln("Why no dtor call on remove?");
}
I was expecting S instance dtor called
S is being destructed
```
I think it *is* being called:
```d
import std.stdio;
struct S
{
int a;
this(int param){
a = param;
}
~this()
{
writeln("S(",a,") is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S(5);
aa.remove(1);
aa[1] = S(10);
//writeln("Why no dtor call on remove?");
}
Output:
S(5) is being destructed
S(10) is being destructed
```
If the destructors were being executed based on the ending of
scope, `S(10)` would have been destroyed first.
Why dtor are not executed when removing a struct from associative arrays?
I was expecting something like going out of scope for that
```(D)
import std.stdio;
struct S
{
~this()
{
writeln("S is being destructed");
}
}
void main()
{
S[int] aa;
aa[1] = S();
aa.remove(1);
writeln("Why no dtor call on remove?");
}
I was expecting S instance dtor called
S is being destructed
```
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 08:31:16 UTC, Ali Çehreli wrote:
Two options for byKey and byKeyValue:
import std;
void main() {
auto aa = [ "WS2" : 42, "WS3" : 43 ];
string strComputerIDunwanted = "WS2";
foreach (key; aa.byKey.filter!(k => k !=
strComputerIDunwanted)) {
writeln(key, ' ', aa[key]);
}
// 't' is the tuple of key and value
foreach (t; aa.byKeyValue.filter!(t => t.key !=
strComputerIDunwanted)) {
writeln(t.key, ' ', t.value);
}
}
Like 'em both, very flexible indeed, I think there are a couple
of places I could implement this. Thanks for your good advice as
usual Ali :) !
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 15:32:09 UTC, wjoe wrote:
something like this ?
``` D
import std.array;
import std.algorithm;
udtComputers.byPair
.filter!(p => p.key != strComputerIDunwanted)
.each!( (p) { /* foreach body */ } );
```
This seems really interesting :)
Almost three weeks with D and I still must learn a lot of things,
there's a lot to :) !
Thanks for your tip !
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 22:08:56 UTC, Steven Schveighoffer
wrote:
It's actually visually shorter than doing the filter.
Indeed; in a few very-specific situations I usually write code
like this since it allows me to concentrate on the task at hand
and not on the details to access the needed data, but for the
clarity of the example I reformatted my code :)
Same when I code RO properties for classes, I usually do:
```d
private pxxxWhatever1; public whatever() @property { return
pxxxWhatever1; }
private pxxxWhatever2; public whatever() @property { return
pxxxWhatever2; }
private pxxxWhatever3; public whatever() @property { return
pxxxWhatever3; }
```
It is far easier to read large chunk of similar constructs
alongside than one below the other, at least, to me.
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 21:00:42 UTC, Elronnd wrote:
Here's how I would do it:
foreach (k, v; coll) {
if (k == unwanted) continue;
...
}
You still have an if, but the actual loop body doesn't have to
be nested, so it's easy to follow the control flow.
almost the same
Re: how to filter associative arrays with foreach ?
On 6/21/21 5:00 PM, Elronnd wrote:
On Monday, 21 June 2021 at 03:59:10 UTC, someone wrote:
Is there a way to filter the collection at the foreach-level to avoid
the inner if ?
Here's how I would do it:
foreach (k, v; coll) {
if (k == unwanted) continue;
...
}
You still have an if, but the actual loop body doesn't have to be
nested, so it's easy to follow the control flow.
Alternatively (if you're ok with this sort of thing):
foreach (k, v; coll) if (k != unwanted) {
}
It's actually visually shorter than doing the filter.
And the benefit to using the if statement over the filter wrapper is
that the compiler has to do a lot less work.
-Steve
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 03:59:10 UTC, someone wrote:
Is there a way to filter the collection at the foreach-level to
avoid the inner if ?
Here's how I would do it:
foreach (k, v; coll) {
if (k == unwanted) continue;
...
}
You still have an if, but the actual loop body doesn't have to be
nested, so it's easy to follow the control flow.
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 03:59:10 UTC, someone wrote:
I often need to iterate through a filtered collection
(associative array) as following:
```d
string strComputerIDunwanted = "WS2"; /// associative array key
to exclude
foreach (strComputerID, udtComputer; udtComputers) { ///
.remove!(a => a == strComputerIDunwanted) ... ?
if (strComputerID != strComputerIDunwanted) {
...
}
}
```
Is there a way to filter the collection at the foreach-level to
avoid the inner if ?
something like this ?
``` D
import std.array;
import std.algorithm;
udtComputers.byPair
.filter!(p => p.key != strComputerIDunwanted)
.each!( (p) { /* foreach body */ } );
```
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 08:35:19 UTC, frame wrote: An associative array is not a range but a struct, so it is extra work to create a range from the AA to apply range functions. You can get a range from it by using something like std.array.byPair() but for this usage you would be better of with your own function or template. Crystal-clear. Thanks !
Re: how to filter associative arrays with foreach ?
On Monday, 21 June 2021 at 03:59:10 UTC, someone wrote:
I often need to iterate through a filtered collection
(associative array) as following:
```d
string strComputerIDunwanted = "WS2"; /// associative array key
to exclude
foreach (strComputerID, udtComputer; udtComputers) { ///
.remove!(a => a == strComputerIDunwanted) ... ?
if (strComputerID != strComputerIDunwanted) {
...
}
}
```
Is there a way to filter the collection at the foreach-level to
avoid the inner if ?
An associative array is not a range but a struct, so it is extra
work to create a range from the AA to apply range functions.
You can get a range from it by using something like
std.array.byPair() but for this usage you would be better of with
your own function or template.
Re: how to filter associative arrays with foreach ?
On 6/20/21 8:59 PM, someone wrote:
I often need to iterate through a filtered collection (associative
array) as following:
```d
string strComputerIDunwanted = "WS2"; /// associative array key to exclude
foreach (strComputerID, udtComputer; udtComputers) { /// ..remove!(a =>
a == strComputerIDunwanted) ... ?
if (strComputerID != strComputerIDunwanted) {
...
}
}
```
Is there a way to filter the collection at the foreach-level to avoid
the inner if ?
Two options for byKey and byKeyValue:
import std;
void main() {
auto aa = [ "WS2" : 42, "WS3" : 43 ];
string strComputerIDunwanted = "WS2";
foreach (key; aa.byKey.filter!(k => k != strComputerIDunwanted)) {
writeln(key, ' ', aa[key]);
}
// 't' is the tuple of key and value
foreach (t; aa.byKeyValue.filter!(t => t.key != strComputerIDunwanted)) {
writeln(t.key, ' ', t.value);
}
}
Ali
how to filter associative arrays with foreach ?
I often need to iterate through a filtered collection
(associative array) as following:
```d
string strComputerIDunwanted = "WS2"; /// associative array key
to exclude
foreach (strComputerID, udtComputer; udtComputers) { ///
.remove!(a => a == strComputerIDunwanted) ... ?
if (strComputerID != strComputerIDunwanted) {
...
}
}
```
Is there a way to filter the collection at the foreach-level to
avoid the inner if ?
Re: Parallel foreach iteration with Associative Arrays
On Saturday, 17 April 2021 at 02:14:50 UTC, Paul Backus wrote:
`parallel` requires a range [1], and an associative array is
not a range. To get a range of an AA's keys and values, you can
use the method `.byKeyValue`:
foreach (pair; parallel(example.byKeyValue)) {
writeln(pair.key, ": ", pair.value);
}
If you're confused about what a "range" is, the short answer is
that it's kind of like an iterator. For the long answer, check
out Andrei Alexandrescu's article "On Iteration" [2], or the
"Ranges" chapter of Ali Çehreli's "Programming in D" [3].
[1]
https://phobos.dpldocs.info/std.parallelism.TaskPool.parallel.2.html
[2] https://www.informit.com/articles/printerfriendly/1407357
[3] http://ddili.org/ders/d.en/ranges.html
That worked! Thanks you!
Re: Parallel foreach iteration with Associative Arrays
On Saturday, 17 April 2021 at 01:57:34 UTC, Kirill wrote:
I'd like to iterate over an associative array and output it's
key and value using parallel from std.parallelism.
But I get an error message: ParallelForeach!(int[string]) error
instantiating.
My code:
auto example = ["apples": 100, "orange": 250, "banana": 175];
foreach(key, value; parallel(example)) { writeln(key, ": ",
value); }
What am I doing wrong?
Thanks in advance.
`parallel` requires a range [1], and an associative array is not
a range. To get a range of an AA's keys and values, you can use
the method `.byKeyValue`:
foreach (pair; parallel(example.byKeyValue)) {
writeln(pair.key, ": ", pair.value);
}
If you're confused about what a "range" is, the short answer is
that it's kind of like an iterator. For the long answer, check
out Andrei Alexandrescu's article "On Iteration" [2], or the
"Ranges" chapter of Ali Çehreli's "Programming in D" [3].
[1]
https://phobos.dpldocs.info/std.parallelism.TaskPool.parallel.2.html
[2] https://www.informit.com/articles/printerfriendly/1407357
[3] http://ddili.org/ders/d.en/ranges.html
Parallel foreach iteration with Associative Arrays
I'd like to iterate over an associative array and output it's key
and value using parallel from std.parallelism.
But I get an error message: ParallelForeach!(int[string]) error
instantiating.
My code:
auto example = ["apples": 100, "orange": 250, "banana": 175];
foreach(key, value; parallel(example)) { writeln(key, ": ",
value); }
What am I doing wrong?
Thanks in advance.
Re: Static initialization of associative arrays
On Thursday, 11 March 2021 at 18:41:08 UTC, Ali Çehreli wrote:
On 3/11/21 10:06 AM, Chris Piker wrote:
>https://dlang.org/spec/hash-map.html#static_initialization
>
> that this feature is not yet implemented.
I use a shared static this() block:
immutable string[int] aa;
shared static this() {
aa = [ 1: "one" ];
}
void main() {
assert(aa.length == 1);
}
And it is possible to build an AA at compile time as the
initial value but it still needs a trivial assigment to the
immutable variable. Assuming that we have the following file at
compile time named 'my_aa':
--- 8< ---
1 one
2 two
--- 8< ---
And remembering that we have to use the -J switch when
compiling (e.g. as -J.), you can parse and build an AA from
that file like this. (Sorry for insisting on the the range
style; it can be done in other ways).
immutable string[int] aa;
shared static this() {
import std.algorithm;
import std.range;
import std.typecons;
import std.conv;
enum compileTimeAA = import("my_aa")
.splitter
.chunks(2)
.map!(a => tuple(a.front.to!int,
a.dropOne.front))
.assocArray;
aa = compileTimeAA;
}
import std.stdio;
void main() {
writeln(aa);
}
Ali
You can however do like this (cheating):
https://run.dlang.io/is/9TSfAB
"The variable myOptions is assigned the result of the literal at
runtime. But because it's immutable, the compiler knows what's in
it. So it can extrapolate back to the literal what it is at
compile time"
We had a discussion in Discord about this last week.
Re: Static initialization of associative arrays
On Thursday, 11 March 2021 at 19:12:34 UTC, H. S. Teoh wrote:
On Thu, Mar 11, 2021 at 06:06:35PM +, Chris Piker via
immutable int[string] aa;
shared static this() {
aa = [ "abc": 123, "def": 456, /* ... */ ];
}
Hi H.S.T
Yes, I'm using static if, but do you know if direct
implementation of immutable associative array assignment (as
given on the language spec page) will be implemented at some
point?
This is not a make-or-break feature that I need, just an
opportunity to gain meta-information. The actual problem is easy
to work around, I'm mostly asking out of curiosity to learn more
about the D ecosystem and how it functions.
Thanks,
Re: Static initialization of associative arrays
On Thursday, 11 March 2021 at 18:41:08 UTC, Ali Çehreli wrote:
On 3/11/21 10:06 AM, Chris Piker wrote:
>https://dlang.org/spec/hash-map.html#static_initialization
>
> that this feature is not yet implemented.
I use a shared static this() block:
immutable string[int] aa;
shared static this() {
aa = [ 1: "one" ];
}
Ali
Hi Ali
Always good to hear from an author. I picked up a copy of your
book the other day... nice work!
I appreciate the tips on compile-time execution (hadn't thought
of that), but do you know how I find out about the implementation
status of a feature in D? I'm aware of the DIP system, but I
though DIPs were just for language changes.
Thanks,
Re: Static initialization of associative arrays
On Thu, Mar 11, 2021 at 06:06:35PM +, Chris Piker via Digitalmars-d-learn
wrote:
[...]
> Today I ran across a situation where an immutable associative array
> would be handy. While perusing the language spec I noticed here:
>
> https://dlang.org/spec/hash-map.html#static_initialization
>
> that this feature is not yet implemented.
[...]
The subsequent section on the linked page gives the solution /
workaround. Just declare your immutable AA without initialization, and
initialize it in a static ctor:
immutable int[string] aa;
shared static this() {
aa = [ "abc": 123, "def": 456, /* ... */ ];
}
T
--
INTEL = Only half of "intelligence".
Re: Static initialization of associative arrays
On 3/11/21 10:06 AM, Chris Piker wrote:
>https://dlang.org/spec/hash-map.html#static_initialization
>
> that this feature is not yet implemented.
I use a shared static this() block:
immutable string[int] aa;
shared static this() {
aa = [ 1: "one" ];
}
void main() {
assert(aa.length == 1);
}
And it is possible to build an AA at compile time as the initial value
but it still needs a trivial assigment to the immutable variable.
Assuming that we have the following file at compile time named 'my_aa':
--- 8< ---
1 one
2 two
--- 8< ---
And remembering that we have to use the -J switch when compiling (e.g.
as -J.), you can parse and build an AA from that file like this. (Sorry
for insisting on the the range style; it can be done in other ways).
immutable string[int] aa;
shared static this() {
import std.algorithm;
import std.range;
import std.typecons;
import std.conv;
enum compileTimeAA = import("my_aa")
.splitter
.chunks(2)
.map!(a => tuple(a.front.to!int,
a.dropOne.front))
.assocArray;
aa = compileTimeAA;
}
import std.stdio;
void main() {
writeln(aa);
}
Ali
Static initialization of associative arrays
Hi D At work I've begun writing programs in D that I would typically write in python. My goal is to get away from split python/C development and just use one language most of the time. Today I ran across a situation where an immutable associative array would be handy. While perusing the language spec I noticed here: https://dlang.org/spec/hash-map.html#static_initialization that this feature is not yet implemented. So where do I go learn about the status of a feature? I'd like to check on the progress of this particular one. For now I'll use a 'static if' construct to ready the array. As someone with a python background it's nice if I can avoid extra lines of code for straight forward ideas, but I understand that switching to faster compiled code doesn't come for free and some extra boiler-plate will be needed. By the way I do like that intended features are documented up front, even if no one's had time to work on them. Thanks,
Re: How to efficiently resolve Associative Arrays not being sorted?
On 6/2/20 3:32 AM, BoQsc wrote:
I want to read a file, put it into an array, make some search and
replace on the content and output the modified text. However Associative
Arrays seem to be unsorted by default. Should I drop the Associative
Arrays and use something else? What are the ways to resolve this
randomness in Associative Arrays?
ReadfileAndCopyContentIntoArray.d
import std.stdio;
int lineNumber = 0;
char[][int] fileInArray;
void main(){
File exampleFile = File("exampleText.txt");
foreach(line; exampleFile.byLine){
lineNumber++;
fileInArray[lineNumber] ~= line;
}
writeln(fileInArray);
}
I have to ask because I don't know if this is similar to your true use
case or not -- but why would you use an AA when you have an ordered list
with keys that go from 0 to N? Why not just use an array?
As others have said, RedBlackTree can be used as an ordered map (though
this requires some definition of a mapping on top of it).
-Steve
Re: How to efficiently resolve Associative Arrays not being sorted?
On 6/2/20 12:32 AM, BoQsc wrote:
> I want to read a file, put it into an array, make some search and
> replace on the content and output the modified text.
How large is the data? If it fits into memory, just read the whole
thing, update it, sort the keys, and then output like this:
import std.stdio;
import std.algorithm;
foreach (key; aa.keys.sort) {
writeln(key, aa[key]);
}
Of course, you can save the sorted array in a local variable as well if
you will use it again:
auto keys = aa.keys.sort;
One great thing about programs that read and write files is that unless
the data is so large that it does not fit into physical memory, you
can't feel the time cost of that sort operation. :)
Ali
Re: How to efficiently resolve Associative Arrays not being sorted?
On Tuesday, 2 June 2020 at 07:32:56 UTC, BoQsc wrote:
I want to read a file, put it into an array, make some search
and replace on the content and output the modified text.
However Associative Arrays seem to be unsorted by default.
Should I drop the Associative Arrays and use something else?
What are the ways to resolve this randomness in Associative
Arrays?
ReadfileAndCopyContentIntoArray.d
import std.stdio;
int lineNumber = 0;
char[][int] fileInArray;
void main(){
File exampleFile = File("exampleText.txt");
foreach(line; exampleFile.byLine){
lineNumber++;
fileInArray[lineNumber] ~= line;
}
writeln(fileInArray);
}
exampleText.txt
The quick brown fox jumps over the lazy dog
Sphinx of black quartz, judge my vow.
How vexingly quick daft zebras jump!
The five boxing wizards jump quickly
Maecenas consectetur risus a lacus sodales iaculis.
Morbi sed tortor sollicitudin, pharetra massa egestas, congue
massa.
Sed sit amet nisi at ligula ultrices posuere quis nec est.
Mauris vel purus viverra, pellentesque elit id, consequat
felis.
The Command Prompt Output
[6:"Morbi sed tortor sollicitudin, pharetra massa egestas,
congue massa.\r", 7:"Sed sit amet nisi at ligula ultrices pos
uere quis nec est.\r", 2:"Sphinx of black quartz, judge my
vow.\r", 3:"How vexingly quick daft zebras jump!\r", 1:"The q
uick brown fox jumps over the lazy dog\r", 8:"Mauris vel purus
viverra, pellentesque elit id, consequat felis.", 5:"Maec
enas consectetur risus a lacus sodales iaculis.\r", 4:"The
five boxing wizards jump quickly\r"]
As can be seen in the Command Prompt Output, the array is not
ordered correctly.
It goes: 6: 7: 2: 3: 1: 8: 5: 4:
Instead of 1: 2: 3: 4: 5: 6: 7: 8:
Associative arrays looks that uses internally a hash map. Hash
maps are unordered. The ouput order, looks that is the result of
where the hash algorithm it's inserting the key/value pair in the
hashmap.
If you need order, you need to use a TreeMap (D std lib,
emsi_containers and containersd have TreeMap implementations) or
something like Java LinkedHashMap if order by insertion it's
enough.
How to efficiently resolve Associative Arrays not being sorted?
I want to read a file, put it into an array, make some search and
replace on the content and output the modified text. However
Associative Arrays seem to be unsorted by default. Should I drop
the Associative Arrays and use something else? What are the ways
to resolve this randomness in Associative Arrays?
ReadfileAndCopyContentIntoArray.d
import std.stdio;
int lineNumber = 0;
char[][int] fileInArray;
void main(){
File exampleFile = File("exampleText.txt");
foreach(line; exampleFile.byLine){
lineNumber++;
fileInArray[lineNumber] ~= line;
}
writeln(fileInArray);
}
exampleText.txt
The quick brown fox jumps over the lazy dog
Sphinx of black quartz, judge my vow.
How vexingly quick daft zebras jump!
The five boxing wizards jump quickly
Maecenas consectetur risus a lacus sodales iaculis.
Morbi sed tortor sollicitudin, pharetra massa egestas, congue
massa.
Sed sit amet nisi at ligula ultrices posuere quis nec est.
Mauris vel purus viverra, pellentesque elit id, consequat felis.
The Command Prompt Output
[6:"Morbi sed tortor sollicitudin, pharetra massa egestas,
congue massa.\r", 7:"Sed sit amet nisi at ligula ultrices pos
uere quis nec est.\r", 2:"Sphinx of black quartz, judge my
vow.\r", 3:"How vexingly quick daft zebras jump!\r", 1:"The q
uick brown fox jumps over the lazy dog\r", 8:"Mauris vel purus
viverra, pellentesque elit id, consequat felis.", 5:"Maec
enas consectetur risus a lacus sodales iaculis.\r", 4:"The five
boxing wizards jump quickly\r"]
As can be seen in the Command Prompt Output, the array is not
ordered correctly.
It goes: 6: 7: 2: 3: 1: 8: 5: 4:
Instead of 1: 2: 3: 4: 5: 6: 7: 8:
Re: Does anyone have gdb pretty printers for buildin types like associative arrays?
On Sunday, 15 March 2020 at 16:42:52 UTC, Panke wrote: Should this just work and by box is not correctly configured or do I need some pretty printers? If so, has someone already made them? Take a look at: https://forum.dlang.org/post/ztyhmmxalpiysgjkv...@forum.dlang.org
Does anyone have gdb pretty printers for buildin types like associative arrays?
At least on my installation they are printed as just a pointer. Should this just work and by box is not correctly configured or do I need some pretty printers? If so, has someone already made them?
Re: += on associative arrays leads to surprising result
On Tuesday, 27 August 2019 at 16:45:53 UTC, Samir wrote: I never understood why the intial value of floats, doubles and reals was NaN. That's for detecting uninitialised variables. If the result of a calculation is NaN, it's likely, that you forgot to initialise the variable.
Re: += on associative arrays leads to surprising result
On Tuesday, 27 August 2019 at 16:12:07 UTC, berni wrote: What's your oppinion on this? As someone relatively new to programming in general and to D in particular, this behavior does, on the surface, seem inconsistent. Good to see that a bug exists for this, per ag0aep6g. I never understood why the intial value of floats, doubles and reals was NaN. Samir
Re: += on associative arrays leads to surprising result
On 27.08.19 18:12, berni wrote:
import std.stdio;
void main()
{
real[int] a;
a[0] += 100;
writeln(a);
}
results (independed of the used compiler) in
[0:100]
I was a little bit surprised, because a[0] += 100 should be the same as
a[0] = a[0]+100, which leads to a range violation error. Furthermore, as
we work with real, I'd expected the result to be NaN...
Is this a bug? I ask, because it would be quite convenient to use it the
way it works now.
For what it's worth, it's in Bugzilla:
https://issues.dlang.org/show_bug.cgi?id=4463
+= on associative arrays leads to surprising result
import std.stdio;
void main()
{
real[int] a;
a[0] += 100;
writeln(a);
}
results (independed of the used compiler) in
[0:100]
I was a little bit surprised, because a[0] += 100 should be the
same as a[0] = a[0]+100, which leads to a range violation error.
Furthermore, as we work with real, I'd expected the result to be
NaN...
Is this a bug? I ask, because it would be quite convenient to use
it the way it works now.
An alternative I found, would be to use object.update. But there
I've to declare the 100 twice which results in code duplication:
a.update(0,()=>100.0L,(ref real v)=>v+100.0L);
Hence, my best solution needs two lines:
if (0 !in a) a[0] = 0;
a[0] += 100;
What's your oppinion on this?
Re: Merging two associative arrays
On Saturday, 24 August 2019 at 19:55:48 UTC, a11e99z wrote: auto ab = a.byPair.chain( b.byPair).assocArray ? Not sure, if it is simpler, but a least without tmp. :) Thanks.
Re: Merging two associative arrays
On Saturday, 24 August 2019 at 19:35:25 UTC, berni wrote: I've got two associative arrays and want to get a new one, which is created out of both of them: This works: string[int] a = [1:"one", 7:"seven"]; string[int] b = [5:"five", 9:"nine"]; string[int] tmp = a.dup; foreach (k,v;b) tmp[k] = v; assert(tmp==[1:"one", 7:"seven", 5:"five", 9:"nine"]); But is there something easier, especially without making that "tmp" explicit. I hoped for a~b, but that didn't work. (I allready know, that there aren't duplicated keys, if that matters.) auto ab = a.byPair.chain( b.byPair).assocArray ?
Merging two associative arrays
I've got two associative arrays and want to get a new one, which is created out of both of them: This works: string[int] a = [1:"one", 7:"seven"]; string[int] b = [5:"five", 9:"nine"]; string[int] tmp = a.dup; foreach (k,v;b) tmp[k] = v; assert(tmp==[1:"one", 7:"seven", 5:"five", 9:"nine"]); But is there something easier, especially without making that "tmp" explicit. I hoped for a~b, but that didn't work. (I allready know, that there aren't duplicated keys, if that matters.)
Re: Why does struct initializer works for arrays but not for associative arrays?
On Wednesday, 14 March 2018 at 15:17:54 UTC, Jonathan M Davis
wrote:
On Wednesday, March 14, 2018 13:36:51 Andre Pany via
Digitalmars-d-learn wrote:
[...]
Well, I think that you have two issues here:
1. Struct literals work in only a few, specific circumstances.
Why, I don't know, but IIRC, someone was writing a DIP to fix
that, and that may or may not fix this case. So, as I
understand it, it's not all that hard to run into places where
they don't work (I confess that I never use them, because I
don't like them any more than I like the fact that struct
constructors are implicitly declared if you don't declare them,
which has caused me bugs when changing the members fo a struct;
both are misfeatures IMHO, though obviously not everyone agrees
on that point).
2. In general in D, the type of an expression is not inferred
based on where it's used. There are a few exceptions where
literals are involved, but in general, if you have something
like
A a = expression;
expression has to evaluate correct on its own without taking A
into account. The fact that something like
Bar b = {s: "str"};
compiles is actually a bit of an oddity in D's semantics in
that respect. So, the fact that it works at all is a bit of a
special case, and clearly, they didn't get everything. My guess
is that the problem is that the dynamic array literal needs a
type, but the compiler is not set up to figure out what type
that is based on the fact that it's being used in a struct
literal.
- Jonathan M Davis
Thanks for the information. As it works fine for dynamic arrays
and the coding
in the compiler seems identical for arrays and associative arrays
I really
wonder why it not work.
Although I really anticipate the DIP, this can be solved as bug
fix.
The chances to solve this a bug fix are much higher than getting
the DIP implemented
in near future (unfortunately).
Kind regards
André
Re: Why does struct initializer works for arrays but not for associative arrays?
On Wednesday, 14 March 2018 at 15:17:54 UTC, Jonathan M Davis wrote: On Wednesday, March 14, 2018 13:36:51 Andre Pany via Digitalmars-d-learn wrote: [...] Well, I think that you have two issues here: 1. Struct literals work in only a few, specific circumstances. Why, I don't know, but IIRC, someone was writing a DIP to fix that, and that may or may not fix this case. For reference, that's the DIP: https://github.com/dlang/DIPs/pull/71
Re: Why does struct initializer works for arrays but not for associative arrays?
On Wednesday, March 14, 2018 13:36:51 Andre Pany via Digitalmars-d-learn
wrote:
> Hi,
>
> I do not understand why struct initializer works for arrays but
> not for
> associative arrays:
>
> struct Bar
> {
> string s;
> }
>
> struct Foo
> {
> Bar[string] asso;
> Bar[] arr;
> }
>
> void main()
> {
> Foo foo = {
> arr: [{s: "123"}],
> asso: ["0": {s: "123"}] // does not work
> };
> }
>
> The coding for both types of arrays looks very similiar:
> https://github.com/dlang/dmd/blob/9ed779a7d68d2ac489338cc4758c10d0cb169b39
> /src/dmd/initsem.d#L634
>
> I cannot spot the difference.
>
> Kind regards
> André
Well, I think that you have two issues here:
1. Struct literals work in only a few, specific circumstances. Why, I don't
know, but IIRC, someone was writing a DIP to fix that, and that may or may
not fix this case. So, as I understand it, it's not all that hard to run
into places where they don't work (I confess that I never use them, because
I don't like them any more than I like the fact that struct constructors are
implicitly declared if you don't declare them, which has caused me bugs when
changing the members fo a struct; both are misfeatures IMHO, though
obviously not everyone agrees on that point).
2. In general in D, the type of an expression is not inferred based on where
it's used. There are a few exceptions where literals are involved, but in
general, if you have something like
A a = expression;
expression has to evaluate correct on its own without taking A into account.
The fact that something like
Bar b = {s: "str"};
compiles is actually a bit of an oddity in D's semantics in that respect.
So, the fact that it works at all is a bit of a special case, and clearly,
they didn't get everything. My guess is that the problem is that the dynamic
array literal needs a type, but the compiler is not set up to figure out
what type that is based on the fact that it's being used in a struct
literal.
- Jonathan M Davis
Re: Why does struct initializer works for arrays but not for associative arrays?
On Wednesday, 14 March 2018 at 13:36:51 UTC, Andre Pany wrote:
Hi,
I do not understand why struct initializer works for arrays but
not for
associative arrays:
struct Bar
{
string s;
}
struct Foo
{
Bar[string] asso;
Bar[] arr;
}
void main()
{
Foo foo = {
arr: [{s: "123"}],
asso: ["0": {s: "123"}] // does not work
};
}
The coding for both types of arrays looks very similiar:
https://github.com/dlang/dmd/blob/9ed779a7d68d2ac489338cc4758c10d0cb169b39/src/dmd/initsem.d#L634
I cannot spot the difference.
Kind regards
André
This might just be a bug. Changing the initializer to an explicit
call to Bar constructor compiles just fine
https://run.dlang.io/is/nuuolx
Even just doing
Foo foo = {
arr: [{s: "123"}],
asso: ["0": {"123"}] // does not work
};
Why does struct initializer works for arrays but not for associative arrays?
Hi,
I do not understand why struct initializer works for arrays but
not for
associative arrays:
struct Bar
{
string s;
}
struct Foo
{
Bar[string] asso;
Bar[] arr;
}
void main()
{
Foo foo = {
arr: [{s: "123"}],
asso: ["0": {s: "123"}] // does not work
};
}
The coding for both types of arrays looks very similiar:
https://github.com/dlang/dmd/blob/9ed779a7d68d2ac489338cc4758c10d0cb169b39/src/dmd/initsem.d#L634
I cannot spot the difference.
Kind regards
André
Re: Associative arrays with keys containing mutable indirections
On 10/30/17 6:49 AM, Tony wrote: I prefer the built-in associative array over using some template library. It has the clean look and ease-of-use that you get with a similar data structure in dynamic languages like Python. I consider it a top feature of D. There is a misunderstanding here. The goal is to replace the implementation of the builtin AA with a template implementation. Right now there are some "magic" pieces of the AA that are helped by the compiler. If we can do this all in a template library, then the syntax doesn't change at all, but now you have better abilities to customize the builtin AAs, or use more specialized templates in some cases. -Steve
Re: Associative arrays with keys containing mutable indirections
I prefer the built-in associative array over using some template library. It has the clean look and ease-of-use that you get with a similar data structure in dynamic languages like Python. I consider it a top feature of D.
Re: Associative arrays with keys containing mutable indirections
On Sunday, October 29, 2017 22:24:57 Nordlöw via Digitalmars-d-learn wrote:
> Shouldn't associative arrays with key types (K) having mutable
> indirections (std.traits.hasAliasing!K) such as
>
> string[ubyte[]]
>
> be disallowed?
>
> If not, in what cases do we want this?
Well, the built-in associative arrays _are_ kind of a mess (though the
situation has improved somewhat over time), but the way that this was
handled does make _some_ sense, even if it's still not correct. If you had
something like
string[ubyte[]] aa;
pragma(msg, (typeof).string);
This will print
string[const(ubyte)[]]
So, similar to C++'s std::map, const gets added for you underneath the hood
for the keys, which does prevent _some_ bad assignments. Unfortunately,
that's still not enough - it really should be immutable. For instance,
void main()
{
string[ubyte[]] aa;
ubyte[] a = [1, 2, 3];
aa[a] = "hello";
}
fails to compile and gives the error
q.d(5): Error: associative arrays can only be assigned values with immutable
keys, not int[]
but this code
void main()
{
string[ubyte[]] aa;
ubyte[] a = [1, 2, 3];
const b = a;
aa[b] = "hello";
}
does compile. So, the way it handles this avoids forcing you to explicitly
put the const or immutable on the key type, which makes for less typing, but
the fact that it's using const and not immutable is definitely a bug. Based
on the error message, clearly it was acknowledged that the keys need to be
immutable, but clearly that hasn't quite made it how the rest of it was
implemented. IIRC, the implementation using void* and casts everywhere, so
it's pretty trivial for it to break the type system if anything gets screwed
up.
But as I said, the built-in AAs are a mess. They work reasonably well for
basic cases but start falling apart when you do stuff like use classes for
keys. While the situation is better than it used to be, historically, we've
had tons of bugs in the implementation, and overall, I'd argue that having
AAs in the language was a mistake. It probably made more sense in the D1
days, but at this point, the only real advantage over a library type is that
the built-in AAs work with AA literals, and a library type wouldn't (though
with a solid DIP, I'm sure that that could be fixed).
Work has been done on templatizing the AA implementation, which should help,
but we'd be much better off sorting out a proper library type and
encouraging its use. One of the suggestions of how to improve things has
been to restrict keys for the built-in AAs to simpler types (e.g. no
classes), but there hasn't been agreement on that, and we'd actually need a
proper library type in Phobos before doing something like that (and the
situation with Phobos and containers is its own mess).
The whole AA mess comes up probably just about every dconf, and I expect
that it will be sorted out eventually, but when that will be, I don't know.
A big part of the question is what's going to happen with Martin's work on
templatizing the implementation. It may be that that can salvage things, but
in the interim, bugs in in the built-in AAs are no surprise. If anything,
it's probably a miracle that they work as well as they do.
- Jonathan M Davis
Associative arrays with keys containing mutable indirections
Shouldn't associative arrays with key types (K) having mutable indirections (std.traits.hasAliasing!K) such as string[ubyte[]] be disallowed? If not, in what cases do we want this?
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 16:34:15 UTC, H. S. Teoh wrote:
On Tue, May 02, 2017 at 02:37:20PM +, ANtlord via
Digitalmars-d-learn wrote:
On Tuesday, 2 May 2017 at 12:41:01 UTC, Jacob Carlborg wrote:
>
> Note that when declared as "enum", all places it's
> referenced, a new associative array will be allocated.
If it is allocated at all places I can move initialization to
module ctor as says evilrat but how can I make an immutable
associative array?
Just declare it immutable. The module ctor can still initialize
it, because ctors are allowed to initialize immutables:
--
immutable string[string] dict;
static this() {
dict = [
"abc": "def",
"ghi": "lmn"
];
}
void main() {
import std.stdio;
writeln(dict["abc"]);
}
--
T
Thanks a lot!
Re: Top level associative arrays
On Tue, May 02, 2017 at 02:37:20PM +, ANtlord via Digitalmars-d-learn wrote:
> On Tuesday, 2 May 2017 at 12:41:01 UTC, Jacob Carlborg wrote:
> >
> > Note that when declared as "enum", all places it's referenced, a new
> > associative array will be allocated.
>
> If it is allocated at all places I can move initialization to module
> ctor as says evilrat but how can I make an immutable associative
> array?
Just declare it immutable. The module ctor can still initialize it,
because ctors are allowed to initialize immutables:
--
immutable string[string] dict;
static this() {
dict = [
"abc": "def",
"ghi": "lmn"
];
}
void main() {
import std.stdio;
writeln(dict["abc"]);
}
--
T
--
Just because you survived after you did it, doesn't mean it wasn't stupid!
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 14:37:20 UTC, ANtlord wrote: On Tuesday, 2 May 2017 at 12:41:01 UTC, Jacob Carlborg wrote: Note that when declared as "enum", all places it's referenced, a new associative array will be allocated. If it is allocated at all places I can move initialization to module ctor as says evilrat but how can I make an immutable associative array? I think it will be more suitable to create singleton of structure.
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 12:41:01 UTC, Jacob Carlborg wrote: Note that when declared as "enum", all places it's referenced, a new associative array will be allocated. If it is allocated at all places I can move initialization to module ctor as says evilrat but how can I make an immutable associative array?
Re: Top level associative arrays
On 2017-05-02 09:48, ANtlord wrote:
Hello! Is it possible to define associative array on top level of module?
I try to compile this code and I get message `Error: non-constant
expression ["s":"q", "ss":"qq"]`
import std.stdio;
auto dict = [
"s": "q",
"ss": "qq"
];
void main()
{
writeln(val);
}
I solved it by replacement of word `auto` by `enum`. It is acceptable
for me. But I notice some inconsistency of logic. When I define simple
array I don't get same compile error and it doesn't lead to define this
array using enum. What is key difference between them in this case?
Thanks. Sorry if my English is not clear.
Note that when declared as "enum", all places it's referenced, a new
associative array will be allocated.
--
/Jacob Carlborg
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 09:50:50 UTC, ANtlord wrote:
On Tuesday, 2 May 2017 at 08:24:09 UTC, evilrat wrote:
Making enum means that value should be available at compile
time and AA's are fully dynamic. But if my memory serves me
well, you can declare empty AA and delay initialization. So
the closest solution is to move initialization of AA to shared
module ctor(note that there is difference between shared and
non-shared, refer to documentation) such as in this example:
static shared this() // <-- module ctors run before main()
{
dict = [
"s": "q",
"ss": "qq"
];
}
string[string] dict;
void main()
{ ... dict is already initialized ... }
I know about D's enums and I know about module ctors but my
question is about difference between array and associative
array in case of definition in top level of module. Why DMD
allows to define array and doesn't allow to define associative
array.
Because it is perfectly fine. They are live in the module scope,
which has its own life time, and from runtime or lifetime
perspective there is no difference here. And since array can be
fixed-sized it is valid to use as enum value. But there is one
catch, in case of enum array it is best to avoid it in favor of
immutable array* because every time you reference it it will
allocate. But thats the difference between enum and not enum, not
the array and map.
This is what I remember from the past, and it is possibly that no
longer relevant anymore.
* not sure if it prevents allocation though, but in theory it
should since it *should* go in to program data section when
compiling
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 08:24:09 UTC, evilrat wrote:
Making enum means that value should be available at compile
time and AA's are fully dynamic. But if my memory serves me
well, you can declare empty AA and delay initialization. So the
closest solution is to move initialization of AA to shared
module ctor(note that there is difference between shared and
non-shared, refer to documentation) such as in this example:
static shared this() // <-- module ctors run before main()
{
dict = [
"s": "q",
"ss": "qq"
];
}
string[string] dict;
void main()
{ ... dict is already initialized ... }
I know about D's enums and I know about module ctors but my
question is about difference between array and associative array
in case of definition in top level of module. Why DMD allows to
define array and doesn't allow to define associative array.
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 07:48:35 UTC, ANtlord wrote:
Hello! Is it possible to define associative array on top level
of module?
I try to compile this code and I get message `Error:
non-constant expression ["s":"q", "ss":"qq"]`
import std.stdio;
auto dict = [
"s": "q",
"ss": "qq"
];
void main()
{
writeln(val);
}
I solved it by replacement of word `auto` by `enum`. It is
acceptable for me. But I notice some inconsistency of logic.
When I define simple array I don't get same compile error and
it doesn't lead to define this array using enum. What is key
difference between them in this case?
Thanks. Sorry if my English is not clear.
Making enum means that value should be available at compile time
and AA's are fully dynamic. But if my memory serves me well, you
can declare empty AA and delay initialization. So the closest
solution is to move initialization of AA to shared module
ctor(note that there is difference between shared and non-shared,
refer to documentation) such as in this example:
static shared this() // <-- module ctors run before main()
{
dict = [
"s": "q",
"ss": "qq"
];
}
string[string] dict;
void main()
{ ... dict is already initialized ... }
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 07:48:35 UTC, ANtlord wrote:
Hello! Is it possible to define associative array on top level
of module?
I try to compile this code and I get message `Error:
non-constant expression ["s":"q", "ss":"qq"]`
import std.stdio;
auto dict = [
"s": "q",
"ss": "qq"
];
void main()
{
writeln(val);
}
I solved it by replacement of word `auto` by `enum`. It is
acceptable for me. But I notice some inconsistency of logic.
When I define simple array I don't get same compile error and
it doesn't lead to define this array using enum. What is key
difference between them in this case?
Thanks. Sorry if my English is not clear.
By the way I notice some strange compile error when I try to
change associatove array defined using enum.
import std.stdio;
enum dict = [
"s": "q",
"ss": "qq"
];
void main()
{
dict["sss"] = "qqq";
}
Compilation of this code returns the error
& el:0x3237ab4 cnt=0 cs=0 & TY* 0x3235794
el:0x3235794 cnt=0 cs=0 call TY* 0x3235744 0x32356f4
el:0x3235744 cnt=0 cs=0 var TYC func _d_assocarrayliteralTX
el:0x32356f4 cnt=0 cs=0 param TYvoid 0x32356a4 0x3235654
el:0x32356a4 cnt=0 cs=0 param TYvoid 0x3234c44 0x3234d34
el:0x3234c44 cnt=0 cs=0 rpair TYucent 0x3234ba4 0x3234bf4
el:0x3234ba4 cnt=0 cs=0 relconst TY* 0+& _TMP8
el:0x3234bf4 cnt=0 cs=0 const TYuns long long 2LL
el:0x3234d34 cnt=0 cs=0 rpair TYucent 0x3234c94 0x3234ce4
el:0x3234c94 cnt=0 cs=0 relconst TY* 0+& _TMP5
el:0x3234ce4 cnt=0 cs=0 const TYuns long long 2LL
el:0x3235654 cnt=0 cs=0 var TY* _D16TypeInfo_HAyaAya6__initZ
Internal error: backend/cgcs.c 352
But when I try to change simple array defined using enum
import std.stdio;
enum arr = [1, 2, 3];
void main()
{
arr ~= 4;
}
I get the clear error `Error: [1, 2, 3] is not an lvalue`
Re: Top level associative arrays
On Tuesday, 2 May 2017 at 07:48:35 UTC, ANtlord wrote:
Hello! Is it possible to define associative array on top level
of module?
I try to compile this code and I get message `Error:
non-constant expression ["s":"q", "ss":"qq"]`
import std.stdio;
auto dict = [
"s": "q",
"ss": "qq"
];
void main()
{
writeln(val);
}
I solved it by replacement of word `auto` by `enum`. It is
acceptable for me. But I notice some inconsistency of logic.
When I define simple array I don't get same compile error and
it doesn't lead to define this array using enum. What is key
difference between them in this case?
Thanks. Sorry if my English is not clear.
Sorry. There is should be `writeln(dict["s"]);` instead
`writeln(val);`
Top level associative arrays
Hello! Is it possible to define associative array on top level of
module?
I try to compile this code and I get message `Error: non-constant
expression ["s":"q", "ss":"qq"]`
import std.stdio;
auto dict = [
"s": "q",
"ss": "qq"
];
void main()
{
writeln(val);
}
I solved it by replacement of word `auto` by `enum`. It is
acceptable for me. But I notice some inconsistency of logic. When
I define simple array I don't get same compile error and it
doesn't lead to define this array using enum. What is key
difference between them in this case?
Thanks. Sorry if my English is not clear.
Re: Best ways to declare associative arrays
On Sunday, 12 March 2017 at 07:58:40 UTC, helxi wrote:
string[string] change(ref string[string] arg_array){
//..
arg_array["first"] = strip(readln());
//..
arg_array["second"] = strip(readln());
//..
return def;
}
Nicholas clarified why your declaration was wrong, but there are
several strange things in your code that you may want to
re-think. Also it looks to me that an associative array is not
the most appropriate type for what you want to do.
To call a function you just pass the names of the arguments, not
their types. So simply change(test), NOT change(string[string]
test)
arg_array is an in-out (ref) parameter, but change() returns
another value of the same type, def, not defined in your code,
and which you do not use in main(). I think you may be interested
only in changing arg_array, so the signature could be instead:
void change(ref ...)
What you seem to want from your associative array is to associate
two strings, "first" and "second" with two values (strings from
the user), and only two. An associate array is more flexible than
that, which is bad, you want your code to restrict you away from
errors. For example if you keep using an associative array you
could at the end of change(): assert(arg_array.length == 2);
I wonder if it's not enough and better for you to use a plain
array. Keys "first" and "second" are not more informative than
numeric indices. You may also use the advantage that an array can
be hard-typed as fixed-length if this is known at compile time
(and if you don't declare it with new), so it restricts your code
in the perfect way:
void change(ref string[2] arg_array) {
arg_array[0] = strip(readln());
arg_array[1] = strip(readln());
}
void main() {
string[2] test;
change(test);
}
Also another disadvantage of associative arrays is that they are
not ordered, so if for example in main() you read through the
values in test with a foreach loop, you may get the result in any
order (second first, and first second is possible). A simple
array will keep order 0, 1.
If you were so bummed about using 0-1 instead of "first"-"second"
you could define:
enum lineKey :size_t {
first = 0,
second
}
void change(ref string[2] arg_array) {
arg_array[lineKey.first ] = strip(readln());
arg_array[lineKey.second] = strip(readln());
}
But at least to me it looks worse. As a programmer you already
know that the first index is 0 and 1 comes next.
Re: Best ways to declare associative arrays
On Sunday, 12 March 2017 at 07:58:40 UTC, helxi wrote:
How would an experienced programmer declare an associative
array of strings that has 2 keys?
My initial impression was string[string][2] my_array; which
does not seem to work.
Here is a snippet of the code I am working on:
import std.string;
import std.stdio;
string[string] change(ref string[string] arg_array){
//..
arg_array["first"] = strip(readln());
//..
arg_array["second"] = strip(readln());
//..
return def;
}
You appear to be confused about the way arrays work.
string[string] foo; //declares an assoc array of strings, index
by string
string[string][2] bar; // declares a static array of two
string[string]
foo = { "key1" : "some data", "key2" : "some other data" }; //
initialise foo with some data
foo["blarg"] = "fxgsdzfcxf"; // insert some more data
associative array of strings that has 2 keys?
is a regular associative array that has two entries. much like
the difference between
int[] baz = [1 ,2]; // has two elements
int[][2] quux; // a static array of length 2 whose elements are
of type int[]
void main(){
string[string][2] test; // remove the `[2]`
change(string[string] test);
just do
change(test);
it is invalid syntax to have a type before the variable in a
function call, unless you were trying to cast it, in which use
`cast(type)(expression)`
}
If you wish to reserve capacity for the array, use
`arr.reserve(N)` to allocate memory enough to hold N elements.
Re: Best ways to declare associative arrays
On Sunday, 12 March 2017 at 07:58:40 UTC, helxi wrote: return def; I meant return arg_array;
Best ways to declare associative arrays
How would an experienced programmer declare an associative array
of strings that has 2 keys?
My initial impression was string[string][2] my_array; which does
not seem to work.
Here is a snippet of the code I am working on:
import std.string;
import std.stdio;
string[string] change(ref string[string] arg_array){
//..
arg_array["first"] = strip(readln());
//..
arg_array["second"] = strip(readln());
//..
return def;
}
void main(){
string[string][2] test;
change(string[string] test);
}
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On Friday, 16 December 2016 at 01:48:59 UTC, Ali Çehreli wrote: On 12/15/2016 05:30 PM, Stefan Koch wrote: On Thursday, 15 December 2016 at 19:30:08 UTC, Ali Çehreli wrote: Yeah, I think the compiler is confused because the function is called in a non-const context during the initialization of an immutable object. I would open an issue: https://issues.dlang.org/enter_bug.cgi?product=D Ali You cannot Assign Associative Arrays at compile-time. Because those are defined by druntime have no stable ABI. Thanks Stefan but at least there should be a better diagnostic instead of the confusing current situation that Ali experienced. Ali Ah kay. Confusing :) Thanks again!
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On 12/15/2016 05:30 PM, Stefan Koch wrote: On Thursday, 15 December 2016 at 19:30:08 UTC, Ali Çehreli wrote: Yeah, I think the compiler is confused because the function is called in a non-const context during the initialization of an immutable object. I would open an issue: https://issues.dlang.org/enter_bug.cgi?product=D Ali You cannot Assign Associative Arrays at compile-time. Because those are defined by druntime have no stable ABI. Thanks Stefan but at least there should be a better diagnostic instead of the confusing current situation that Ali experienced. Ali
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On Thursday, 15 December 2016 at 19:30:08 UTC, Ali Çehreli wrote: Yeah, I think the compiler is confused because the function is called in a non-const context during the initialization of an immutable object. I would open an issue: https://issues.dlang.org/enter_bug.cgi?product=D Ali You cannot Assign Associative Arrays at compile-time. Because those are defined by druntime have no stable ABI.
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On 12/14/2016 04:02 AM, Ali wrote:
> On Tuesday, 13 December 2016 at 23:29:31 UTC, Ali Çehreli wrote:
>> On 12/13/2016 01:36 PM, Ali wrote:
>>
>>> Now about that second part of my problem
>>
>> I'm not entirely sure whether this should work but I think the problem
>> is with mutating the 'frequencies' member of an immutable element of
>> 'rooms'. The error message means that those non-const expressions
>> cannot be shared by a member of an immutable AA.
>
> I'm not sure I fully follow here. Because the variable in the parse
> function is not immutable and the frequencies member is not "created"
> yet, so to say. So after I create the frequencies object, I assign it to
> the member of a Room object. The following illustrates this more clearly
> I think:
>
> struct A { int x; }
> struct B { int[char] x; }
> auto f1() {
> int x;
> x = 3;
> return A(x);
> }
>
> auto f2() {
> int[char] x;
> x['A'] = 2;
> return B(x);
> }
>
> static immutable a = f1();
> static immutable b = f2();
>
> pragma(msg, a);
> pragma(msg, b);
>
> This fails to compile with "Error: non-constant expression ['A':2]".
> But, the pragma(msg) prints out the correct information for both a and b.
Yeah, I think the compiler is confused because the function is called in
a non-const context during the initialization of an immutable object.
I would open an issue:
https://issues.dlang.org/enter_bug.cgi?product=D
Ali
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On Tuesday, 13 December 2016 at 23:29:31 UTC, Ali Çehreli wrote:
On 12/13/2016 01:36 PM, Ali wrote:
Now about that second part of my problem
I'm not entirely sure whether this should work but I think the
problem is with mutating the 'frequencies' member of an
immutable element of 'rooms'. The error message means that
those non-const expressions cannot be shared by a member of an
immutable AA.
I'm not sure I fully follow here. Because the variable in the
parse function is not immutable and the frequencies member is not
"created" yet, so to say. So after I create the frequencies
object, I assign it to the member of a Room object. The following
illustrates this more clearly I think:
struct A { int x; }
struct B { int[char] x; }
auto f1() {
int x;
x = 3;
return A(x);
}
auto f2() {
int[char] x;
x['A'] = 2;
return B(x);
}
static immutable a = f1();
static immutable b = f2();
pragma(msg, a);
pragma(msg, b);
This fails to compile with "Error: non-constant expression
['A':2]". But, the pragma(msg) prints out the correct information
for both a and b.
What's the deal with static immutable associative arrays and D
anyway? Is there some kind of roadmap somewhere or bugs that need
to be fixed before they work properly in D? I'm confused because
it seems like the data actually gets inside the struct object
that has an AA as a member, but then the compiler just decides
"nah, just kidding, this ain't actually ok. bye".
I moved the frequencies to 'data' and hte following worked.
Yeah that is indeed the workaround I mentioned and hence the need
for the pointer to room problem I was having :p This does work.
But I guess I'm just looking for a more ideal solution.
Thanks for all the help so far! :)
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On 12/13/2016 01:36 PM, Ali wrote:
Now about that second part of my problem
I'm not entirely sure whether this should work but I think the problem
is with mutating the 'frequencies' member of an immutable element of
'rooms'. The error message means that those non-const expressions cannot
be shared by a member of an immutable AA.
I moved the frequencies to 'data' and hte following worked. Used 'shared
static this' so that the data is populated per application instead of
per thread. Additionally, the 'd.room.name.writeln' expression works
just fine with DMD64 D Compiler v2.072.1.
import std.algorithm: splitter, map;
import std.array: array;
import std.typecons: Tuple;
import std.stdio: writeln;
static immutable Room[] rooms = import("data.txt").splitter.map!parse.array;
struct Room {
string name;
}
static Tuple!(const(Room*), "room", int[char],
"frequencies")[rooms.length] data;
shared static this() {
foreach (i, ref room; rooms) {
data[i].room =
foreach (letter; room.name) {
data[i].frequencies[letter]++;
}
}
}
Room parse(string line) pure {
immutable name = line;
return Room(name);
}
void main() {
foreach (d; data) {
d.room.name.writeln;
d.frequencies.writeln;
}
}
Ali
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On Tuesday, 13 December 2016 at 21:33:11 UTC, Ali Çehreli wrote:
On 12/13/2016 12:30 PM, Ali wrote:
> foreach (i, room; rooms) {
> data[i].room =
That is the address of the local variable room. You need to use
'ref' there:
foreach (i, ref room; rooms) {
> - Ali
Ahh true!! Cheers. Now about that second part of my problem
Ali
"the real one :o)"
Haha :)
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On 12/13/2016 12:30 PM, Ali wrote:
> foreach (i, room; rooms) {
> data[i].room =
That is the address of the local variable room. You need to use 'ref' there:
foreach (i, ref room; rooms) {
> - Ali
Ali
"the real one :o)"
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
2016-12-13
Thread
bauss (wtf happend to my name took some old cached title LOL??) via Digitalmars-d-learn
On Tuesday, 13 December 2016 at 21:21:34 UTC, Ali wrote: On Tuesday, 13 December 2016 at 21:08:31 UTC, drug007 wrote: (*d.room).name Oh yeah, tried that too. That at least compiles but gives a runtime exception (bad address). Try (*cast(Room*)(d.room)).name
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On Tuesday, 13 December 2016 at 21:08:31 UTC, drug007 wrote: (*d.room).name Oh yeah, tried that too. That at least compiles but gives a runtime exception (bad address).
Re: Accessing members through pointers to structs (also, CTFE associative arrays)
On 13.12.2016 23:30, Ali wrote:
Hi, Long time watcher and recently started playing with D a bit more.
Ran in to a couple of snags that I'll combine in one post. It involves a
data set that contains a list of strings. Each string represents a Room
name. What I'm trying to do is pluck out the room names and also
calculate the frequency each letter occurs in a name, per room.
First problem is to do with pointers to structs. Here's the code:
static immutable rooms = import("data.txt").split("\n").map!parse.array;
static Tuple!(const(Room*), "room", int[char],
"frequencies")[rooms.length] data;
static this() {
foreach (i, room; rooms) {
data[i].room =
// Also calculate frequencies, but that's not important yet.
}
}
void main() {
foreach (d; data) {
d.room.name.writeln; // <-- How do I access name here??
}
}
I've tried d.(*room).name but that didn't work. There's no arrow
I'm sleepy, sorry for quick and probable wrong answer - try (*d.room).name
Accessing members through pointers to structs (also, CTFE associative arrays)
Hi, Long time watcher and recently started playing with D a bit
more. Ran in to a couple of snags that I'll combine in one post.
It involves a data set that contains a list of strings. Each
string represents a Room name. What I'm trying to do is pluck out
the room names and also calculate the frequency each letter
occurs in a name, per room.
First problem is to do with pointers to structs. Here's the code:
static immutable rooms =
import("data.txt").split("\n").map!parse.array;
static Tuple!(const(Room*), "room", int[char],
"frequencies")[rooms.length] data;
static this() {
foreach (i, room; rooms) {
data[i].room =
// Also calculate frequencies, but that's not important
yet.
}
}
void main() {
foreach (d; data) {
d.room.name.writeln; // <-- How do I access name here??
}
}
I've tried d.(*room).name but that didn't work. There's no arrow
operator. I've tried making my tuple a ref Room instead, but
that's a no go as well. I can copy the Room object directly in to
the tuple, but since it's already there in static immutable data
I'd rather just have a pointer to it.
Is there a way to do that?
Second problem is to do with associative arrays. At first the
Room object had a frequencies object in it (ie: int[char] <-
number of times a character appears in the name).
In my parse function, if I add a foreach loop that loops through
the letters in the room's name, and adds populates an associative
array like so:
Room parse(string line) {
immutable name = // blah
int[char] frequencies;
foreach (letter; name) {
frequencies[letter] += 1
}
return Room(name, frequencies);
}
pragma(msg, rooms); // <- this works!
In the above case the pragma actually prints out all the Room
objects, with their respective frequencies calculated correctly.
But *after* it has printed it out, then a whole list of
compilation errors that all look like this:
Error: non-constant expression ['d':1, 'r':3, 'x':1, 'e':1,
'v':2, 'k':2, 'z':1, 't':1, 'u':1, 'p':2, 'c':1, 's':1, 'f':2,
'i':2]
But it seems that it was calculated correctly, it just can't be
assigned to the actual variable.
My current workaround includes taking frequencies out of the Room
struct and calculating them inside a module constructor (hence
the first question on the Tuple and Room *)
Are there other workarounds?
Cheers, and thanks for any help!
- Ali
Re: Accessing contents of associative arrays in an optimal way
Thank you!
Re: Accessing contents of associative arrays in an optimal way
On 07/09/2016 10:32 PM, phant0m wrote:
As far as I know, AA implemented as a hashtable. So, will there be two
searches performed (one search for each line)?
records[3].value = 10;
records[3].name = "name";
Yup. A good optimizer may be able to eliminate one, but conceptually
there are two lookups.
How can I access elements of this AA by a "reference"? In C++ I can use
reference to an element of the map:
Foo& foo = records.find(3).second;
foo.value = 10;
foo.name = "name";
You can take the address of `records[3]`:
Foo* foo = [3];
foo.value = 10;
foo.name = "name";
If `records[3]` may be not set, you can use `3 in records` to get a
pointer to the value or null if the key isn't set:
Foo* foo = 3 in records;
if (foo is null) {records[0] = Foo.init; foo = 3 in records;}
foo.value = 10;
foo.name = "name";
Re: Accessing contents of associative arrays in an optimal way
On 07/09/2016 01:32 PM, phant0m wrote:
> Suppose I have AA of structures:
>
> struct Foo {
> int value;
> string name;
> }
>
> Foo[int] records;
>
> As far as I know, AA implemented as a hashtable. So, will there be two
> searches performed (one search for each line)?
> records[3].value = 10;
> records[3].name = "name";
Yes, two searches. Although it is slower, it's still O(1). :)
> How can I access elements of this AA by a "reference"?
The 'in' operator returns a pointer to the element:
import std.stdio;
struct Foo {
int value;
string name;
}
void main() {
Foo[int] records;
records[3] = Foo(42, "hello");
if (auto record = 3 in records) {
record.value = 10;
record.name = "name";
}
writeln(records);
}
Ali
Accessing contents of associative arrays in an optimal way
Suppose I have AA of structures:
struct Foo {
int value;
string name;
}
Foo[int] records;
As far as I know, AA implemented as a hashtable. So, will there
be two searches performed (one search for each line)?
records[3].value = 10;
records[3].name = "name";
How can I access elements of this AA by a "reference"? In C++ I
can use reference to an element of the map:
Foo& foo = records.find(3).second;
foo.value = 10;
foo.name = "name";
I found that in D I can use a "with" keyword to achieve the same
effect:
with(values[0]) {
value = 10;
name = "name";
}
Is this the only optimal way?
Re: Reserving capacity in associative arrays
On Tuesday, 16 February 2016 at 19:49:55 UTC, H. S. Teoh wrote: On Tue, Feb 16, 2016 at 07:34:07PM +, Jon D via Digitalmars-d-learn wrote: On Tuesday, 16 February 2016 at 16:37:07 UTC, Steven Schveighoffer wrote: >On 2/14/16 10:22 PM, Jon D wrote: >>Is there a way to reserve capacity in associative arrays? >>[snip] >>The underlying implementation of associative arrays appears >>to take >>an initial number of buckets, and there's a private resize() >>method, >>but it's not clear if there's a public way to use these. Rehashing (aa.rehash) would resize the number of buckets, but if you don't already have the requisite number of keys, it wouldn't help. Thanks for the reply and the detailed example for manually controlling GC. I haven't experimented with taking control over GC that way. Regarding reserving capacity, the relevant method is aa.resize(), not aa.rehash(). See: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/aaA.d#L141. This allocates space for the buckets, doesn't matter if the keys are known. Note that every time the buckets array is resized the old bucket array is walked and elements reinserted. Preallocating allocating a large bucket array would avoid this. See also the private constructor in the same file (line 51). It takes an initial size. --Jon
Re: Reserving capacity in associative arrays
On Tue, Feb 16, 2016 at 07:34:07PM +, Jon D via Digitalmars-d-learn wrote:
> On Tuesday, 16 February 2016 at 16:37:07 UTC, Steven Schveighoffer wrote:
> >On 2/14/16 10:22 PM, Jon D wrote:
> >>Is there a way to reserve capacity in associative arrays?
> >>[snip]
> >>The underlying implementation of associative arrays appears to take
> >>an initial number of buckets, and there's a private resize() method,
> >>but it's not clear if there's a public way to use these.
Rehashing (aa.rehash) would resize the number of buckets, but if you
don't already have the requisite number of keys, it wouldn't help.
[...]
> >I would caution to be sure of this cause, however, before thinking it
> >would solve the problem. The AA not only uses an array for buckets,
> >but allocates a memory location for each element as well. I'm often
> >wrong when I assume what the problem is when it comes to GC issues...
> >
> Completely agree. After posting I decided to take a more methodical
> look. Not finished yet, but I can share part of it. Key thing so far
> is noticeable step function related to GC costs related to AA size
> (likely not a surprise).
>
> My programs work with large data sets. Size is open-ended, what I'm
> trying to do is get an idea of the data set sizes they will handle
> reasonably. For purposes of illustration, word-count is a reasonable
> proxy for what I'm doing. It was in this context that I saw
> significant performance drop-off after 'size_t[string]' AAs reached
> about 10 million entries.
I also have a program that builds very large AA's (also up to millions
of keys, sometimes more) that essentially last for the entire duration
of the program, and I've found that a lot of the performance degradation
can be attributed to overly-frequent GC collection runs. As a
workaround, I did something like this:
size_t counter = 0;
void main() {
GC.disable();
doWork();
GC.enable();// optional
}
void doWork() {
Data[Key] aa = ...;
mainloop: while(...) {
... // lots of stuff, including adding things to the AA
// Manually schedule GC collections.
// The 10_000 is just an example number, tweak
// as you see fit.
if (++counter % 10_000)
GC.collect();
}
}
Basically, I run GC collections on my own schedule, with the frequency
controlled by tweaking the counter check. (Obviously, how you implement
this depends on whether there are regular or semi-regular units of work
that your program does, and whether they are a good basis for scheduling
collections.)
By tweaking the frequency of the manual GC collections, I've been able
to obtain 30-40% speedups in my program (in some cases, it could get as
high as 50%). YMMV.
Of course, as with all performance-related questions, the only way to be
sure that the GC (or anything else) is the cause of your problem, is to
profile. More often than not, I've found that the profiler has proven
me wrong about where the real bottleneck is, so I try not to assume
anything until I see the profile data.
T
--
If Java had true garbage collection, most programs would delete
themselves upon execution. -- Robert Sewell
Re: Reserving capacity in associative arrays
On Tuesday, 16 February 2016 at 16:37:07 UTC, Steven
Schveighoffer wrote:
On 2/14/16 10:22 PM, Jon D wrote:
Is there a way to reserve capacity in associative arrays?
[snip]
The underlying implementation of associative arrays
appears to take an initial number of buckets, and there's
a private resize() method, but it's not clear if there's a
public way to use these.
There is not a public way to access these methods unfortunately.
It would be a good addition to druntime I believe.
Recently, I added a clear method to the AA, which does not
reduce capacity. So if you frequently build large AAs, and then
throw them away, you could instead reuse the memory.
My programs build AAs lasting the lifetime of the program.
I would caution to be sure of this cause, however, before
thinking it would solve the problem. The AA not only uses an
array for buckets, but allocates a memory location for each
element as well. I'm often wrong when I assume what the problem
is when it comes to GC issues...
Completely agree. After posting I decided to take a more
methodical look. Not finished yet, but I can share part of it.
Key thing so far is noticeable step function related to GC costs
related to AA size (likely not a surprise).
My programs work with large data sets. Size is open-ended, what
I'm trying to do is get an idea of the data set sizes they will
handle reasonably. For purposes of illustration, word-count is a
reasonable proxy for what I'm doing. It was in this context that
I saw significant performance drop-off after 'size_t[string]' AAs
reached about 10 million entries.
I've started measuring with a simple program. Basically:
StopWatch sw;
sw.start;
size_t[size_t] counts;
foreach (i; 0..iterations)
counts[uniform(0, uniqMax)]++;
sw.stop;
Same thing with string as key ('size_t[string]') AAs.
'iterations' and 'uniqMax' are varied between runs. GC stats are
printed (via "--DRT-gcopt=profile:1"), plus timing and AA size.
(Runs use LDC 17, release mode compiles, a fast 16GB MacBook).
For the integer as key case ('size_t[size_t]', there are notable
jumps in GC total time and GC max pause time as AA size crosses
specific size thresholds. This makes sense, as the AA needs to
grow. Approximate steps:
| entries | gc_total (ms) | gc_max_pause (ms) |
|-+---+---|
| 2M |30 |60 |
| 4M | 200 | 100 |
| 12M | 650 | 330 |
| 22M | 1650 | 750 |
| 44M | 5300 | 3200 |
Iterations didn't matter, and gc total time and gc max time were
largely flat between these jumps.
This suggests AA resize is the likely driver, and that
preallocating a large size might address it.
To the point about being sure about cause - my programs use
strings as keys, not integers. The performance drop-off with
strings was quite a bit more significant than with integers. That
analysis seems a bit trickier, I'm not done with that yet.
Different memory allocation, perhaps effects from creating
short-lived, temporary strings to test AA membership. Could
easily be that string use or the combo of AAs with strings as key
is a larger effect.
The other thing that jumps out from the table is the GC max pause
time gets to be multiple seconds. Not an issue for my tools,
which aren't interactive at those points, but would be
significant issue for many interactive apps.
--Jon
Re: Reserving capacity in associative arrays
On Tuesday, 16 February 2016 at 17:05:11 UTC, Basile B. wrote: On Tuesday, 16 February 2016 at 16:37:07 UTC, Steven Schveighoffer wrote: There is not a public way to access these methods unfortunately. It would be a good addition to druntime I believe. -Steve After reading the topic i've added this enhancement proposal, not quite sure if it's possible: https://issues.dlang.org/show_bug.cgi?id=15682 The idea is to concatenate smallers AA into the destination. There is also this: https://issues.dlang.org/show_bug.cgi?id=2504
Re: Reserving capacity in associative arrays
On Tuesday, 16 February 2016 at 16:37:07 UTC, Steven Schveighoffer wrote: There is not a public way to access these methods unfortunately. It would be a good addition to druntime I believe. Recently, I added a clear method to the AA, which does not reduce capacity. So if you frequently build large AAs, and then throw them away, you could instead reuse the memory. I would caution to be sure of this cause, however, before thinking it would solve the problem. The AA not only uses an array for buckets, but allocates a memory location for each element as well. I'm often wrong when I assume what the problem is when it comes to GC issues... -Steve After reading the topic i've added this enhancement proposal, not quite sure if it's possible: https://issues.dlang.org/show_bug.cgi?id=15682 The idea is to concatenate smallers AA into the destination.
