Unexpected behaviour in associative array
Hi all, I'm using a custom Struct as the key type in an associative array. I have defined the toHash() and opEquals(...) functions, and the problem I'm having is that the test mykey in aa always fails (returns null) even though there are keys in the aa that return identical toHash() values than mykey and return true for opEquals. This is beyond frustrating, because at this point I'm pretty much out of ideas. Have you had this problem before? Any tips or suggestions would be much appreciated. Arredondo.
Re: Convert array to custom type when passing function argument
On 04/17/2019 05:21 PM, Stefanos Baziotis wrote:
> On Wednesday, 17 April 2019 at 23:44:42 UTC, Adam D. Ruppe wrote:
>>explicit Test(const char* foo) {} // this opts out of the cool
>> thing above
> Actually, I asked initially because in C++ you can do the thing I
> described.
> I thought that you meant something I missed with out-out.
So, Adam meant opt-out but wrote out-out. :)
Ali
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 11:10:16 UTC, Arredondo wrote: Hi all, I'm using a custom Struct as the key type in an associative array. I have defined the toHash() and opEquals(...) functions, and the problem I'm having is that the test mykey in aa always fails (returns null) even though there are keys in the aa that return identical toHash() values than mykey and return true for opEquals. This is beyond frustrating, because at this point I'm pretty much out of ideas. Have you had this problem before? Any tips or suggestions would be much appreciated. Arredondo. Could you please post the coding, otherwise it is quite hard to help you. Kind regards Andre
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 11:53:37 UTC, Andre Pany wrote: On Friday, 19 April 2019 at 11:41:53 UTC, Arredondo wrote: On Friday, 19 April 2019 at 11:32:17 UTC, Andre Pany wrote: Could you please post the coding, otherwise it is quite hard to help you. Kind regards Andre Yes, I'm working on isolating the problem. It's a bit laborious because the custom Struct is actually a wrapper around an ndslice matrix, and I still don't know if the issue is reproducible without this dependency. With D the tool Dustmite is included, it reduces the code automatically for you. You just provide the expected compiler or linked output. It is also very well integrated into Dub. Kind regards Andre https://dub.pm/commandline.html#dustmite
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 11:32:17 UTC, Andre Pany wrote:
Could you please post the coding, otherwise it is quite hard to
help you.
Here's a reasonably-sized code fragment that demonstrates the
issue. I hope the comments along the way are descriptive enough
Thanks,
Arredondo
--
// this is a thin wrapper around a 2D byte matrix
// that uses an ndslice internally
struct State {
import std.digest.murmurhash;
import mir.ndslice;
this(byte rows, byte cols) inout @safe pure nothrow {
payload = slice!byte([rows, cols], 0);
}
size_t toHash() pure nothrow {
byte[] data = payload.field();
immutable digest = digest!(MurmurHash3!(128, 64))(data);
immutable hash = *cast(size_t*) [0];
return hash;
}
bool opEquals(ref inout State q) inout @safe pure nothrow {
return payload == q;
}
Slice!(Contiguous, [2], byte*) payload;
alias payload this;
}
void main(string[] args) {
import std.stdio;
// create a key
auto key1 = State(2, 2);
key1[0, 0] = cast(byte) 1;
// insert it in an assoc. array
int[State] map;
map[key1] = 101;
// create the exact same key
auto key2 = State(2, 2);
key2[0, 0] = cast(byte) 1;
// it is an identical key as far as the aa is concerned
assert(key1.opEquals(key2));
assert(key2.opEquals(key1));
assert(key1.toHash == key2.toHash);
// yet it is not in the map
writeln(key1 in map); // prints some memory address
writeln(key2 in map); // prints null <-- unexpected behaviour
}
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 11:41:53 UTC, Arredondo wrote: On Friday, 19 April 2019 at 11:32:17 UTC, Andre Pany wrote: Could you please post the coding, otherwise it is quite hard to help you. Kind regards Andre Yes, I'm working on isolating the problem. It's a bit laborious because the custom Struct is actually a wrapper around an ndslice matrix, and I still don't know if the issue is reproducible without this dependency. With D the tool Dustmite is included, it reduces the code automatically for you. You just provide the expected compiler or linked output. It is also very well integrated into Dub. Kind regards Andre
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 11:55:41 UTC, Andre Pany wrote: https://dub.pm/commandline.html#dustmite Thanks, but it appears that this tool is used to isolate the cause of build errors, and I'm not having build errors, just unexpected behavior at runtime. Something I have observed while continuing the tinkering is that sometimes the call key in aa segfaults (Program exited with code -11) when key is not in the aa. Very strange indeed.
gtkDcoding Blog Post # 28 - Menu II - CheckMenuItem
Even though it's Good Friday, there's still a new blog post. As the thread title says, it's about using the CheckMenuItem, two examples this time. You can find it here: http://gtkdcoding.com/2019/04/19/0028-checkmenuitems.html Have a great long weekend, everyone!
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 12:03:33 UTC, Arredondo wrote: key in aa Keep in mind that D's `in` operator returns a *pointer* to the element, or null if it isn't there. If you aren't treating the return value as a pointer, you could hit trouble.
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 12:43:06 UTC, Adam D. Ruppe wrote: On Friday, 19 April 2019 at 12:03:33 UTC, Arredondo wrote: key in aa Keep in mind that D's `in` operator returns a *pointer* to the element, or null if it isn't there. If you aren't treating the return value as a pointer, you could hit trouble. I understand that. The issue is that it should't return null if theres a matching element in the aa!
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 12:37:10 UTC, Arredondo wrote: Here's a reasonably-sized code fragment that demonstrates the issue. Oh dear, I don't know what's going on there. It might just be that toHash is secretly dependent on various attributes in the signature. I'd try to match the attrs exactly from this: https://dlang.org/spec/hash-map.html#using_struct_as_key size_t toHash() const @safe pure nothrow; bool opEquals(ref const typeof(this) s) const @safe pure nothrow; and see if it makes a difference. idk though.
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 11:32:17 UTC, Andre Pany wrote: Could you please post the coding, otherwise it is quite hard to help you. Kind regards Andre Yes, I'm working on isolating the problem. It's a bit laborious because the custom Struct is actually a wrapper around an ndslice matrix, and I still don't know if the issue is reproducible without this dependency.
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 12:03:33 UTC, Arredondo wrote: On Friday, 19 April 2019 at 11:55:41 UTC, Andre Pany wrote: https://dub.pm/commandline.html#dustmite Thanks, but it appears that this tool is used to isolate the cause of build errors, and I'm not having build errors, just unexpected behavior at runtime. Something I have observed while continuing the tinkering is that sometimes the call key in aa segfaults (Program exited with code -11) when key is not in the aa. Very strange indeed. You can also use it for runtime, either use --program-status or --program-regex. Kind regards Andre
Re: How to debug long-lived D program memory usage?
On Friday, 19 April 2019 at 03:27:04 UTC, Adam D. Ruppe wrote: On Friday, 19 April 2019 at 02:58:34 UTC, Alex wrote: Curious, what are these programs? A terminal emulator gui (like xterm), a detachable terminal emulator (like gnu screen), a slack client, an irc client, and a bunch of http servers including d doc search, a work program, and a personal utility. Ok, nothing useful ;) I was thinking you might be doing stuff like running a security system that did computer vision, or some type of advanced house monitoring and control(voice activated doors or something) ;) Could you not, as a quick fix, just reboot and automate restarting those? Maybe design an auto-restart which saves the state, shuts itself down and then relaunches itself and loads the data? This could be done as a fail safe when memory consumption get's too high. All of them would show growing memory time, some worse than others. You can see a lot of difference in them - gui programs, terminal programs, network server programs. But, I did write it all myself, so it could be a mistake I keep on making. So far, I basically tracked down the terminal emulator things to being inefficient scrollback storage. I made the structs smaller and limited the amount saved more than before and cut this by half. The ddoc search was keeping the index in memory, that's fixed, but it still shows growing usage over time. Of course, restarting that is trivial if need be, but still, I wanna make sure I am doing it right too - especially if it is one of my underlying libraries to blame. Gonna have to be one of those things you track down because it could be something as simple as the GC or something more serious. You might have hook in to the GC and just write out stats, I believe there is a stats collector somewhere though. Yes, indeed. I am starting to make serious progress now - mostly the fault is me storing excessive histories inefficiently. Should have been obvious in retrospect, but I didn't realize just how much overhead there was in my designs! D should have a very good memory statistics library built(I guess it has something with the switches)... since it should have no issues tracking memory usage. Every memory allocation must have a corresponding deallocation for stable programs. All allocations and deallocations have specific file locations or occur in the GC. I don't see why it would be difficult to monitor this stuff. As I mentioned, I generally never use new precisely so I can track this stuff myself and have a nice printout of memory usage when I need it and even verify the net memory allocation 0 zero on program exit. I haven't messed with the GC but I imagine it too shouldn't be hard to add the info too.
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 12:48:32 UTC, Adam D. Ruppe wrote: It might just be that toHash is secretly dependent on various attributes in the signature. You nailed it. This was it. It was not trivial to add the missing @safe and const attributes, but it worked. Thanks!
Re: gtkDcoding Blog Post # 28 - Menu II - CheckMenuItem
On Friday, 19 April 2019 at 12:12:28 UTC, Ron Tarrant wrote: Even though it's Good Friday, there's still a new blog post. As the thread title says, it's about using the CheckMenuItem, two examples this time. You can find it here: http://gtkdcoding.com/2019/04/19/0028-checkmenuitems.html Have a great long weekend, everyone! Hi, thanks! The function's closing comments (first code example) need some maintenance, 'exit' also in the code on the page itself: ``` } // exit() ... } // keep() ... } // toss() ```
Re: Unexpected behaviour in associative array
On Friday, 19 April 2019 at 12:37:10 UTC, Arredondo wrote: Here's a reasonably-sized code fragment that demonstrates the issue. I hope the comments along the way are descriptive enough Hmm. Have you tried using a different compiler or 32/64 bit? I had a weird "null out of nowhere" bug going on with associative array some time ago - https://issues.dlang.org/show_bug.cgi?id=19662
Re: How to debug long-lived D program memory usage?
On Friday, 19 April 2019 at 17:30:50 UTC, Alex wrote: I was thinking you might be doing stuff like running a security system that did computer vision, or some type of advanced house monitoring and control(voice activated doors or something) ;) LOL, now *that* would be totally useless! Of course, I can restart stuff, it is just a hassle, and besides, I also wanna make sure my libs aren't too badly written. D should have a very good memory statistics library built(I guess it has something with the switches)... since it should have no issues tracking memory usage. Indeed, I am sorta starting to make a hacky add-on module that will provide some of that info. But I need to hook deallocations too and haven't gotten that yet. It'll be cool to get a report out of the program at any time that tells me which lines have how much outstanding allocations.
Re: gtkDcoding Blog Post # 28 - Menu II - CheckMenuItem
On Friday, 19 April 2019 at 17:36:34 UTC, number wrote: The function's closing comments (first code example) need some maintenance, 'exit' also in the code on the page itself: ``` } // exit() ... } // keep() ... } // toss() ``` :) Thanks, as always, number. Fixed and updated.
Re: Unexpected behaviour in associative array
On Fri, Apr 19, 2019 at 08:15:22PM +, Arredondo via Digitalmars-d-learn wrote: > On Friday, 19 April 2019 at 12:48:32 UTC, Adam D. Ruppe wrote: > > It might just be that toHash is secretly dependent on various > > attributes in the signature. > > > > You nailed it. This was it. It was not trivial to add the missing > @safe and const attributes, but it worked. [...] For the future, also note that if you want to test opCmp it's better to use == instead of calling opCmp directly, that way you know for sure that the compiler picked up your definition of opCmp. (IIRC if the signature doesn't match for whatever reason the compiler may just silently revert to the default implementation.) Similarly, if you want to test toHash, use typeid(obj).getHash(), rather than calling toHash directly. That way you know for sure that the compiler has picked up your custom toHash, rather than just the default implementation. T -- "Maybe" is a strange word. When mom or dad says it it means "yes", but when my big brothers say it it means "no"! -- PJ jr.
