After digging around a bit more, I found even weirder results. when I fix initial size of hash table, it returns different value. When I modify line 3 of my original code to var dp = initTable[int, int](4) Run
so that resize should not be occur, and it also returns 5, which isn't the same as my first result. Modifying line 3 to var dp = newSeq[int](4) Run also returns 5, so I suspect something occured while resizing hash table. I'll also try gdb debugging, but post just as a quick note.