Hi Jonathan --
Following up on my previous response, David Iten dug into this a bit further
and raised a question as to whether this was truly a bug or not which we'll
probably need to wrestle with a bit further, but the supporting information was
interesting and useful. Let's look at a simpler example:
record array_of_reals {
var v: [1..0] real;
}
var A, B: array_of_reals;
A.v.push_back(1.2);
A.v.push_back(2.3);
writeln(A);
B = A;
writeln(B);
Here, intuitively, we'd like B to take on A's value. But literally what's
happening is that B.v is a 0-element array and A.v is a 2-element array, so we
get an error message similar to the one you reported. This is because the
default assignment for records of this type will effectively do:
B.v = A.v
and we don't support auto-resizing of arrays based on assignment, hence the
size/shape mismatch error. Your program was effectively doing the same thing
except that it was the hash table reallocation code that was attempting to copy
one record into the other rather than user code.
A quick fix for this that David suggested (and which is a bit more satisfying
than my requestCapacity() workaround) is to change your record to store an
explicit domain as follows:
record array_of_reals {
var D: domain(1);
var v: [D] real;
}
var A, B: array_of_reals;
A.v.push_back(1.2);
A.v.push_back(2.3);
writeln(A);
B = A;
writeln(B);
Now, the default assignment for records will be equivalent to:
B.D = A.D;
B.v = A.v;
That is, first update v's index set (B.D) which results in resizing v, and then
copy the values from one v to another.
This also suggests an alternate workaround to the original program which would
be to add an explicit assignment overload for the array_of_reals record:
proc =(ref X: array_of_reals, Y: array_of_reals) {
for e in Y.v do
X.v.push_back(e);
}
Doing either of these changes for your record similarly allows your test to
pass.
For me, the slightly open semantic question is whether our associative array
implementation should and could be doing more to either protect the user from
such things (where my current guess is "no", but I'm still wrestling with it)
and/or whether it could generate a clearer error message in cases like this.
In any case, I hope this explanation brings slightly more closure and
understanding than this morning's.
-Brad
________________________________
From: Brad Chamberlain <[email protected]>
Sent: Monday, June 26, 2017 9:34:54 AM
To: Jonathan Dursi
Cc: [email protected]
Subject: Re: Ragged associative arrays: "assigning between arrays of different
shapes in dimension 1"
Hi Jonathan --
To your main question, with a quick look, this definitely appears to be a
bug in the implementation to me. I believe it is triggered when the
associative domain resizes itself. One way that you may be able to work
around the bug is to use the 'requestCapacity' method on the associative
domain to make it sufficiently big that it need not be resized (if your
program is amenable to such a change). For example, I inserted a call:
strs.requestCapacity(maxsize*3);
right after the declaration of 'strs' and this permitted me to run for a
variety of 'maxsize's without issue. Meanwhile, we'll dig in a bit deeper
and see if we can get a proper fix in short order.
At a higher level, your desire to simply have a ragged array of varying
sized arrays is something that the language has long intended to support
(we refer to them as skyline arrays in documentation), but for which we
haven't yet put in the effort, in part due to the fact that workarounds
like yours are available. So your approach is a reasonable one even
though the record is a bit regrettable...
To your procedural query, questions can be directed here (or
chapel-developers), or on IRC, or on StackOverflow, or on GitHub issues.
My suggestion would be:
* If something seems to be (or obviously is) a bug (or feature request),
use GitHub issues
* If something is a question whose answer might benefit future users (and
it is appropriate for the context), ask it on StackOverflow.
* Otherwise, ask it on a mailing list or on IRC (where mailing lists are
better for things that aren't transitory or throwaway by nature).
We receive wake-up calls for new GitHub issues and StackOverflow posts
tagged 'chapel' so will see any of these in roughly a similar amount of
time.
Thanks for bringing this to our attention,
-Brad
On Sat, 24 Jun 2017, Jonathan Dursi wrote:
> Hi, all:
>
> (Is StackOverflow now the preferred channel for questions, or is here still
> good?)
>
> I’m trying to write a program with a ragged associative array to describe a
> graph’s edges - where the key is a string (say) and the value is an array of
> non-fixed size - and I quickly get runtime errors once the associative array
> gets long enough. A simple example follows below (and in attachment):
>
> module ragged_associative_array {
> record array_of_ints {
> var arr : [1..0] int;
> }
>
> config const maxsize = 12;
>
> proc main() {
> var strs: domain(string);
> var dict_of_array_of_ints : [strs] array_of_ints;
>
> for i in 1..maxsize do
> dict_of_array_of_ints[i:string].arr.push_back(i);
>
> for str in strs do
> writeln(str, ": ", dict_of_array_of_ints[str]);
> }
> }
>
> Running gives:
>
> $ chpl -o ragged_associative_array ragged_associative_array.chpl
> $ ./ragged_associative_array --maxsize=11
> 10: (arr = 10)
> 11: (arr = 11)
> 9: (arr = 9)
> 8: (arr = 8)
> 5: (arr = 5)
> 4: (arr = 4)
> 7: (arr = 7)
> 6: (arr = 6)
> 1: (arr = 1)
> 3: (arr = 3)
> 2: (arr = 2)
> $ ./ragged_associative_array --maxsize=12
> ragged_associative_array.chpl:13: error: halt reached - assigning between
> arrays of different shapes in dimension 1: 0 vs. 1
>
> I don’t hit this problem for a non-associative array (e.g., var
> dict_of_array_of_ints : [1..maxsize] array_of_ints), at least not at the
> same problem size. (Having a record array_of_ints seems to be necessary; I
> can’t use a type alias or just have e.g. var dict_of_array_of_ints :
> [1..maxsize] [1..0] int or else I I immediately the runtime error “cannot
> call push_back on an array defined over a domain with multiple arrays”
> ,which I guess I can understand). Having a fixed maximum size and count
> field works, but is something I’m trying to avoid.
>
> This is on my mac with the current homebrew install:
>
> machine info: Darwin coredump.local 16.6.0 Darwin Kernel Version 16.6.0: Fri
> Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
> CHPL_HOME: *
> script location: /usr/local/Cellar/chapel/1.15.0/libexec/util
> CHPL_TARGET_PLATFORM: darwin
> CHPL_TARGET_COMPILER: clang
> CHPL_TARGET_ARCH: native
> CHPL_LOCALE_MODEL: flat
> CHPL_COMM: none
> CHPL_TASKS: qthreads
> CHPL_LAUNCHER: none
> CHPL_TIMERS: generic
> CHPL_UNWIND: none
> CHPL_MEM: jemalloc
> CHPL_MAKE: make
> CHPL_ATOMICS: intrinsics
> CHPL_GMP: none
> CHPL_HWLOC: hwloc
> CHPL_REGEXP: re2
> CHPL_WIDE_POINTERS: struct
> CHPL_AUX_FILESYS: none
>
> chpl Version 1.15.0
> Copyright (c) 2004-2017, Cray Inc. (See LICENSE file for more details)
>
> $ clang --version
> Apple LLVM version 8.1.0 (clang-802.0.42)
> Target: x86_64-apple-darwin16.6.0
> Thread model: posix
> InstalledDir:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolch
> ain/usr/bin
>
>
> Am I doing something silly - is there another, better way to deal with
> arrays of non-fixed sizes?
>
> Best,
>
> Jonathan
>
>
> ---------------------------------------------------------------------------
> ---
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Chapel-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/chapel-users
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
