Ah, Gary! Thanks. I blame it on my ability filter out code comments.
Thanks for the clarification. In my current prototype I have a single
assoc array with a domain that is operated on in parallel in different
locales (with each locale owning a partition of the key space). Based on
my understanding, it sounds like additions to the domain / assoc. array on
a locale will also span locales, potentially causing the same
parallelization issues we mentioned. So much for the naive approach.
it may help to see my code:
https://github.com/briangu/libev-examples/blob/master/chapel/search.chpl#L66
On Fri, Apr 24, 2015 at 1:55 PM, Ben Harshbarger <[email protected]>
wrote:
> Hi Brian,
>
> There is an example of implicitly adding via array assignment, but it's
> pretty short:
>
> https://github.com/chapel-lang/chapel/blob/release/1.11/test/release/examples/primers/associative.chpl#L234
>
> Your interpretation is correct. It is not safe to modify the domain and
> access the array in parallel due to resizing. Adding an index to an
> associative domain may cause the underlying arrays in both the
> associative domain and associative array to be reallocated.
>
> Does that help?
>
> -Ben
>
> From: Brian Guarraci <[email protected]>
> Date: Friday, April 24, 2015 at 1:23 PM
> To: Ben Harshbarger <[email protected]>
> Cc: "[email protected]" <
> [email protected]>
>
> Subject: Re: [Chapel-developers] associative array questions
>
> Hi Ben,
>
> Thanks for the clarification. I was indeed wondering about the parallel
> operations on the domain as an independent set of operations and the
> corresponding thread safety. As you describe it, the behavior of the
> associative arrays is I would expect in Java or similar - don't expect it
> to be well-behaved collection under parallel operations.
>
> Follow on questions:
>
> Does adding/removing to a domain cause any action in the associate
> array? Is another way to interpret your answer: if I operate on a domain
> in parallel, accessing the associative array (even reading) may cause
> errors due to realloc? Or does the underlying assoc array only update when
> there's an assignment on a previously unaccessed domain key? Basically,
> can I feel free to operate on the domain in parallel, read from the assoc
> array in parallel without error, but expect to use something like a
> reader/writer lock for a general solution?
>
> I don't think there's an example in associative.chpl that shows being
> able to implicitly add to a domain via an assoc. array assignment -- unless
> I missed something, all the examples show adds to the domain first.
>
>
>
>
> On Fri, Apr 24, 2015 at 12:23 PM, Ben Harshbarger <[email protected]>
> wrote:
>
>> Hi Brian,
>>
>> In our implementation, operations on associative domains are parallel
>> safe by default. You can add and remove keys safely inside of parallel
>> constructs.
>> You can toggle this safety when declaring the associative domain:
>> var dom : domain(int, parSafe=false);
>>
>> Associative arrays are not as safe, and one should take precautions
>> when using them in parallel. For example, if one tasks reassigns a domain
>> while another task is iterating over the array, you may very well encounter
>> errors in your program.
>>
>> With enough parallelism and a large enough problem size, I think your
>> test would expose a similar issue where the underlying array inside the
>> domain is reallocated. Once the underlying array is resized to store more
>> indices, references to those elements (returned from the array access) are
>> no longer valid.
>>
>> Also, if you haven't found it already, the associative domain/array
>> primer may be a useful resource (though not for the question of parallel
>> safety):
>>
>> https://github.com/chapel-lang/chapel/blob/release/1.11/test/release/examples/primers/associative.chpl
>>
>> -Ben
>>
>> From: Brian Guarraci <[email protected]>
>> Date: Friday, April 24, 2015 at 9:43 AM
>> To: "[email protected]" <
>> [email protected]>
>> Subject: Re: [Chapel-developers] associative array questions
>>
>> I found the answer to my first question in the reference (I missed
>> this before):
>>
>> 20.3.2 Associative Array Indexing
>>
>> For associative arrays such assignment will add the index to the array’s
>> domain, and the array can be indexed with the newly added indices:
>>
>> On Fri, Apr 24, 2015 at 8:01 AM, Brian Guarraci <[email protected]> wrote:
>>
>>> Hi,
>>>
>>> I have a few questions that don't seem covered in the docs:
>>>
>>> 1) The associative array example seems to imply that you need to first
>>> update the domain before you use the key as an accessor, however I've
>>> empirically found that you can do the assignment and the key gets added.
>>> Is this expected behavior?
>>>
>>> 2) Regarding concurrency, it seems intuitively the case that
>>> operations on an associate domain and the array should be thread / task
>>> safe, but i'm curious if there are any precautions I should be taking.
>>> I've empirically found that it seems to be safe, but wonder what the
>>> official guidance is.
>>>
>>> Here's a little snippet I used to test some ideas:
>>>
>>> use Random;
>>>
>>> var Indices: domain(real);
>>> var Entries: [Indices] real;
>>>
>>> proc writeEntryForIndex(idx: real) {
>>> writeln("key: ", idx, " value: ", Entries[idx]);
>>> }
>>>
>>> proc dumpEntries() {
>>> writeln('dumping entries');
>>> for idx in Indices.sorted() {
>>> writeEntryForIndex(idx);
>>> }
>>> writeln();
>>> }
>>>
>>> // add w/o previously updating the domain
>>> Entries[0] = 2.71828;
>>> dumpEntries();
>>>
>>> Entries[1] = 3.141592;
>>> dumpEntries();
>>>
>>> // add a bunch of keys in parallel
>>> coforall i in 1..here.maxTaskPar {
>>> var randStream: RandomStream = new RandomStream();
>>> for j in 1..10 {
>>> var key = randStream.getNext();
>>> var value = randStream.getNext();
>>> writeln("task #", j, " adding key: ", key, " value: ", value);
>>> Entries[key] = value;
>>> }
>>> }
>>>
>>> dumpEntries();
>>>
>>>
>>
>
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Chapel-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-developers
