Re: [Chapel-developers] Fwd: question about data distribution

[Hui Zhang]

Hmmm, I've verified your words, the compiler does simply do a load
operation instead of some complex wide reference. So all the constants and
params are automatically replicated on each locale right?

Thanks :)

On Tue, Aug 8, 2017 at 5:01 PM, David Iten  wrote:

> Hi Hui,
>
> What I believe you're seeing is that the compiler does an optimization
> where global constants are automatically replicated so they can be accessed
> locally from any locale.  Since the tuple was already replicated by the
> compiler, using it with the Replicated distribution adds some overhead for
> the distribution, but with no added benefit of avoiding communication.
>
> If the tuple was a variable instead of a constant, this compiler
> optimization would not apply, and you would see a pretty big benefit from
> the Replicated distribution.
>
> David
>
>
> On 8/8/17 12:21 PM, Hui Zhang wrote:
>
> Hello, David
>
> I've tried it and it works. However, the performance of using repVar seems
> to be not that good (honestly, really bad). I was gonna use it to improve
> the locality and so the performance, but it turned out to be a retroaction.
> Is it the fact or I'm using it incorrectly? I simply replicate a constant
> tuple which will be accessed by all locales in the execution.
>
> thanks
>
> On Tue, Aug 8, 2017 at 1:08 PM, Hui Zhang 
> wrote:
>
>> Thanks so much!
>>
>> On Tue, Aug 8, 2017 at 12:46 PM, David Iten  wrote:
>>
>>> Hi Hui,
>>>
>>> Yes, a replicated variable can replicate any type:
>>>
>>> // a replicated heterogeneous tuple
>>> var repTuple: [rcDomain] (int, real, complex);
>>>
>>> // the local complex component
>>> writeln(repTuple[1](3));
>>>
>>> David
>>>
>>>
>>> On 8/8/17 11:38 AM, Hui Zhang wrote:
>>>
>>> Thank you David, that's exactly what I need! One question: the type of
>>> repVar can be anything right? what if I need a tuple? how can I access the
>>> local copy?
>>>
>>>
>>>
>>> On Tue, Aug 8, 2017 at 12:16 PM, David Iten  wrote:
>>>
 Hi Hui,

 You can use the 'ReplicatedDist' distribution to get a copy per
 locale.  There is a module 'UtilReplicatedVar' to simplify its use.

 http://chapel.cray.com/docs/latest/modules/standard/UtilRepl
 icatedVar.html
 http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html

 use UtilReplicatedVar;

 var regularInt = 42;

 // rcDomain is declared in UtilReplicatedVar.  It maps one value to
 each locale
 var repVar: [rcDomain] int;

 // assign 42 to the replicated var on all locales
 rcReplicate(repVar, regularInt);

 // access the local copy of the replicated var
 // the first form must use 1 as the index
 repVar[1] = 0;
 writeln(rcLocal(repVar));

 // access a remote copy
 rcRemote(repVar, remoteLocale);

 David



 On 8/8/17 10:36 AM, Hui Zhang wrote:


 -- Forwarded message --
 From: Hui Zhang 
 Date: Tue, Aug 8, 2017 at 11:36 AM
 Subject: Re: [Chapel-developers] question about data distribution
 To: Tom MacDonald 
 Cc: Brad Chamberlain 


 Hello,

 I'm wondering that is there an easy way to make a copy of a global
 variable on each locale so that later each locale will directly access its
 local copy instead of accessing the original variable stored in locale0 ?

 thanks

 On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald 
 wrote:

> We have all made this kind of mistake Hui.  No need to feel bad.
> Happy to hear you discovered the problem.
>
>
>
> Tom MacDOnald
>
>
>
> *From:* Hui Zhang [mailto:wayne.huizh...@gmail.com]
> *Sent:* Monday, August 07, 2017 4:56 PM
> *To:* Brad Chamberlain 
> *Cc:* Chapel Sourceforge Developers List <
> chapel-developers@lists.sourceforge.net>
> *Subject:* Re: [Chapel-developers] question about data distribution
>
>
>
> Sorry, I made a very basic mistake that made me feel so dumm： I mixed
> up the name of distributed and non-distributed domains that I used, which
> caused all the confusion and proved that easily-distinguishable naming is
> such an important thing, again. :P
>
> Thanks Brad, everything works as it should be.
>
>
>
> On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain 
> wrote:
>
>
> This looks like a bug to me, though maybe someone else will remember
> something that I've forgotten (as I mentioned in my previous message,
> 'local' clauses are a little surprising sometimes -- you may have seen 
> that
> we're actually hoping to retire them from the language to focus on more
> data-centric approaches).
>
> In either case, I think this is worth opening a 

Re: [Chapel-developers] Fwd: question about data distribution

[David Iten]

Hi Hui,

What I believe you're seeing is that the compiler does an optimization 
where global constants are automatically replicated so they can be 
accessed locally from any locale.  Since the tuple was already 
replicated by the compiler, using it with the Replicated distribution 
adds some overhead for the distribution, but with no added benefit of 
avoiding communication.


If the tuple was a variable instead of a constant, this compiler 
optimization would not apply, and you would see a pretty big benefit 
from the Replicated distribution.


David

On 8/8/17 12:21 PM, Hui Zhang wrote:

Hello, David

I've tried it and it works. However, the performance of using repVar 
seems to be not that good (honestly, really bad). I was gonna use it 
to improve the locality and so the performance, but it turned out to 
be a retroaction. Is it the fact or I'm using it incorrectly? I simply 
replicate a constant tuple which will be accessed by all locales in 
the execution.


thanks

On Tue, Aug 8, 2017 at 1:08 PM, Hui Zhang > wrote:


Thanks so much!

On Tue, Aug 8, 2017 at 12:46 PM, David Iten > wrote:

Hi Hui,

Yes, a replicated variable can replicate any type:

// a replicated heterogeneous tuple
var repTuple: [rcDomain] (int, real, complex);

// the local complex component
writeln(repTuple[1](3));

David


On 8/8/17 11:38 AM, Hui Zhang wrote:

Thank you David, that's exactly what I need! One question:
the type of repVar can be anything right? what if I need a
tuple? how can I access the local copy?



On Tue, Aug 8, 2017 at 12:16 PM, David Iten > wrote:

Hi Hui,

You can use the 'ReplicatedDist' distribution to get a
copy per locale.  There is a module 'UtilReplicatedVar'
to simplify its use.


http://chapel.cray.com/docs/latest/modules/standard/UtilReplicatedVar.html


http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html



use UtilReplicatedVar;

var regularInt = 42;

// rcDomain is declared in UtilReplicatedVar.  It maps
one value to each locale
var repVar: [rcDomain] int;

// assign 42 to the replicated var on all locales
rcReplicate(repVar, regularInt);

// access the local copy of the replicated var
// the first form must use 1 as the index
repVar[1] = 0;
writeln(rcLocal(repVar));

// access a remote copy
rcRemote(repVar, remoteLocale);

David



On 8/8/17 10:36 AM, Hui Zhang wrote:


-- Forwarded message --
From: *Hui Zhang* >
Date: Tue, Aug 8, 2017 at 11:36 AM
Subject: Re: [Chapel-developers] question about data
distribution
To: Tom MacDonald >
Cc: Brad Chamberlain >


Hello,

I'm wondering that is there an easy way to make a copy
of a global variable on each locale so that later each
locale will directly access its local copy instead of
accessing the original variable stored in locale0 ?

thanks

On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald
> wrote:

We have all made this kind of mistake Hui. No need
to feel bad. Happy to hear you discovered the problem.

Tom MacDOnald

*From:*Hui Zhang [mailto:wayne.huizh...@gmail.com
]
*Sent:* Monday, August 07, 2017 4:56 PM
*To:* Brad Chamberlain >
*Cc:* Chapel Sourceforge Developers List
>
*Subject:* Re: [Chapel-developers] question about
data distribution

Sorry, I made a very basic mistake that made me feel
so dumm：I mixed up the name of distributed and
non-distributed domains that I used, which caused
all the confusion and proved that
easily-distinguishable naming is such an important
thing, again. :P

Re: [Chapel-developers] Fwd: question about data distribution

[Hui Zhang]

Hello, Brad

Good point! Didn't know StackOverflow is also followed by the Chapel team,
will post this question on that.
Thanks

On Tue, Aug 8, 2017 at 1:44 PM, Brad Chamberlain  wrote:

>
> Hi Hui --
>
> Just as a general note, this type of "how do I do xyz in Chapel?" or "can
> I do xyz in Chapel?" style questions would be awesome to see posted to
> StackOverflow with a Chapel tag (which will cause us to wake up and answer
> them there) so that they will be helpful to other users in the future as
> well (without dealing with reading through poorly managed mailing list
> archives).  If for future questions you can think about "is this something
> that's generally useful for the public ("How do I do xyz in Chapel?") vs.
> not ("Here's a question very specific to my use case..."), that'd be
> great.  You could even retroactively post questions like this one about
> replicating variables to StackOverflow and David could re-answer it there
> to get more eyes on it.
>
> Thanks,
> -Brad
>
>
>
> On Tue, 8 Aug 2017, Hui Zhang wrote:
>
> Thank you David, that's exactly what I need! One question: the type of
>> repVar can be anything right? what if I need a tuple? how can I access the
>> local copy?
>>
>>
>>
>> On Tue, Aug 8, 2017 at 12:16 PM, David Iten  wrote:
>>
>> Hi Hui,
>>>
>>> You can use the 'ReplicatedDist' distribution to get a copy per locale.
>>> There is a module 'UtilReplicatedVar' to simplify its use.
>>>
>>> http://chapel.cray.com/docs/latest/modules/standard/UtilRepl
>>> icatedVar.html
>>> http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html
>>>
>>> use UtilReplicatedVar;
>>>
>>> var regularInt = 42;
>>>
>>> // rcDomain is declared in UtilReplicatedVar.  It maps one value to each
>>> locale
>>> var repVar: [rcDomain] int;
>>>
>>> // assign 42 to the replicated var on all locales
>>> rcReplicate(repVar, regularInt);
>>>
>>> // access the local copy of the replicated var
>>> // the first form must use 1 as the index
>>> repVar[1] = 0;
>>> writeln(rcLocal(repVar));
>>>
>>> // access a remote copy
>>> rcRemote(repVar, remoteLocale);
>>>
>>> David
>>>
>>>
>>>
>>> On 8/8/17 10:36 AM, Hui Zhang wrote:
>>>
>>>
>>> -- Forwarded message --
>>> From: Hui Zhang 
>>> Date: Tue, Aug 8, 2017 at 11:36 AM
>>> Subject: Re: [Chapel-developers] question about data distribution
>>> To: Tom MacDonald 
>>> Cc: Brad Chamberlain 
>>>
>>>
>>> Hello,
>>>
>>> I'm wondering that is there an easy way to make a copy of a global
>>> variable on each locale so that later each locale will directly access
>>> its
>>> local copy instead of accessing the original variable stored in locale0 ?
>>>
>>> thanks
>>>
>>> On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald 
>>> wrote:
>>>
>>> We have all made this kind of mistake Hui.  No need to feel bad.  Happy
 to hear you discovered the problem.



 Tom MacDOnald



 *From:* Hui Zhang [mailto:wayne.huizh...@gmail.com]
 *Sent:* Monday, August 07, 2017 4:56 PM
 *To:* Brad Chamberlain 
 *Cc:* Chapel Sourceforge Developers List 
 *Subject:* Re: [Chapel-developers] question about data distribution




 Sorry, I made a very basic mistake that made me feel so dumm： I mixed up
 the name of distributed and non-distributed domains that I used, which
 caused all the confusion and proved that easily-distinguishable naming
 is
 such an important thing, again. :P

 Thanks Brad, everything works as it should be.



 On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain 
 wrote:


 This looks like a bug to me, though maybe someone else will remember
 something that I've forgotten (as I mentioned in my previous message,
 'local' clauses are a little surprising sometimes -- you may have seen
 that
 we're actually hoping to retire them from the language to focus on more
 data-centric approaches).

 In either case, I think this is worth opening a GitHub issue for.

 -Brad





 On Mon, 7 Aug 2017, Hui Zhang wrote:

 update:

 After playing for a little bit, I'm more confused now:
 if I have a distributed domain D, and array B declared on D, the
 following
 code 1 will work and 2 will fail in runtime with an error "cannot access
 remote data in local block", why is that?
 1. work
 forall b in B {
local {
 b = here.locale.id;
}
 }

 2. fail
 forall d in D {
 local {
  B[d] = here.locale.id;
 }
 }



 On Mon, Aug 7, 2017 at 1:06 PM, Hui Zhang 
 wrote:

 Hello,

 I have a question about the data distribution in Chapel, for example we
 

Re: [Chapel-developers] Fwd: question about data distribution

[Brad Chamberlain]

Hi Hui --

Just as a general note, this type of "how do I do xyz in Chapel?" or "can 
I do xyz in Chapel?" style questions would be awesome to see posted to 
StackOverflow with a Chapel tag (which will cause us to wake up and answer 
them there) so that they will be helpful to other users in the future as 
well (without dealing with reading through poorly managed mailing list 
archives).  If for future questions you can think about "is this something 
that's generally useful for the public ("How do I do xyz in Chapel?") vs. 
not ("Here's a question very specific to my use case..."), that'd be 
great.  You could even retroactively post questions like this one about 
replicating variables to StackOverflow and David could re-answer it there 
to get more eyes on it.


Thanks,
-Brad


On Tue, 8 Aug 2017, Hui Zhang wrote:


Thank you David, that's exactly what I need! One question: the type of
repVar can be anything right? what if I need a tuple? how can I access the
local copy?



On Tue, Aug 8, 2017 at 12:16 PM, David Iten  wrote:


Hi Hui,

You can use the 'ReplicatedDist' distribution to get a copy per locale.
There is a module 'UtilReplicatedVar' to simplify its use.

http://chapel.cray.com/docs/latest/modules/standard/UtilReplicatedVar.html
http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html

use UtilReplicatedVar;

var regularInt = 42;

// rcDomain is declared in UtilReplicatedVar.  It maps one value to each
locale
var repVar: [rcDomain] int;

// assign 42 to the replicated var on all locales
rcReplicate(repVar, regularInt);

// access the local copy of the replicated var
// the first form must use 1 as the index
repVar[1] = 0;
writeln(rcLocal(repVar));

// access a remote copy
rcRemote(repVar, remoteLocale);

David



On 8/8/17 10:36 AM, Hui Zhang wrote:


-- Forwarded message --
From: Hui Zhang 
Date: Tue, Aug 8, 2017 at 11:36 AM
Subject: Re: [Chapel-developers] question about data distribution
To: Tom MacDonald 
Cc: Brad Chamberlain 


Hello,

I'm wondering that is there an easy way to make a copy of a global
variable on each locale so that later each locale will directly access its
local copy instead of accessing the original variable stored in locale0 ?

thanks

On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald  wrote:


We have all made this kind of mistake Hui.  No need to feel bad.  Happy
to hear you discovered the problem.



Tom MacDOnald



*From:* Hui Zhang [mailto:wayne.huizh...@gmail.com]
*Sent:* Monday, August 07, 2017 4:56 PM
*To:* Brad Chamberlain 
*Cc:* Chapel Sourceforge Developers List 
*Subject:* Re: [Chapel-developers] question about data distribution



Sorry, I made a very basic mistake that made me feel so dumm： I mixed up
the name of distributed and non-distributed domains that I used, which
caused all the confusion and proved that easily-distinguishable naming is
such an important thing, again. :P

Thanks Brad, everything works as it should be.



On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain  wrote:


This looks like a bug to me, though maybe someone else will remember
something that I've forgotten (as I mentioned in my previous message,
'local' clauses are a little surprising sometimes -- you may have seen that
we're actually hoping to retire them from the language to focus on more
data-centric approaches).

In either case, I think this is worth opening a GitHub issue for.

-Brad





On Mon, 7 Aug 2017, Hui Zhang wrote:

update:

After playing for a little bit, I'm more confused now:
if I have a distributed domain D, and array B declared on D, the following
code 1 will work and 2 will fail in runtime with an error "cannot access
remote data in local block", why is that?
1. work
forall b in B {
   local {
b = here.locale.id;
   }
}

2. fail
forall d in D {
local {
 B[d] = here.locale.id;
}
}



On Mon, Aug 7, 2017 at 1:06 PM, Hui Zhang 
wrote:

Hello,

I have a question about the data distribution in Chapel, for example we
have:

const D = {1..8} dmapped Block({1..8});
var B: [D] real;
forall d in D {
   var A : 8*real;
   foo(A, B[d]);
}

So in multilocale, even though the domain D is distributed,​ the variable
A declared in each iteration is still allocated on node0 right? which
means
we can not add "local"  clause around the loop body since foo needs to
access A from other nodes which requires communication, like:
local {
var A: 8*real;
foo(A, B[d]);
}  ​

or

var A: 8*real;
local {
 foo(A, B[d]);
}


Further, is it true that the only ways to put data on nodes other than 0
are:
1. use explict "on" clause, everything inside it will be allocated on the
specific locale
2. use distributed domain: piece of the data will be allocated on each
locale.
?

Thanks
--
Best regards


Hui Zhang




--
Best regards


Hui 

Re: [Chapel-developers] Fwd: question about data distribution

[Hui Zhang]

Hello, David

I've tried it and it works. However, the performance of using repVar seems
to be not that good (honestly, really bad). I was gonna use it to improve
the locality and so the performance, but it turned out to be a retroaction.
Is it the fact or I'm using it incorrectly? I simply replicate a constant
tuple which will be accessed by all locales in the execution.

thanks

On Tue, Aug 8, 2017 at 1:08 PM, Hui Zhang  wrote:

> Thanks so much!
>
> On Tue, Aug 8, 2017 at 12:46 PM, David Iten  wrote:
>
>> Hi Hui,
>>
>> Yes, a replicated variable can replicate any type:
>>
>> // a replicated heterogeneous tuple
>> var repTuple: [rcDomain] (int, real, complex);
>>
>> // the local complex component
>> writeln(repTuple[1](3));
>>
>> David
>>
>>
>> On 8/8/17 11:38 AM, Hui Zhang wrote:
>>
>> Thank you David, that's exactly what I need! One question: the type of
>> repVar can be anything right? what if I need a tuple? how can I access the
>> local copy?
>>
>>
>>
>> On Tue, Aug 8, 2017 at 12:16 PM, David Iten  wrote:
>>
>>> Hi Hui,
>>>
>>> You can use the 'ReplicatedDist' distribution to get a copy per locale.
>>> There is a module 'UtilReplicatedVar' to simplify its use.
>>>
>>> http://chapel.cray.com/docs/latest/modules/standard/UtilRepl
>>> icatedVar.html
>>> http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html
>>>
>>> use UtilReplicatedVar;
>>>
>>> var regularInt = 42;
>>>
>>> // rcDomain is declared in UtilReplicatedVar.  It maps one value to each
>>> locale
>>> var repVar: [rcDomain] int;
>>>
>>> // assign 42 to the replicated var on all locales
>>> rcReplicate(repVar, regularInt);
>>>
>>> // access the local copy of the replicated var
>>> // the first form must use 1 as the index
>>> repVar[1] = 0;
>>> writeln(rcLocal(repVar));
>>>
>>> // access a remote copy
>>> rcRemote(repVar, remoteLocale);
>>>
>>> David
>>>
>>>
>>>
>>> On 8/8/17 10:36 AM, Hui Zhang wrote:
>>>
>>>
>>> -- Forwarded message --
>>> From: Hui Zhang 
>>> Date: Tue, Aug 8, 2017 at 11:36 AM
>>> Subject: Re: [Chapel-developers] question about data distribution
>>> To: Tom MacDonald 
>>> Cc: Brad Chamberlain 
>>>
>>>
>>> Hello,
>>>
>>> I'm wondering that is there an easy way to make a copy of a global
>>> variable on each locale so that later each locale will directly access its
>>> local copy instead of accessing the original variable stored in locale0 ?
>>>
>>> thanks
>>>
>>> On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald 
>>> wrote:
>>>
 We have all made this kind of mistake Hui.  No need to feel bad.  Happy
 to hear you discovered the problem.



 Tom MacDOnald



 *From:* Hui Zhang [mailto:wayne.huizh...@gmail.com]
 *Sent:* Monday, August 07, 2017 4:56 PM
 *To:* Brad Chamberlain 
 *Cc:* Chapel Sourceforge Developers List 
 *Subject:* Re: [Chapel-developers] question about data distribution



 Sorry, I made a very basic mistake that made me feel so dumm： I mixed
 up the name of distributed and non-distributed domains that I used, which
 caused all the confusion and proved that easily-distinguishable naming is
 such an important thing, again. :P

 Thanks Brad, everything works as it should be.



 On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain 
 wrote:


 This looks like a bug to me, though maybe someone else will remember
 something that I've forgotten (as I mentioned in my previous message,
 'local' clauses are a little surprising sometimes -- you may have seen that
 we're actually hoping to retire them from the language to focus on more
 data-centric approaches).

 In either case, I think this is worth opening a GitHub issue for.

 -Brad





 On Mon, 7 Aug 2017, Hui Zhang wrote:

 update:

 After playing for a little bit, I'm more confused now:
 if I have a distributed domain D, and array B declared on D, the
 following
 code 1 will work and 2 will fail in runtime with an error "cannot access
 remote data in local block", why is that?
 1. work
 forall b in B {
local {
 b = here.locale.id;
}
 }

 2. fail
 forall d in D {
 local {
  B[d] = here.locale.id;
 }
 }



 On Mon, Aug 7, 2017 at 1:06 PM, Hui Zhang 
 wrote:

 Hello,

 I have a question about the data distribution in Chapel, for example we
 have:

 const D = {1..8} dmapped Block({1..8});
 var B: [D] real;
 forall d in D {
var A : 8*real;
foo(A, B[d]);
 }

 So in multilocale, even though the domain D is distributed,​ the
 

Re: [Chapel-developers] Fwd: question about data distribution

[Hui Zhang]

Thanks so much!

On Tue, Aug 8, 2017 at 12:46 PM, David Iten  wrote:

> Hi Hui,
>
> Yes, a replicated variable can replicate any type:
>
> // a replicated heterogeneous tuple
> var repTuple: [rcDomain] (int, real, complex);
>
> // the local complex component
> writeln(repTuple[1](3));
>
> David
>
>
> On 8/8/17 11:38 AM, Hui Zhang wrote:
>
> Thank you David, that's exactly what I need! One question: the type of
> repVar can be anything right? what if I need a tuple? how can I access the
> local copy?
>
>
>
> On Tue, Aug 8, 2017 at 12:16 PM, David Iten  wrote:
>
>> Hi Hui,
>>
>> You can use the 'ReplicatedDist' distribution to get a copy per locale.
>> There is a module 'UtilReplicatedVar' to simplify its use.
>>
>> http://chapel.cray.com/docs/latest/modules/standard/UtilRepl
>> icatedVar.html
>> http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html
>>
>> use UtilReplicatedVar;
>>
>> var regularInt = 42;
>>
>> // rcDomain is declared in UtilReplicatedVar.  It maps one value to each
>> locale
>> var repVar: [rcDomain] int;
>>
>> // assign 42 to the replicated var on all locales
>> rcReplicate(repVar, regularInt);
>>
>> // access the local copy of the replicated var
>> // the first form must use 1 as the index
>> repVar[1] = 0;
>> writeln(rcLocal(repVar));
>>
>> // access a remote copy
>> rcRemote(repVar, remoteLocale);
>>
>> David
>>
>>
>>
>> On 8/8/17 10:36 AM, Hui Zhang wrote:
>>
>>
>> -- Forwarded message --
>> From: Hui Zhang 
>> Date: Tue, Aug 8, 2017 at 11:36 AM
>> Subject: Re: [Chapel-developers] question about data distribution
>> To: Tom MacDonald 
>> Cc: Brad Chamberlain 
>>
>>
>> Hello,
>>
>> I'm wondering that is there an easy way to make a copy of a global
>> variable on each locale so that later each locale will directly access its
>> local copy instead of accessing the original variable stored in locale0 ?
>>
>> thanks
>>
>> On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald 
>> wrote:
>>
>>> We have all made this kind of mistake Hui.  No need to feel bad.  Happy
>>> to hear you discovered the problem.
>>>
>>>
>>>
>>> Tom MacDOnald
>>>
>>>
>>>
>>> *From:* Hui Zhang [mailto:wayne.huizh...@gmail.com]
>>> *Sent:* Monday, August 07, 2017 4:56 PM
>>> *To:* Brad Chamberlain 
>>> *Cc:* Chapel Sourceforge Developers List >> eforge.net>
>>> *Subject:* Re: [Chapel-developers] question about data distribution
>>>
>>>
>>>
>>> Sorry, I made a very basic mistake that made me feel so dumm： I mixed
>>> up the name of distributed and non-distributed domains that I used, which
>>> caused all the confusion and proved that easily-distinguishable naming is
>>> such an important thing, again. :P
>>>
>>> Thanks Brad, everything works as it should be.
>>>
>>>
>>>
>>> On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain  wrote:
>>>
>>>
>>> This looks like a bug to me, though maybe someone else will remember
>>> something that I've forgotten (as I mentioned in my previous message,
>>> 'local' clauses are a little surprising sometimes -- you may have seen that
>>> we're actually hoping to retire them from the language to focus on more
>>> data-centric approaches).
>>>
>>> In either case, I think this is worth opening a GitHub issue for.
>>>
>>> -Brad
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 7 Aug 2017, Hui Zhang wrote:
>>>
>>> update:
>>>
>>> After playing for a little bit, I'm more confused now:
>>> if I have a distributed domain D, and array B declared on D, the
>>> following
>>> code 1 will work and 2 will fail in runtime with an error "cannot access
>>> remote data in local block", why is that?
>>> 1. work
>>> forall b in B {
>>>local {
>>> b = here.locale.id;
>>>}
>>> }
>>>
>>> 2. fail
>>> forall d in D {
>>> local {
>>>  B[d] = here.locale.id;
>>> }
>>> }
>>>
>>>
>>>
>>> On Mon, Aug 7, 2017 at 1:06 PM, Hui Zhang 
>>> wrote:
>>>
>>> Hello,
>>>
>>> I have a question about the data distribution in Chapel, for example we
>>> have:
>>>
>>> const D = {1..8} dmapped Block({1..8});
>>> var B: [D] real;
>>> forall d in D {
>>>var A : 8*real;
>>>foo(A, B[d]);
>>> }
>>>
>>> So in multilocale, even though the domain D is distributed,​ the variable
>>> A declared in each iteration is still allocated on node0 right? which
>>> means
>>> we can not add "local"  clause around the loop body since foo needs to
>>> access A from other nodes which requires communication, like:
>>> local {
>>> var A: 8*real;
>>> foo(A, B[d]);
>>> }  ​
>>>
>>> or
>>>
>>> var A: 8*real;
>>> local {
>>>  foo(A, B[d]);
>>> }
>>>
>>>
>>> Further, is it true that the only ways to put data on nodes other than 0
>>> are:
>>> 1. use explict "on" clause, everything inside it will be allocated on the
>>> specific locale
>>> 2. use distributed domain: piece of the data will be allocated on each
>>> 

Re: [Chapel-developers] Fwd: question about data distribution

[Hui Zhang]

Thank you David, that's exactly what I need! One question: the type of
repVar can be anything right? what if I need a tuple? how can I access the
local copy?



On Tue, Aug 8, 2017 at 12:16 PM, David Iten  wrote:

> Hi Hui,
>
> You can use the 'ReplicatedDist' distribution to get a copy per locale.
> There is a module 'UtilReplicatedVar' to simplify its use.
>
> http://chapel.cray.com/docs/latest/modules/standard/UtilReplicatedVar.html
> http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html
>
> use UtilReplicatedVar;
>
> var regularInt = 42;
>
> // rcDomain is declared in UtilReplicatedVar.  It maps one value to each
> locale
> var repVar: [rcDomain] int;
>
> // assign 42 to the replicated var on all locales
> rcReplicate(repVar, regularInt);
>
> // access the local copy of the replicated var
> // the first form must use 1 as the index
> repVar[1] = 0;
> writeln(rcLocal(repVar));
>
> // access a remote copy
> rcRemote(repVar, remoteLocale);
>
> David
>
>
>
> On 8/8/17 10:36 AM, Hui Zhang wrote:
>
>
> -- Forwarded message --
> From: Hui Zhang 
> Date: Tue, Aug 8, 2017 at 11:36 AM
> Subject: Re: [Chapel-developers] question about data distribution
> To: Tom MacDonald 
> Cc: Brad Chamberlain 
>
>
> Hello,
>
> I'm wondering that is there an easy way to make a copy of a global
> variable on each locale so that later each locale will directly access its
> local copy instead of accessing the original variable stored in locale0 ?
>
> thanks
>
> On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald  wrote:
>
>> We have all made this kind of mistake Hui.  No need to feel bad.  Happy
>> to hear you discovered the problem.
>>
>>
>>
>> Tom MacDOnald
>>
>>
>>
>> *From:* Hui Zhang [mailto:wayne.huizh...@gmail.com]
>> *Sent:* Monday, August 07, 2017 4:56 PM
>> *To:* Brad Chamberlain 
>> *Cc:* Chapel Sourceforge Developers List > eforge.net>
>> *Subject:* Re: [Chapel-developers] question about data distribution
>>
>>
>>
>> Sorry, I made a very basic mistake that made me feel so dumm： I mixed up
>> the name of distributed and non-distributed domains that I used, which
>> caused all the confusion and proved that easily-distinguishable naming is
>> such an important thing, again. :P
>>
>> Thanks Brad, everything works as it should be.
>>
>>
>>
>> On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain  wrote:
>>
>>
>> This looks like a bug to me, though maybe someone else will remember
>> something that I've forgotten (as I mentioned in my previous message,
>> 'local' clauses are a little surprising sometimes -- you may have seen that
>> we're actually hoping to retire them from the language to focus on more
>> data-centric approaches).
>>
>> In either case, I think this is worth opening a GitHub issue for.
>>
>> -Brad
>>
>>
>>
>>
>>
>> On Mon, 7 Aug 2017, Hui Zhang wrote:
>>
>> update:
>>
>> After playing for a little bit, I'm more confused now:
>> if I have a distributed domain D, and array B declared on D, the following
>> code 1 will work and 2 will fail in runtime with an error "cannot access
>> remote data in local block", why is that?
>> 1. work
>> forall b in B {
>>local {
>> b = here.locale.id;
>>}
>> }
>>
>> 2. fail
>> forall d in D {
>> local {
>>  B[d] = here.locale.id;
>> }
>> }
>>
>>
>>
>> On Mon, Aug 7, 2017 at 1:06 PM, Hui Zhang 
>> wrote:
>>
>> Hello,
>>
>> I have a question about the data distribution in Chapel, for example we
>> have:
>>
>> const D = {1..8} dmapped Block({1..8});
>> var B: [D] real;
>> forall d in D {
>>var A : 8*real;
>>foo(A, B[d]);
>> }
>>
>> So in multilocale, even though the domain D is distributed,​ the variable
>> A declared in each iteration is still allocated on node0 right? which
>> means
>> we can not add "local"  clause around the loop body since foo needs to
>> access A from other nodes which requires communication, like:
>> local {
>> var A: 8*real;
>> foo(A, B[d]);
>> }  ​
>>
>> or
>>
>> var A: 8*real;
>> local {
>>  foo(A, B[d]);
>> }
>>
>>
>> Further, is it true that the only ways to put data on nodes other than 0
>> are:
>> 1. use explict "on" clause, everything inside it will be allocated on the
>> specific locale
>> 2. use distributed domain: piece of the data will be allocated on each
>> locale.
>> ?
>>
>> Thanks
>> --
>> Best regards
>>
>>
>> Hui Zhang
>>
>>
>>
>>
>> --
>> Best regards
>>
>>
>> Hui Zhang
>>
>>
>>
>>
>> --
>>
>> Best regards
>>
>>
>> Hui Zhang
>>
>
>
>
> --
> Best regards
>
>
> Hui Zhang
>
>
>
> --
> Best regards
>
>
> Hui Zhang
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> ___
> 

Re: [Chapel-developers] Fwd: question about data distribution

[David Iten]

Hi Hui,

You can use the 'ReplicatedDist' distribution to get a copy per locale.  
There is a module 'UtilReplicatedVar' to simplify its use.


http://chapel.cray.com/docs/latest/modules/standard/UtilReplicatedVar.html
http://chapel.cray.com/docs/latest/modules/dists/ReplicatedDist.html

use UtilReplicatedVar;

var regularInt = 42;

// rcDomain is declared in UtilReplicatedVar.  It maps one value to each 
locale

var repVar: [rcDomain] int;

// assign 42 to the replicated var on all locales
rcReplicate(repVar, regularInt);

// access the local copy of the replicated var
// the first form must use 1 as the index
repVar[1] = 0;
writeln(rcLocal(repVar));

// access a remote copy
rcRemote(repVar, remoteLocale);

David


On 8/8/17 10:36 AM, Hui Zhang wrote:


-- Forwarded message --
From: *Hui Zhang* >

Date: Tue, Aug 8, 2017 at 11:36 AM
Subject: Re: [Chapel-developers] question about data distribution
To: Tom MacDonald >
Cc: Brad Chamberlain >


Hello,

I'm wondering that is there an easy way to make a copy of a global 
variable on each locale so that later each locale will directly access 
its local copy instead of accessing the original variable stored in 
locale0 ?


thanks

On Mon, Aug 7, 2017 at 6:19 PM, Tom MacDonald > wrote:


We have all made this kind of mistake Hui.  No need to feel bad. 
Happy to hear you discovered the problem.


Tom MacDOnald

*From:*Hui Zhang [mailto:wayne.huizh...@gmail.com
]
*Sent:* Monday, August 07, 2017 4:56 PM
*To:* Brad Chamberlain >
*Cc:* Chapel Sourceforge Developers List
>
*Subject:* Re: [Chapel-developers] question about data distribution

Sorry, I made a very basic mistake that made me feel so dumm：I
mixed up the name of distributed and non-distributed domains that
I used, which caused all the confusion and proved that
easily-distinguishable naming is such an important thing, again. :P

Thanks Brad, everything works as it should be.

On Mon, Aug 7, 2017 at 4:01 PM, Brad Chamberlain > wrote:


This looks like a bug to me, though maybe someone else will
remember something that I've forgotten (as I mentioned in my
previous message, 'local' clauses are a little surprising
sometimes -- you may have seen that we're actually hoping to
retire them from the language to focus on more data-centric
approaches).

In either case, I think this is worth opening a GitHub issue for.

-Brad





On Mon, 7 Aug 2017, Hui Zhang wrote:

update:

After playing for a little bit, I'm more confused now:
if I have a distributed domain D, and array B declared on
D, the following
code 1 will work and 2 will fail in runtime with an error
"cannot access
remote data in local block", why is that?
1. work
forall b in B {
   local {
b = here.locale.id ;
   }
}

2. fail
forall d in D {
local {
 B[d] = here.locale.id ;
}
}



On Mon, Aug 7, 2017 at 1:06 PM, Hui Zhang
> wrote:

Hello,

I have a question about the data distribution in
Chapel, for example we
have:

const D = {1..8} dmapped Block({1..8});
var B: [D] real;
forall d in D {
   var A : 8*real;
   foo(A, B[d]);
}

So in multilocale, even though the domain D is
distributed,​ the variable
A declared in each iteration is still allocated on
node0 right? which means
we can not add "local" clause around the loop body
since foo needs to
access A from other nodes which requires
communication, like:
local {
var A: 8*real;
foo(A, B[d]);
}  ​

or

var A: 8*real;
local {
 foo(A, B[d]);
}


Further, is it true that the only ways to put data on
nodes other than 0
are:
1. use explict "on" clause, everything inside it will