[petsc-users] VecGhost memory layout

2013-08-08 Thread Mohammad Mirzadeh
Hi guys,

I'm running into a bug that has made me question my understanding of
memory layout in VecGhost.

First, I remember reading somewhere before (in the manual or mailing
list which I cannot find now) that the way they are internally
organized is all local values followed by all ghost values. In other
words ghost values are ALWAYS padded to the end of the array. Is this
correct?

Second, when I want to access both local and ghosted values, I do the following,

VecGhostGetLocalForm(F, F_loc);
VecGetArray(F_loc, F_loc_ptr);
// do computation on F_loc_ptr
VecRestoreArray(F_loc, F_loc_ptr);
VecGhostRestoreLocalForm(F, F_loc);

here I assume that in accessing F_loc_ptr, all indecies from [0,
numLocal) are local values and the rest are ghost values. Once I'm
done, I need every processor to update its ghost region and I call

VecGhostUpdateBegin(F, INSERT_VALUES, SCATTER_FORWARD);
VecGhostUpdateEnd(F, INSERT_VALUES, SCATTER_FORWARD);

Is there any flaw in what I'm doing? Also, as a side question, if I
call VecGetArray directly on F (and not F_loc) do I get junk values?

Thanks,
M


Re: [petsc-users] VecGhost memory layout

2013-08-08 Thread Barry Smith

On Aug 8, 2013, at 2:56 PM, Mohammad Mirzadeh mirza...@gmail.com wrote:

 Hi guys,
 
 I'm running into a bug that has made me question my understanding of
 memory layout in VecGhost.
 
 First, I remember reading somewhere before (in the manual or mailing
 list which I cannot find now) that the way they are internally
 organized is all local values followed by all ghost values. In other
 words ghost values are ALWAYS padded to the end of the array. Is this
 correct?
 

   Yes

 Second, when I want to access both local and ghosted values, I do the 
 following,
 
 VecGhostGetLocalForm(F, F_loc);
 VecGetArray(F_loc, F_loc_ptr);
 // do computation on F_loc_ptr
 VecRestoreArray(F_loc, F_loc_ptr);
 VecGhostRestoreLocalForm(F, F_loc);
 
 here I assume that in accessing F_loc_ptr, all indecies from [0,
 numLocal) are local values and the rest are ghost values. Once I'm
 done, I need every processor to update its ghost region and I call
 
   Good

 VecGhostUpdateBegin(F, INSERT_VALUES, SCATTER_FORWARD);
 VecGhostUpdateEnd(F, INSERT_VALUES, SCATTER_FORWARD);
 
 Is there any flaw in what I'm doing?

   No

 Also, as a side question, if I
 call VecGetArray directly on F (and not F_loc) do I get junk values?

No, they actually share the same array so you will get the same values.

   Barry

 
 Thanks,
 M



Re: [petsc-users] VecGhost memory layout

2013-08-08 Thread Mohammad Mirzadeh
Awesome. Thanks Barry for the quick response.

On Thu, Aug 8, 2013 at 1:42 PM, Barry Smith bsm...@mcs.anl.gov wrote:

 On Aug 8, 2013, at 2:56 PM, Mohammad Mirzadeh mirza...@gmail.com wrote:

 Hi guys,

 I'm running into a bug that has made me question my understanding of
 memory layout in VecGhost.

 First, I remember reading somewhere before (in the manual or mailing
 list which I cannot find now) that the way they are internally
 organized is all local values followed by all ghost values. In other
 words ghost values are ALWAYS padded to the end of the array. Is this
 correct?


Yes

 Second, when I want to access both local and ghosted values, I do the 
 following,

 VecGhostGetLocalForm(F, F_loc);
 VecGetArray(F_loc, F_loc_ptr);
 // do computation on F_loc_ptr
 VecRestoreArray(F_loc, F_loc_ptr);
 VecGhostRestoreLocalForm(F, F_loc);

 here I assume that in accessing F_loc_ptr, all indecies from [0,
 numLocal) are local values and the rest are ghost values. Once I'm
 done, I need every processor to update its ghost region and I call

Good

 VecGhostUpdateBegin(F, INSERT_VALUES, SCATTER_FORWARD);
 VecGhostUpdateEnd(F, INSERT_VALUES, SCATTER_FORWARD);

 Is there any flaw in what I'm doing?

No

 Also, as a side question, if I
 call VecGetArray directly on F (and not F_loc) do I get junk values?

 No, they actually share the same array so you will get the same values.

Barry


 Thanks,
 M