[petsc-users] VecGhost memory layout
Hi guys, I'm running into a bug that has made me question my understanding of memory layout in VecGhost. First, I remember reading somewhere before (in the manual or mailing list which I cannot find now) that the way they are internally organized is all local values followed by all ghost values. In other words ghost values are ALWAYS padded to the end of the array. Is this correct? Second, when I want to access both local and ghosted values, I do the following, VecGhostGetLocalForm(F, F_loc); VecGetArray(F_loc, F_loc_ptr); // do computation on F_loc_ptr VecRestoreArray(F_loc, F_loc_ptr); VecGhostRestoreLocalForm(F, F_loc); here I assume that in accessing F_loc_ptr, all indecies from [0, numLocal) are local values and the rest are ghost values. Once I'm done, I need every processor to update its ghost region and I call VecGhostUpdateBegin(F, INSERT_VALUES, SCATTER_FORWARD); VecGhostUpdateEnd(F, INSERT_VALUES, SCATTER_FORWARD); Is there any flaw in what I'm doing? Also, as a side question, if I call VecGetArray directly on F (and not F_loc) do I get junk values? Thanks, M
Re: [petsc-users] VecGhost memory layout
On Aug 8, 2013, at 2:56 PM, Mohammad Mirzadeh mirza...@gmail.com wrote: Hi guys, I'm running into a bug that has made me question my understanding of memory layout in VecGhost. First, I remember reading somewhere before (in the manual or mailing list which I cannot find now) that the way they are internally organized is all local values followed by all ghost values. In other words ghost values are ALWAYS padded to the end of the array. Is this correct? Yes Second, when I want to access both local and ghosted values, I do the following, VecGhostGetLocalForm(F, F_loc); VecGetArray(F_loc, F_loc_ptr); // do computation on F_loc_ptr VecRestoreArray(F_loc, F_loc_ptr); VecGhostRestoreLocalForm(F, F_loc); here I assume that in accessing F_loc_ptr, all indecies from [0, numLocal) are local values and the rest are ghost values. Once I'm done, I need every processor to update its ghost region and I call Good VecGhostUpdateBegin(F, INSERT_VALUES, SCATTER_FORWARD); VecGhostUpdateEnd(F, INSERT_VALUES, SCATTER_FORWARD); Is there any flaw in what I'm doing? No Also, as a side question, if I call VecGetArray directly on F (and not F_loc) do I get junk values? No, they actually share the same array so you will get the same values. Barry Thanks, M
Re: [petsc-users] VecGhost memory layout
Awesome. Thanks Barry for the quick response. On Thu, Aug 8, 2013 at 1:42 PM, Barry Smith bsm...@mcs.anl.gov wrote: On Aug 8, 2013, at 2:56 PM, Mohammad Mirzadeh mirza...@gmail.com wrote: Hi guys, I'm running into a bug that has made me question my understanding of memory layout in VecGhost. First, I remember reading somewhere before (in the manual or mailing list which I cannot find now) that the way they are internally organized is all local values followed by all ghost values. In other words ghost values are ALWAYS padded to the end of the array. Is this correct? Yes Second, when I want to access both local and ghosted values, I do the following, VecGhostGetLocalForm(F, F_loc); VecGetArray(F_loc, F_loc_ptr); // do computation on F_loc_ptr VecRestoreArray(F_loc, F_loc_ptr); VecGhostRestoreLocalForm(F, F_loc); here I assume that in accessing F_loc_ptr, all indecies from [0, numLocal) are local values and the rest are ghost values. Once I'm done, I need every processor to update its ghost region and I call Good VecGhostUpdateBegin(F, INSERT_VALUES, SCATTER_FORWARD); VecGhostUpdateEnd(F, INSERT_VALUES, SCATTER_FORWARD); Is there any flaw in what I'm doing? No Also, as a side question, if I call VecGetArray directly on F (and not F_loc) do I get junk values? No, they actually share the same array so you will get the same values. Barry Thanks, M