On Fri, Jan 6, 2023 at 9:37 AM Nicholas Arnold-Medabalimi < [email protected]> wrote:
> Hi Matt > > I made a typo on the line statVecV(offset) = <set to something> in my > example, I agree. (I wrote that offhand since the actual assignment is much > larger) I should be statVecV(offset+1) = <assignment> so I'm confident it's > not a 1 0 indexing thing. > > My question is more related to what is happening in the offsets. c0 and c1 > are pulled using DMplexgetheight stratum, so they are zero-indexed (which > is why I loop from c0 to (c1-1)). > > For the size inquiries. on processor 0 > Petsc VecGetSize(stateVec) 5390 > I need to see VecGetLocalSize() Matt > size(stateVecV) 2640 > > on processor 1 > Petsc VecGetSize 5390 > size(stateVecV) 2750 > > It's quite weird to me that processor one can have a positive offset that > is less than its starting ownership index (in the initial email output). > > Thanks for the assistance > Nicholas > > > On Fri, Jan 6, 2023 at 9:20 AM Matthew Knepley <[email protected]> wrote: > >> On Fri, Jan 6, 2023 at 2:28 AM Nicholas Arnold-Medabalimi < >> [email protected]> wrote: >> >>> Hi Petsc Users, >>> >>> I'm working with a dmplex system with a subsampled mesh distributed with >>> an overlap of 1. >>> >>> I'm encountering unusual situations when using VecGetOwnershipRange to >>> adjust the offset received from a global section. The logic of the >>> following code is first to get the offset needed to index a global vector >>> while still being able to check if it is an overlapped cell and skip if >>> needed while counting the owned cells. >>> >> >> >>> >>> call DMGetGlobalSection(dmplex,section,ierr) >>> call VecGetArrayF90(stateVec,stateVecV,ierr) >>> call VecGetOwnershipRange(stateVec,oStart,oEnd,ierr) >>> do i = c0, (c1-1) >>> >>> call PetscSectionGetOffset(section,i,offset,ierr) >>> write(*,*) "cell",i,"offset",offset,'oStart',oStart, offset-oStart >>> >>> if(offset<0) then >>> cycle >>> endif >>> offset=offset-oStart >>> plexcells=plexcells+1 >>> stateVecV(offset)= <set to something> enddo >>> >>> I'm noticing some very weird results that I've appended below. The >>> GetOffset documentation notes that a negative offset indicates an unowned >>> point (which I use to cycle). However, the offset subtraction with oStart >>> will yield an illegal index for the Vector access. I see that on the >>> documentation for GetOwnershipRange, it notes that this may be >>> "ill-defined" but I wanted to see if this is type of ill-defined I can >>> expect or there is just something terribly wrong with my PetscSection.(both >>> the Vec and Section were produced from DMPlexDistributeField so should by >>> definition have synchronized section information) I was wondering if there >>> is a possible output and/or the best way to index the vector. I'm thinking >>> of subtracting the offset of cell 0 perhaps? >>> >> >> Can you show your vector sizes? Are you sure it is not the fact that F90 >> arrays use 1-based indices, but these are 0-based offsets? >> >> Thanks, >> >> Matt >> >> >>> on rank 0 >>> >>> cell 0 offset 0 oStart 0 0 >>> cell 1 offset 55 oStart 0 55 >>> cell 2 offset 110 oStart 0 110 >>> cell 3 offset 165 oStart 0 165 >>> cell 4 offset 220 oStart 0 220 >>> cell 5 offset 275 oStart 0 275 >>> cell 6 offset 330 oStart 0 330 >>> cell 7 offset 385 oStart 0 385 >>> cell 8 offset 440 oStart 0 440 >>> cell 9 offset 495 oStart 0 495 >>> cell 10 offset 550 oStart 0 550 >>> cell 11 offset 605 oStart 0 605 >>> cell 12 offset 660 oStart 0 660 >>> cell 13 offset 715 oStart 0 715 >>> >>> and on rank one >>> cell 0 offset 2475 oStart 2640 -165 >>> cell 1 offset 2530 oStart 2640 -110 >>> cell 2 offset 2585 oStart 2640 -55 >>> cell 3 offset 2640 oStart 2640 0 >>> cell 4 offset 2695 oStart 2640 55 >>> cell 5 offset 2750 oStart 2640 110 >>> cell 6 offset 2805 oStart 2640 165 >>> cell 7 offset 2860 oStart 2640 220 >>> cell 8 offset 2915 oStart 2640 275 >>> cell 9 offset 2970 oStart 2640 330 >>> cell 10 offset 3025 oStart 2640 385 >>> cell 11 offset 3080 oStart 2640 440 >>> cell 12 offset 3135 oStart 2640 495 >>> cell 13 offset 3190 oStart 2640 550 >>> cell 14 offset 3245 oStart 2640 605 >>> cell 15 offset -771 oStart 2640 -3411 >>> >>> >>> Sincerely >>> Nicholas >>> >>> -- >>> Nicholas Arnold-Medabalimi >>> >>> Ph.D. Candidate >>> Computational Aeroscience Lab >>> University of Michigan >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> >> > > > -- > Nicholas Arnold-Medabalimi > > Ph.D. Candidate > Computational Aeroscience Lab > University of Michigan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
