I created a for cycle where I call TSTrajectoryGetVecs, but only the 0 rank
seems to enter in this cycle and I do not know why.
I thought the following might be a solution, but it is not working as I would
like to, i.e. the final vector has the same local parts, a copy of the values
obtained with the 0-rank. How should I change this, please?
Vec U, partial, Uloc;
PetscScalar *Ui, *partlocal;
PetscInt i;
ierr = VecCreateMPI(PETSC_COMM_WORLD,PETSC_DECIDE,14,&partial);CHKERRQ(ierr);
for(i=0; i<40; i++) {
PetscReal ttime = i+1;
ierr =
TSTrajectoryGetVecs(appctx.tj,appctx.ts,PETSC_DECIDE,&ttime,U,NULL);CHKERRQ(ierr);
ierr = DMGetLocalVector(appctx.da,&Uloc);CHKERRQ(ierr);
ierr = DMGlobalToLocalBegin(appctx.da,U,INSERT_VALUES,Uloc);CHKERRQ(ierr);
ierr = DMGlobalToLocalEnd(appctx.da,U,INSERT_VALUES,Uloc);CHKERRQ(ierr);
ierr = DMDAVecGetArray(appctx.da,Uloc,&Ui);CHKERRQ(ierr);
ierr = VecGetArray(partial,&partlocal);CHKERRQ(ierr);
partlocal[i] = Ui[1];
ierr = DMDAVecRestoreArray(appctx.da,U,&Ui);CHKERRQ(ierr);
ierr = VecRestoreArray(partial,&partlocal);CHKERRQ(ierr);
ierr = DMRestoreLocalVector(appctx.da,&Uloc);CHKERRQ(ierr);
}
> Il giorno 27 mag 2021, alle ore 01:15, Barry Smith <[email protected]> ha
> scritto:
>
>
>
>> On May 26, 2021, at 10:39 AM, Francesco Brarda <[email protected]>
>> wrote:
>>
>> Thank you very much.
>>> Based on the error message it appears that your code is requesting
>>> different times on different MPI ranks. Is that what you intend to do?
>> Yes. I want to save different times across a vector built with multiple MPI
>> ranks (PETSC_DECIDE for the local length).
>> The function is called only by the first proc (rank=0) and not from the
>> others. Is there a way to force also other ranks to call that routine?
>
> Yes, just have all ranks call it and ignore the result on the other ranks.
>
>> Should I build everything into an external function outside the main?
>
> It can be called in main, does not need to be in a different function.
>
>>
>> Francesco
>>
>>
>>> Il giorno 26 mag 2021, alle ore 16:20, Barry Smith <[email protected]> ha
>>> scritto:
>>>
>>>
>>>
>>>
>>> TSTrajectoryGetVecs() is listed as Collective on TS. This means all ranks
>>> must call it with the same times in the same order of operations on all
>>> ranks that share the TS.
>>>
>>> You do not need to use VecScatter. Each process must call
>>> TSTrajectoryGetVecs with the same time but then you can have only the rank
>>> you care about select the entries from the resulting vectors you care about
>>> while the other ranks for that time just ignore the vectors since they do
>>> not need to values from it.
>>>
>>> Barry
>>>
>>>
>>>
>>>> On May 26, 2021, at 5:20 AM, Francesco Brarda <[email protected]>
>>>> wrote:
>>>>
>>>> Hi!
>>>>
>>>> I solved an ODE system with TS. Now I would like to save one of the
>>>> trajectories in specific times. To do so, I used TSTrajectoryGetVecs.
>>>> The values of the variable I am interested in is on one processor. I want
>>>> to collect these values in a parallel vector, but I had the error:
>>>>
>>>> [0]PETSC ERROR: Invalid argument
>>>> [0]PETSC ERROR: Real value must be same on all processes, argument # 2
>>>> [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html
>>>> for trouble shooting.
>>>> [0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
>>>> [0]PETSC ERROR: ./petsc_sir on a arch-debug named srvulx13 by fbrarda Wed
>>>> May 26 12:00:42 2021
>>>> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++
>>>> --with-fc=gfortran --with-openblas-dir=/opt/packages/openblas/0.2.13-gcc
>>>> --download-mpich PETSC_ARCH=arch-debug
>>>> [0]PETSC ERROR: #1 TSHistoryGetLocFromTime() line 134 in
>>>> /home/fbrarda/petsc/src/ts/interface/tshistory.c
>>>> [0]PETSC ERROR: #2 TSTrajectoryReconstruct_Private() line 55 in
>>>> /home/fbrarda/petsc/src/ts/trajectory/utils/reconstruct.c
>>>> [0]PETSC ERROR: #3 TSTrajectoryGetVecs() line 239 in
>>>> /home/fbrarda/petsc/src/ts/trajectory/interface/traj.c
>>>>
>>>> Is there any specific routine I can use to overcome this issue? Should I
>>>> use VecScatter?
>>>>
>>>> I hope I made myself clear.
>>>> Best,
>>>> Francesco
>>>
>>
>
