On May 15, 2013, at 12:41 PM, Roc Wang <pengxwang at hotmail.com> wrote:
> Thanks, please take a look my further questions. > > > Subject: Re: [petsc-users] local Vec to global Vec and global 3-D array > > From: bsmith at mcs.anl.gov > > Date: Wed, 15 May 2013 11:40:03 -0500 > > CC: petsc-users at mcs.anl.gov > > To: pengxwang at hotmail.com > > > > > > On May 15, 2013, at 10:19 AM, Roc Wang <pengxwang at hotmail.com> wrote: > > > > > Hello, > > > > > > 1. I am trying to save the solution of a PDE for a 3-D geometry domain. I > > > used DM to manage the matrix and vector. > > > The solution vector was converted to local 3D arrays on each process > > > successfully. But there was errors when the functions > > > DMLocalToGlobalBegin() and DMLocalToGlobalEnd() were called. > > > > > > I built the code by following the example in > > > /petsc-3.3-p6/src/dm/examples/tutorials/ex3.c. The codes for this are as > > > followings : > > > > > > /*Get the solution vec */ > > > ierr = KSPSolve(ksp,PETSC_NULL,PETSC_NULL);CHKERRQ(ierr); > > > ierr = KSPGetSolution(ksp,&x);CHKERRQ(ierr); > > > > > > /* local 3d arrays */ > > > ierr = DMDAVecGetArray(da, x, &localArray3d ); CHKERRQ(ierr); > > > > > > /* global 3d array */ > > > ierr = DMCreateGlobalVector(da,&gsol3d);CHKERRQ(ierr); > > > ierr = DMLocalToGlobalBegin(da,x,INSERT_VALUES,gsol3d);CHKERRQ(ierr); > > > ierr = DMLocalToGlobalEnd(da,x,INSERT_VALUES,gsol3d);CHKERRQ(ierr); > > > > The x here must not have been obtained from this da. You need to have > > obtained the x somewhere before with DMCreateLocalVector(da,&x) or with a > > VecDuplicate from such a vector. Likely the vector put into the KSP is not > > from this da. > > !*************************************** > The procedure of creating x is like this: > > call KSPCreate() to build ksp; > call DMDACreate3d() to build da; > ierr = KSPSetDM(ksp,da); // associate da with ksp; > > Then after ksp was solved: > > ierr = KSPSolve(ksp,PETSC_NULL,PETSC_NULL);CHKERRQ(ierr); > ierr = KSPGetSolution(ksp,&x);CHKERRQ(ierr); > ierr = DMDAVecGetArray(da, x, &localArray3d ); CHKERRQ(ierr); > > Here, da was associated with the ksp by calling ierr = KSPSetDM(ksp,da). > The x here was obtained by calling the KSPGetSolution(ksp,&x), before this I > didn't call DMCreateLocalVector(da,&x). It seems Vec x is associated with da > since there is no errors when DMDAVecGetArray(da, x, &localArray3d ) was > called. The x from KSPGetSolution() IS a global vector, it is not a global vector. You cannot do a local to global from the x global vector. > Do you mean I should call DMCreateLocalVector(da,&x) explicitly before ierr = > KSPGetSolution(ksp,&x)? > !*************************************** > > > > > > > > > > 2. I am trying to visualize the solution in a 3-D domain by using some > > > software such as Tecplot. To my understand, I need to output the solution > > > in the format of 3-D array. The vectors of KSP solution in PETSc are > > > local vectors on each process. So they have to be assembled to a global > > > vector and then converted to a 3-D global array. I am not sure if my > > > approach is a good way and if there is some functions in PETSc to output > > > the global solutions directly? > > > > Yes. VecView() on DMDA vectors automatically reorders the entries on the > > file to use the natural ordering. There are variety of possible viewers you > > can use including binary, ASCII (no good for anything but tiny problems), > > VTK, HDF5 > > !*************************************** > VecView only outputs the vector itself. How to output the indexes of nodes > in 3 dimemsions (i,j,k) for each element of the vector, such as: > > i j k x > 0 0 0 1.0 > 0 0 1 2.0 > ... > m n p 10.0 > !*************************************** If you want some ASCII format like that, then write a STAND ALONE sequential program that reads from the binary file with VecLoad() and then outputs the format you want. Never Never ever try to do a parallel output of data in this kind of format, it will be slow and hard to write and there is no reason to write it. Barry > > > > Barry > > > > > > > > Part of the error information when DMLocalToGlobalBegin() and > > > DMLocalToGlobalEnd() were called > > > > > > [0]PETSC ERROR: [1]PETSC ERROR: [2]PETSC ERROR: --------------------- > > > Error Message ------------------------------------ > > > [3]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > [2]PETSC ERROR: [3]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > Nonconforming object sizes! > > > Nonconforming object sizes! > > > [2]PETSC ERROR: [3]PETSC ERROR: Vector wrong size 30 for scatter 60 > > > (scatter forward and vector from != ctx from size)! > > > Vector wrong size 20 for scatter 45 (scatter forward and vector from != > > > ctx from size)! > >
