On Tue, Sep 24, 2013 at 1:42 PM, Jed Brown <[email protected]> wrote:
> Analabha Roy <[email protected]> writes: > > > Hi all, > > > > > > Compiling and running this > > code< > https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c>that > > builds a petsc matrix gives different results when run with different > > number of processors. > > Thanks for the reply. > Uh, if you call rand() on different processors, why would you expect it > to give the same results? > > Right, I get that. The rand() was a placeholder. This original much larger code<https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c>replicates the same loop structure and runs the same Petsc subroutines, but running it by mpirun -np $N ./eth -lattice_size 5 -vector_size 1 -repulsion 0.0 -draw_out -draw_pause -1 with N=1,2,3,4 gives different results for the matrix dumped out by lines 514-519<https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c#514>. The matrix itself is evaluated in parallel, created in lines 263-275 <https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c#263>and evaluated in lines 294-356<https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c#294> (you can click on the line numbers above to navigate directly to them) Here is a sample <http://i43.tinypic.com/zyhf2f.jpg> of the output of lines 514-519<https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c#514> for N=1,2,3,4 procs left to right. Thty're different for different procs. They should be the same, since none of my input parameters are numprocs dependent, and I don't explicitly use the size or rank anywhere in the code. > for (sitestride1 = Istart; sitestride1 < Iend; sitestride1++) > { > for (sitestride2 = 0; sitestride2 < matsize; sitestride2++) > { > for (alpha = 0; alpha < dim; alpha++) > { > for (mu = 0; mu < dim; mu++) > for (lambda = 0; lambda < dim; lambda++) > { > vecval = rand () / rand (); > } > > VecSetValue (BDB_AA, alpha, vecval, INSERT_VALUES); > > } > VecAssemblyBegin (BDB_AA); > VecAssemblyEnd (BDB_AA); > VecSum (BDB_AA, &element); > colvalues[sitestride2] = element; > > } > //Insert the array of colvalues to the sitestride1^th row of H > MatSetValues (AVG_BDIBJ, 1, &sitestride1, matsize, idx, colvalues, > INSERT_VALUES); > > } > > > The code is large and complex, so I have created a smaller program > > with the same > > loop structure here. <http://pastebin.ca/2457643> > > > > Compile it and run it with "mpirun -np $N ./test -draw_pause -1" gives > > different results for different values of N even though it's not supposed > > to. > > What do you expect to see? > > > Here is a sample output <http://i42.tinypic.com/2s16ccw.jpg> for > N=1,2,3,4 > > from left to right. > > > > Can anyone guide me as to what I'm doing wrong? Are any of the petssc > > routines used not parallelizable? > > > > Thanks in advance, > > > > Regards. > > > > -- > > --- > > *Analabha Roy* > > C.S.I.R <http://www.csir.res.in> Senior Research > > Associate<http://csirhrdg.res.in/poolsra.htm> > > Saha Institute of Nuclear Physics <http://www.saha.ac.in> > > Section 1, Block AF > > Bidhannagar, Calcutta 700064 > > India > > *Emails*: [email protected], [email protected] > > *Webpage*: http://www.ph.utexas.edu/~daneel/ > -- --- *Analabha Roy* C.S.I.R <http://www.csir.res.in> Senior Research Associate<http://csirhrdg.res.in/poolsra.htm> Saha Institute of Nuclear Physics <http://www.saha.ac.in> Section 1, Block AF Bidhannagar, Calcutta 700064 India *Emails*: [email protected], [email protected] *Webpage*: http://www.ph.utexas.edu/~daneel/
