Thank you for your reply. Actually, I succeeded in running with the PetscSpliOwnership() code that you gave to me. I think I didn't understand how to write a code in parallel. Thank you again.
----- Original Message ----- From: "Jed Brown" <[email protected]> To: "Joon Hee Choi" <[email protected]> Cc: [email protected] Sent: Friday, September 13, 2013 5:34:33 PM Subject: Re: [petsc-users] About MatGetOwnershipRange() Joon Hee Choi <[email protected]> writes: >>>Did you run in parallel? >> >> Yes. I tested my code with 2 nodes and 4 ppn(processors per node). >Then print what you are passing in and what you get back on each >process. The PetscSplitOwnership code is short. >>>> Also, tuples are the total pairs of (row dimension, column dimension, >>>> element), values of sparse matrix, which are read from a file. >> >>>Using files this way is a massive bottleneck that you'll have to >>>eliminate if you want your code to be scalable. >> >> What do you mean a massive bottleneck? >Writing to a file is at least 1000 times slower than accessing memory >and it does not parallelize as well. >> Actually, I succeeded in setting up the large matrix using seqaij type >> and calculating products between huge matrices. However, I need to use >> multiple machines because I have to run loops of the product many >> times. And I am a novice of mpi. >I'm not sure what you're asking, but you should create the matrices in >memory, preferably in parallel and mostly where the entries will need to >be stored. >>>> The tuples are sorted and distributed. >> >>>When you distribute, are you sure that each process really gets the >>>entire row, or would it be possible to cut in the middle of a row? >> >> I think each process gets the entire row. The following is the code for >> getting >>tuples from text file: >> >> ierr = PetscFOpen(PETSC_COMM_WORLD, inputFile, "r", &file); CHKERRQ(ierr); >> while (fscanf(file, "%d %d %d", &ii, &jj, &vv) == 3) >> { >> tups.push_back(std::tr1::make_tuple (ii-1, jj-1, vv)); >> if (ii > I) I = ii; >> if (jj > J) J = jj; >> } >> ierr = PetscFClose(PETSC_COMM_WORLD, file); CHKERRQ(ierr); >Nothing in this code snippet indicates that the row partition is >non-overlapping.
