Hi, It might be unclear. The error here is that the submat[1] does have a (1,2) entry, but MatSetValues at (1,2) gives an error message. GASM and ASM give the same error.
Thanks! On May 21, 2012, at 9:11 PM, Hui Zhang wrote: > Sorry, I uses PCASMSetSortIndices for GASM. Now, I correct it and the output > becomes the > same as PCASM. See attached. > > <GASM_test.zip> > > > On May 21, 2012, at 8:59 PM, Hui Zhang wrote: > >> Dmitry, >> >> finally, I am able to make a simple program and reproduce the error. Since I >> had moved to >> petsc-dev (before I used petsc-3.2), the error is a little different. >> >> Download the program attached and run the run.sh. You will see the output >> from PCASM. >> There is a submatrix (submat[1] on rank 0) which has a row like >> >> row 1: (0, -1) (1, 0 + 0.4 i) (4, -1) (2, -1) >> >> Note that the column 2 is placed in the last and the column numbers are not >> in the normal order. >> Following that, we can also see the error message like >> >> [0]PETSC ERROR: New nonzero at (1,2) caused a malloc! >> >> You can also try with GASM, use >> >> mpirun -np 2 ./gasm_test -dd_type gasm -n 4 -px 1 -py 2 -sx 2 -sy 1 >> >> which partitions the domain to 1 * 2 and mapped to the two processors, and >> for each processor >> we further partition its local domain to 2 * 1 subdomains. >> >> Then, you will see that the row 1 of submat[1] on rank 1 becomes >> >> row 1: (0, -1) (1, 0 + 0.4 i) (3, -1) (6, -1) >> >> which is totally wrong because from the IS for this overlapping subdomain >> and the big matrix A, >> row 1 should have non-zeros like the output from PCASM. >> >> I guess the problem is due to that we set PCASMSetSortIndices to FALSE and >> something goes wrong >> in asm.c and gasm.c. >> >> Thanks! >> Hui >> >> >> <GASM_test.zip> >> >> >> >> >> >>> On Sun, May 20, 2012 at 5:06 AM, Hui Zhang <mike.hui.zhang at hotmail.com> >>> wrote: >>> I have the following MatView output: >>> >>> row 0: (0, -0.0351719 - 0.638814 i) (1, -0.517586 - 0.617823 i) (4, >>> -0.517586 - 0.617823 i) (1, -1.03517) (5, -0.508793) >>> >>> Note that there are two values for the column 1, is this normal? >>> >>> The above matrix is obtained from PCASMGetSubKSP(pc,subksp) and >>> KSPGetOperators(subksp[0], ...). >>> >>> This is odd. Can you explain a bit more what leads up to this, so we can >>> try and reproduce the problem? >>> Thanks. >>> Dmitry. >>> >>> Thanks! >>> >>> >>> >>> On May 11, 2012, at 7:17 PM, Dmitry Karpeev wrote: >>> >>>> You can call PCSetUp(pc) on either ASM or GASM, and that will destroy and >>>> recreate the matrices (including calling >>>> your modification subroutine), but not the subdomains or the subdomain >>>> solvers. >>>> If you just want to modify the submatrices, you can call >>>> PC(G)ASMGetSubmatrices() and modify the matrices it returns >>>> (in the same order as the subdomains were set). That's a bit of a hack, >>>> since you will essentially be modifying the PC's internal data structures. >>>> As long as you are careful, you should be okay, since you already >>>> effectively have the same type of access to the submatrices through the >>>> Modify callback. >>>> >>>> Dmitry. >>>> >>>> On Fri, May 11, 2012 at 11:52 AM, Hui Zhang <mike.hui.zhang at >>>> hotmail.com> wrote: >>>> I just have a question about reuse of PCASM or PCGASM. >>>> Suppose I have seted up the PCASM and related KSP and I solved one time. >>>> Next for the same linear system (matrix and RHS), I just want PCASM modify >>>> the submatrices (PCSetModifySubmatrices) in a different way, using the >>>> same routine for modifying but with >>>> different user context for the modifying routine. >>>> >>>> What can I do for this task? Currently, I destroy the KSP and >>>> re-construct it. I guess >>>> even for PCASM I can re-use it because the partition of subdomains remain >>>> the same. >>>> >>>> Thanks! >>>> >>>> >>>> On May 10, 2012, at 6:37 PM, Dmitry Karpeev wrote: >>>> >>>>> Hui, >>>>> There've been several changes to PCGASM ahead of the new release. >>>>> Let me go back and see if it affected the convergence problem. >>>>> Dmitry. >>>>> >>>>> On Thu, May 10, 2012 at 4:16 AM, Hui Zhang <mike.hui.zhang at >>>>> hotmail.com> wrote: >>>>> Hi Dmitry, >>>>> >>>>> is there any news about PCGASM? >>>>> >>>>> thanks, >>>>> Hui >>>>> >>>>> On Feb 20, 2012, at 6:38 PM, Dmitry Karpeev wrote: >>>>> >>>>>> Okay, thanks. >>>>>> I'll take a look. >>>>>> >>>>>> Dmitry. >>>>>> >>>>>> On Mon, Feb 20, 2012 at 11:30 AM, Hui Zhang <mike.hui.zhang at >>>>>> hotmail.com> wrote: >>>>>> For reference, my results are attached. >>>>>> >>>>>> asm1.txt for asm with 1 process, >>>>>> asm2.txt for asm with 2 processes, >>>>>> gasm1.txt for gasm with 1 process, (with the iteration numbers different >>>>>> from others) >>>>>> gasm2.txt for gasm with 2 processes >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> thank you, >>>>>> Hui >>>>>> >>>>>> On Feb 20, 2012, at 3:06 PM, Dmitry Karpeev wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Feb 20, 2012 at 12:59 AM, Hui Zhang <mike.hui.zhang at >>>>>>> hotmail.com> wrote: >>>>>>> >>>>>>> On Feb 20, 2012, at 12:41 AM, Dmitry Karpeev wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Feb 19, 2012 at 3:08 PM, Hui Zhang <mike.hui.zhang at >>>>>>>> hotmail.com> wrote: >>>>>>>> I have a new problem: the results from ASM and GASM are different and >>>>>>>> it seems >>>>>>>> GASM has something wrong with SetModifySubMatrices. Numerical tests >>>>>>>> are with >>>>>>>> each subdomain supported only by one subdomain. There are no problems >>>>>>>> when >>>>>>>> I did not modify submatrices. But when I modify submatrices, there >>>>>>>> are problems >>>>>>>> with GASM but no problems with ASM. >>>>>>>> >>>>>>>> For example, I use two subdomains. In the first case each subdomain is >>>>>>>> supported by >>>>>>>> one processor and there seems no problem with GASM. But when I use run >>>>>>>> my program >>>>>>>> with only one proc. so that it supports both of the two subdomains, >>>>>>>> the iteration >>>>>>>> number is different from the first case and is much larger. On the >>>>>>>> other hand >>>>>>>> ASM has no such problem. >>>>>>>> >>>>>>>> Are the solutions the same? >>>>>>>> What problem are you solving? >>>>>>> >>>>>>> Yes, the solutions are the same. That's why ASM gives the same results >>>>>>> with one or >>>>>>> two processors. But GASM did not. >>>>>>> Sorry, I wasn't clear: ASM and GASM produced different solutions in the >>>>>>> case of two domains per processor? >>>>>>> I'm solving the Helmholtz equation. Maybe >>>>>>> I can prepare a simpler example to show this difference. >>>>>>> That would be helpful. >>>>>>> Thanks. >>>>>>> >>>>>>> Dmitry. >>>>>>> >>>>>>>> >>>>>>>> Dmitry. >>>>>>>> >>>>>>>> >>>>>>>> On Feb 15, 2012, at 6:46 PM, Dmitry Karpeev wrote: >>>>>>>> >>>>>>>>> You should be able to. >>>>>>>>> This behavior is the same as in PCASM, >>>>>>>>> except in GASM the matrices live on subcommunicators. >>>>>>>>> I am in transit right now, but I can take a closer look in Friday. >>>>>>>>> >>>>>>>>> Dmitry >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Feb 15, 2012, at 8:07, Hui Zhang <mike.hui.zhang at hotmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On Feb 15, 2012, at 11:19 AM, Hui Zhang wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dmitry, >>>>>>>>>>> >>>>>>>>>>> thanks a lot! Currently, I'm not using ISColoring. Just comes >>>>>>>>>>> another question >>>>>>>>>>> on PCGASMSetModifySubMatrices(). The user provided function has the >>>>>>>>>>> prototype >>>>>>>>>>> >>>>>>>>>>> func (PC pc,PetscInt nsub,IS *row,IS *col,Mat *submat,void >>>>>>>>>>> *ctx); >>>>>>>>>>> >>>>>>>>>>> I think the coloumns from the parameter 'col' are always the same >>>>>>>>>>> as the rows >>>>>>>>>>> from the parameter 'row'. Because PCGASMSetLocalSubdomains() only >>>>>>>>>>> accepts >>>>>>>>>>> index sets but not rows and columns. Has I misunderstood something? >>>>>>>>>> >>>>>>>>>> As I tested, the row and col are always the same. >>>>>>>>>> >>>>>>>>>> I have a new question. Am I allowed to SetLocalToGlobalMapping() for >>>>>>>>>> the submat's >>>>>>>>>> in the above func()? >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Hui >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> Hui >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Feb 11, 2012, at 3:36 PM, Dmitry Karpeev wrote: >>>>>>>>>>> >>>>>>>>>>>> Yes, that's right. >>>>>>>>>>>> There is no good way to help the user assemble the subdomains at >>>>>>>>>>>> the moment beyond the 2D stuff. >>>>>>>>>>>> It is expected that they are generated from mesh subdomains. >>>>>>>>>>>> Each IS does carry the subdomains subcomm. >>>>>>>>>>>> >>>>>>>>>>>> There is ISColoringToList() that is supposed to convert a >>>>>>>>>>>> "coloring" of indices to an array of ISs, >>>>>>>>>>>> each having the indices with the same color and the subcomm that >>>>>>>>>>>> supports that color. It is >>>>>>>>>>>> largely untested, though. You could try using it and give us >>>>>>>>>>>> feedback on any problems you encounter. >>>>>>>>>>>> >>>>>>>>>>>> Dmitry. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Feb 11, 2012 at 6:06 AM, Hui Zhang <mike.hui.zhang at >>>>>>>>>>>> hotmail.com> wrote: >>>>>>>>>>>> About PCGASMSetLocalSubdomains(), in the case of one subdomain >>>>>>>>>>>> supported by >>>>>>>>>>>> multiple processors, shall I always create the arguments 'is[s]' >>>>>>>>>>>> and 'is_local[s]' >>>>>>>>>>>> in a subcommunicator consisting of processors supporting the >>>>>>>>>>>> subdomain 's'? >>>>>>>>>>>> >>>>>>>>>>>> The source code of PCGASMCreateSubdomains2D() seemingly does so. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Hui >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120521/eb173c57/attachment.html>
