It's ok. I found the example program.
On May 16, 2012, at 10:46 AM, Hui Zhang wrote:
>
> A question about PCASMGetSubKSP(), it says that we must call KSPSetUp()
> before calling
> PCASMGetSubKSP(). But what ksp should be SetUp, the ksp using PCASM or what?
>
> Thanks!
>
>
>
>>>
>>> On Tue, May 15, 2012 at 10:30 AM, Hui Zhang <mike.hui.zhang at hotmail.com>
>>> wrote:
>>> thanks for reply!
>>> On May 15, 2012, at 5:19 PM, Dmitry Karpeev wrote:
>>>
>>>>
>>>>
>>>> On Tue, May 15, 2012 at 9:55 AM, Hui Zhang <mike.hui.zhang at hotmail.com>
>>>> wrote:
>>>> Dmitry,
>>>>
>>>> thanks for remind. I have a new question about PCASM / PCGASM:
>>>>
>>>> can I get the restricted extension operators, which maps an overlapping
>>>> subdomain solution
>>>> to the global domain?
>>>>
>>>> I'm not exactly sure what you mean.
>>>>
>>>> Are you talking about embedding the subdomain vectors back into the
>>>> original vector?
>>>
>>> Yes, exactly.
>>>
>>>> If so, there is substantial difference in how this is handled in ASM and
>>>> GASM:
>>>> ASM has a bunch of sequential vectors that can be scattered back into the
>>>> global vector,
>>>
>>> Yes. Is there a method to get the scatter?
>>>
>>> In the ASM case it's a bunch of scatters -- one for each subdomain.
>>> Currently there is no method to
>>> retrieve them.
>>
>> this hint is very helpful. Thanks!
>>
>>> What requires this functionality?
>>
>> I am writing some modified ASM method. In construction of energy
>> minimization coarse basis,
>> I need to solve individual subdomain problems and not to sum them, just to
>> extend them separately.
>> I wonder whether you guys have ever done this coarse basis.
>> Is there a reference where the basis is described?
>> Dmtiry.
>>
>> Thanks,
>> Hui
>>
>>>
>>> In the ASM case you can construct the scatters yourself easily enough,
>>> since you have all of the requisite information -- the array of subdomain
>>> ISs and the global vector x.
>>> The only piece of data you might not have is the set of outer subdomains
>>> that have been obtained
>>> by applying overlap increase to the original inner (nonoverlapping)
>>> subdomains.
>>>
>>>> because the subdomains are always local to at most one processor.
>>>>
>>>> In the GASM case this is rather different, since the subdomains can live
>>>> on arbitrary subcommunicators
>>>> and there is only one scatter, which is applied to the direct sum of all
>>>> the subdomain vectors on the original communicator. I'm not sure how
>>>> useful that last scatter would be for you, since the details of the
>>>> structure
>>>> of the direct sum vector are internal to GASM.
>>>
>>> I would prefer to have the scatter for individual subdomain before direct
>>> sum.
>>> But if I can get the scatter PCGASM has, maybe it is still useful. Please
>>> tell me how to get it?
>>> There are no individual subdomain scatters, but, as in the case of ASM, you
>>> can construct them
>>> easily enough, except that those would have to operate on subcommunicators.
>>> In GASM we pack them into a single scatter on the original communicator.
>>> Currently there is no method
>>> to expose this scatter. Why do you need this functionality?
>>>
>>> Dmitry.
>>> Thanks!
>>>
>>>>
>>>> Dmitry.
>>>>
>>>> Thanks!
>>>>
>>>> On May 15, 2012, at 3:29 PM, Dmitry Karpeev wrote:
>>>>
>>>>> There are some additional minor fixes that mostly have to do with
>>>>> outputting the subdomain information with -pc_gasm_view_subdomains (in
>>>>> PCView()) and with -pc_gasm_print_subdomains (during PCSetUp()).
>>>>> You might want to pull those latest patches, but it won't interfere with
>>>>> your work if you don't use subdomain output.
>>>>>
>>>>> Thanks.
>>>>> Dmitry.
>>>>>
>>>>> On Tue, May 15, 2012 at 7:14 AM, Hui Zhang <mike.hui.zhang at
>>>>> hotmail.com> wrote:
>>>>> Dmitry,
>>>>>
>>>>> thanks for reply. I re-download the codes and tried it again and now it
>>>>> works correctly!
>>>>>
>>>>> Everything seems ok.
>>>>>
>>>>> Thanks,
>>>>> Hui
>>>>>
>>>>>
>>>>> On May 15, 2012, at 2:01 PM, Dmitry Karpeev wrote:
>>>>>
>>>>>> Hui,
>>>>>> I'm trying to reproduce this problem, unsuccessfully, so far.
>>>>>> One thing that looks odd is that the output below claims the PC is of
>>>>>> type "asm", even though you are running with -dd_type gasm. Could you
>>>>>> verify that's the correct output?
>>>>>>
>>>>>> Here's the output I get with
>>>>>> ${PETSC_DIR}/${PETSC_ARCH}/bin/mpiexec -np 1 ./gasm_test -n 64 -dd_type
>>>>>> asm -dd_ksp_view
>>>>>>
>>>>>> PC Object:(dd_) 1 MPI processes
>>>>>> type: asm
>>>>>> Additive Schwarz: total subdomain blocks = 2, user-defined overlap
>>>>>> Additive Schwarz: restriction/interpolation type - RESTRICT
>>>>>> Local solve is same for all blocks, in the following KSP and PC
>>>>>> objects:
>>>>>> KSP Object: (dd_sub_) 1 MPI processes
>>>>>> type: preonly
>>>>>> maximum iterations=10000, initial guess is zero
>>>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000
>>>>>> left preconditioning
>>>>>> using NONE norm type for convergence test
>>>>>> <snip>
>>>>>>
>>>>>> and with
>>>>>> ${PETSC_DIR}/${PETSC_ARCH}/bin/mpiexec -np 1 ./gasm_test -n 64 -dd_type
>>>>>> gasm -dd_ksp_view
>>>>>>
>>>>>> PC Object:(dd_) 1 MPI processes
>>>>>> type: gasm
>>>>>> Generalized additive Schwarz:
>>>>>> Restriction/interpolation type: RESTRICT
>>>>>> user-defined overlap
>>>>>> total number of subdomains = 2
>>>>>> number of local subdomains = 2
>>>>>> max number of local subdomains = 2
>>>>>> [0:1] number of locally-supported subdomains = 2
>>>>>> Subdomain solver info is as follows:
>>>>>> <snip>
>>>>>>
>>>>>> What convergence are you seeing with the two PC types? It should be the
>>>>>> same with 1 and 2 procs for both PCASM and PCGASM.
>>>>>>
>>>>>> Thanks.
>>>>>> Dmitry.
>>>>>>
>>>>>> On Tue, May 15, 2012 at 4:03 AM, Hui Zhang <mike.hui.zhang at
>>>>>> hotmail.com> wrote:
>>>>>> Dmitry,
>>>>>>
>>>>>> I got the newest petsc-dev and I run the test by
>>>>>>
>>>>>> mpirun -np 1 ./gasm_test -dd_type gasm -n 64 -dd_ksp_view
>>>>>>
>>>>>> which gives the following output
>>>>>>
>>>>>> PC Object:(dd_) 1 MPI processes
>>>>>> type: asm
>>>>>> Additive Schwarz: total subdomain blocks = 1, amount of overlap = 1
>>>>>> ^^^
>>>>>> note the above number, it
>>>>>> should be 2
>>>>>>
>>>>>> While PCASM has no such problem.
>>>>>>
>>>>>> Thanks,
>>>>>> Hui
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hui,
>>>>>>>
>>>>>>> The convergence issue should be resolved in the latest petsc-dev.
>>>>>>> I'm attaching a slightly modified gasm_test.c (reflecting some upcoming
>>>>>>> API changes)
>>>>>>> that should verify that.
>>>>>>>
>>>>>>> Let me know if it works for you.
>>>>>>> Thanks.
>>>>>>> Dmitry.
>>>>>>> On Fri, May 11, 2012 at 12:31 PM, Hui Zhang <mike.hui.zhang at
>>>>>>> hotmail.com> wrote:
>>>>>>> Hi Dmitry,
>>>>>>>
>>>>>>> thanks for useful hints. Good day!
>>>>>>>
>>>>>>> Hui
>>>>>>>
>>>>>>> On May 11, 2012, at 7:17 PM, Dmitry Karpeev wrote:
>>>>>>>
>>>>>>>> You can call PCSetUp(pc) on either ASM or GASM, and that will destroy
>>>>>>>> and recreate the matrices (including calling
>>>>>>>> your modification subroutine), but not the subdomains or the subdomain
>>>>>>>> solvers.
>>>>>>>> If you just want to modify the submatrices, you can call
>>>>>>>> PC(G)ASMGetSubmatrices() and modify the matrices it returns
>>>>>>>> (in the same order as the subdomains were set). That's a bit of a
>>>>>>>> hack, since you will essentially be modifying the PC's internal data
>>>>>>>> structures. As long as you are careful, you should be okay, since you
>>>>>>>> already effectively have the same type of access to the submatrices
>>>>>>>> through the Modify callback.
>>>>>>>>
>>>>>>>> Dmitry.
>>>>>>>>
>>>>>>>> On Fri, May 11, 2012 at 11:52 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>> hotmail.com> wrote:
>>>>>>>> I just have a question about reuse of PCASM or PCGASM.
>>>>>>>> Suppose I have seted up the PCASM and related KSP and I solved one
>>>>>>>> time.
>>>>>>>> Next for the same linear system (matrix and RHS), I just want PCASM
>>>>>>>> modify the submatrices (PCSetModifySubmatrices) in a different way,
>>>>>>>> using the same routine for modifying but with
>>>>>>>> different user context for the modifying routine.
>>>>>>>>
>>>>>>>> What can I do for this task? Currently, I destroy the KSP and
>>>>>>>> re-construct it. I guess
>>>>>>>> even for PCASM I can re-use it because the partition of subdomains
>>>>>>>> remain the same.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> On May 10, 2012, at 6:37 PM, Dmitry Karpeev wrote:
>>>>>>>>
>>>>>>>>> Hui,
>>>>>>>>> There've been several changes to PCGASM ahead of the new release.
>>>>>>>>> Let me go back and see if it affected the convergence problem.
>>>>>>>>> Dmitry.
>>>>>>>>>
>>>>>>>>> On Thu, May 10, 2012 at 4:16 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>>> hotmail.com> wrote:
>>>>>>>>> Hi Dmitry,
>>>>>>>>>
>>>>>>>>> is there any news about PCGASM?
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> Hui
>>>>>>>>>
>>>>>>>>> On Feb 20, 2012, at 6:38 PM, Dmitry Karpeev wrote:
>>>>>>>>>
>>>>>>>>>> Okay, thanks.
>>>>>>>>>> I'll take a look.
>>>>>>>>>>
>>>>>>>>>> Dmitry.
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 20, 2012 at 11:30 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>> For reference, my results are attached.
>>>>>>>>>>
>>>>>>>>>> asm1.txt for asm with 1 process,
>>>>>>>>>> asm2.txt for asm with 2 processes,
>>>>>>>>>> gasm1.txt for gasm with 1 process, (with the iteration numbers
>>>>>>>>>> different from others)
>>>>>>>>>> gasm2.txt for gasm with 2 processes
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> thank you,
>>>>>>>>>> Hui
>>>>>>>>>>
>>>>>>>>>> On Feb 20, 2012, at 3:06 PM, Dmitry Karpeev wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 20, 2012 at 12:59 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Feb 20, 2012, at 12:41 AM, Dmitry Karpeev wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Feb 19, 2012 at 3:08 PM, Hui Zhang <mike.hui.zhang at
>>>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>>>> I have a new problem: the results from ASM and GASM are different
>>>>>>>>>>>> and it seems
>>>>>>>>>>>> GASM has something wrong with SetModifySubMatrices. Numerical
>>>>>>>>>>>> tests are with
>>>>>>>>>>>> each subdomain supported only by one subdomain. There are no
>>>>>>>>>>>> problems when
>>>>>>>>>>>> I did not modify submatrices. But when I modify submatrices,
>>>>>>>>>>>> there are problems
>>>>>>>>>>>> with GASM but no problems with ASM.
>>>>>>>>>>>>
>>>>>>>>>>>> For example, I use two subdomains. In the first case each
>>>>>>>>>>>> subdomain is supported by
>>>>>>>>>>>> one processor and there seems no problem with GASM. But when I use
>>>>>>>>>>>> run my program
>>>>>>>>>>>> with only one proc. so that it supports both of the two
>>>>>>>>>>>> subdomains, the iteration
>>>>>>>>>>>> number is different from the first case and is much larger. On
>>>>>>>>>>>> the other hand
>>>>>>>>>>>> ASM has no such problem.
>>>>>>>>>>>>
>>>>>>>>>>>> Are the solutions the same?
>>>>>>>>>>>> What problem are you solving?
>>>>>>>>>>>
>>>>>>>>>>> Yes, the solutions are the same. That's why ASM gives the same
>>>>>>>>>>> results with one or
>>>>>>>>>>> two processors. But GASM did not.
>>>>>>>>>>> Sorry, I wasn't clear: ASM and GASM produced different solutions in
>>>>>>>>>>> the case of two domains per processor?
>>>>>>>>>>> I'm solving the Helmholtz equation. Maybe
>>>>>>>>>>> I can prepare a simpler example to show this difference.
>>>>>>>>>>> That would be helpful.
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> Dmitry.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Dmitry.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Feb 15, 2012, at 6:46 PM, Dmitry Karpeev wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> You should be able to.
>>>>>>>>>>>>> This behavior is the same as in PCASM,
>>>>>>>>>>>>> except in GASM the matrices live on subcommunicators.
>>>>>>>>>>>>> I am in transit right now, but I can take a closer look in Friday.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dmitry
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Feb 15, 2012, at 8:07, Hui Zhang <mike.hui.zhang at
>>>>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Feb 15, 2012, at 11:19 AM, Hui Zhang wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Dmitry,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks a lot! Currently, I'm not using ISColoring. Just comes
>>>>>>>>>>>>>>> another question
>>>>>>>>>>>>>>> on PCGASMSetModifySubMatrices(). The user provided function has
>>>>>>>>>>>>>>> the prototype
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> func (PC pc,PetscInt nsub,IS *row,IS *col,Mat *submat,void
>>>>>>>>>>>>>>> *ctx);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think the coloumns from the parameter 'col' are always the
>>>>>>>>>>>>>>> same as the rows
>>>>>>>>>>>>>>> from the parameter 'row'. Because PCGASMSetLocalSubdomains()
>>>>>>>>>>>>>>> only accepts
>>>>>>>>>>>>>>> index sets but not rows and columns. Has I misunderstood
>>>>>>>>>>>>>>> something?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I tested, the row and col are always the same.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a new question. Am I allowed to SetLocalToGlobalMapping()
>>>>>>>>>>>>>> for the submat's
>>>>>>>>>>>>>> in the above func()?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>> Hui
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>>> Hui
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Feb 11, 2012, at 3:36 PM, Dmitry Karpeev wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, that's right.
>>>>>>>>>>>>>>>> There is no good way to help the user assemble the subdomains
>>>>>>>>>>>>>>>> at the moment beyond the 2D stuff.
>>>>>>>>>>>>>>>> It is expected that they are generated from mesh subdomains.
>>>>>>>>>>>>>>>> Each IS does carry the subdomains subcomm.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is ISColoringToList() that is supposed to convert a
>>>>>>>>>>>>>>>> "coloring" of indices to an array of ISs,
>>>>>>>>>>>>>>>> each having the indices with the same color and the subcomm
>>>>>>>>>>>>>>>> that supports that color. It is
>>>>>>>>>>>>>>>> largely untested, though. You could try using it and give us
>>>>>>>>>>>>>>>> feedback on any problems you encounter.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dmitry.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Feb 11, 2012 at 6:06 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>>>>>>>> About PCGASMSetLocalSubdomains(), in the case of one subdomain
>>>>>>>>>>>>>>>> supported by
>>>>>>>>>>>>>>>> multiple processors, shall I always create the arguments
>>>>>>>>>>>>>>>> 'is[s]' and 'is_local[s]'
>>>>>>>>>>>>>>>> in a subcommunicator consisting of processors supporting the
>>>>>>>>>>>>>>>> subdomain 's'?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The source code of PCGASMCreateSubdomains2D() seemingly does
>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Hui
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> <gasm_test.c>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120516/6d8a6573/attachment-0001.htm>
