On May 15, 2012, at 5:42 PM, Dmitry Karpeev wrote:
>
>
> On Tue, May 15, 2012 at 10:30 AM, Hui Zhang <mike.hui.zhang at hotmail.com>
> wrote:
> thanks for reply!
> On May 15, 2012, at 5:19 PM, Dmitry Karpeev wrote:
>
>>
>>
>> On Tue, May 15, 2012 at 9:55 AM, Hui Zhang <mike.hui.zhang at hotmail.com>
>> wrote:
>> Dmitry,
>>
>> thanks for remind. I have a new question about PCASM / PCGASM:
>>
>> can I get the restricted extension operators, which maps an overlapping
>> subdomain solution
>> to the global domain?
>>
>> I'm not exactly sure what you mean.
>>
>> Are you talking about embedding the subdomain vectors back into the original
>> vector?
>
> Yes, exactly.
>
>> If so, there is substantial difference in how this is handled in ASM and
>> GASM:
>> ASM has a bunch of sequential vectors that can be scattered back into the
>> global vector,
>
> Yes. Is there a method to get the scatter?
>
> In the ASM case it's a bunch of scatters -- one for each subdomain.
> Currently there is no method to
> retrieve them.
this hint is very helpful. Thanks!
> What requires this functionality?
I am writing some modified ASM method. In construction of energy minimization
coarse basis,
I need to solve individual subdomain problems and not to sum them, just to
extend them separately.
I wonder whether you guys have ever done this coarse basis.
Thanks,
Hui
>
> In the ASM case you can construct the scatters yourself easily enough,
> since you have all of the requisite information -- the array of subdomain ISs
> and the global vector x.
> The only piece of data you might not have is the set of outer subdomains that
> have been obtained
> by applying overlap increase to the original inner (nonoverlapping)
> subdomains.
>
>> because the subdomains are always local to at most one processor.
>>
>> In the GASM case this is rather different, since the subdomains can live on
>> arbitrary subcommunicators
>> and there is only one scatter, which is applied to the direct sum of all the
>> subdomain vectors on the original communicator. I'm not sure how useful that
>> last scatter would be for you, since the details of the structure
>> of the direct sum vector are internal to GASM.
>
> I would prefer to have the scatter for individual subdomain before direct
> sum.
> But if I can get the scatter PCGASM has, maybe it is still useful. Please
> tell me how to get it?
> There are no individual subdomain scatters, but, as in the case of ASM, you
> can construct them
> easily enough, except that those would have to operate on subcommunicators.
> In GASM we pack them into a single scatter on the original communicator.
> Currently there is no method
> to expose this scatter. Why do you need this functionality?
>
> Dmitry.
> Thanks!
>
>>
>> Dmitry.
>>
>> Thanks!
>>
>> On May 15, 2012, at 3:29 PM, Dmitry Karpeev wrote:
>>
>>> There are some additional minor fixes that mostly have to do with
>>> outputting the subdomain information with -pc_gasm_view_subdomains (in
>>> PCView()) and with -pc_gasm_print_subdomains (during PCSetUp()).
>>> You might want to pull those latest patches, but it won't interfere with
>>> your work if you don't use subdomain output.
>>>
>>> Thanks.
>>> Dmitry.
>>>
>>> On Tue, May 15, 2012 at 7:14 AM, Hui Zhang <mike.hui.zhang at hotmail.com>
>>> wrote:
>>> Dmitry,
>>>
>>> thanks for reply. I re-download the codes and tried it again and now it
>>> works correctly!
>>>
>>> Everything seems ok.
>>>
>>> Thanks,
>>> Hui
>>>
>>>
>>> On May 15, 2012, at 2:01 PM, Dmitry Karpeev wrote:
>>>
>>>> Hui,
>>>> I'm trying to reproduce this problem, unsuccessfully, so far.
>>>> One thing that looks odd is that the output below claims the PC is of type
>>>> "asm", even though you are running with -dd_type gasm. Could you verify
>>>> that's the correct output?
>>>>
>>>> Here's the output I get with
>>>> ${PETSC_DIR}/${PETSC_ARCH}/bin/mpiexec -np 1 ./gasm_test -n 64 -dd_type
>>>> asm -dd_ksp_view
>>>>
>>>> PC Object:(dd_) 1 MPI processes
>>>> type: asm
>>>> Additive Schwarz: total subdomain blocks = 2, user-defined overlap
>>>> Additive Schwarz: restriction/interpolation type - RESTRICT
>>>> Local solve is same for all blocks, in the following KSP and PC
>>>> objects:
>>>> KSP Object: (dd_sub_) 1 MPI processes
>>>> type: preonly
>>>> maximum iterations=10000, initial guess is zero
>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000
>>>> left preconditioning
>>>> using NONE norm type for convergence test
>>>> <snip>
>>>>
>>>> and with
>>>> ${PETSC_DIR}/${PETSC_ARCH}/bin/mpiexec -np 1 ./gasm_test -n 64 -dd_type
>>>> gasm -dd_ksp_view
>>>>
>>>> PC Object:(dd_) 1 MPI processes
>>>> type: gasm
>>>> Generalized additive Schwarz:
>>>> Restriction/interpolation type: RESTRICT
>>>> user-defined overlap
>>>> total number of subdomains = 2
>>>> number of local subdomains = 2
>>>> max number of local subdomains = 2
>>>> [0:1] number of locally-supported subdomains = 2
>>>> Subdomain solver info is as follows:
>>>> <snip>
>>>>
>>>> What convergence are you seeing with the two PC types? It should be the
>>>> same with 1 and 2 procs for both PCASM and PCGASM.
>>>>
>>>> Thanks.
>>>> Dmitry.
>>>>
>>>> On Tue, May 15, 2012 at 4:03 AM, Hui Zhang <mike.hui.zhang at hotmail.com>
>>>> wrote:
>>>> Dmitry,
>>>>
>>>> I got the newest petsc-dev and I run the test by
>>>>
>>>> mpirun -np 1 ./gasm_test -dd_type gasm -n 64 -dd_ksp_view
>>>>
>>>> which gives the following output
>>>>
>>>> PC Object:(dd_) 1 MPI processes
>>>> type: asm
>>>> Additive Schwarz: total subdomain blocks = 1, amount of overlap = 1
>>>> ^^^
>>>> note the above number, it should
>>>> be 2
>>>>
>>>> While PCASM has no such problem.
>>>>
>>>> Thanks,
>>>> Hui
>>>>
>>>>
>>>>
>>>>> Hui,
>>>>>
>>>>> The convergence issue should be resolved in the latest petsc-dev.
>>>>> I'm attaching a slightly modified gasm_test.c (reflecting some upcoming
>>>>> API changes)
>>>>> that should verify that.
>>>>>
>>>>> Let me know if it works for you.
>>>>> Thanks.
>>>>> Dmitry.
>>>>> On Fri, May 11, 2012 at 12:31 PM, Hui Zhang <mike.hui.zhang at
>>>>> hotmail.com> wrote:
>>>>> Hi Dmitry,
>>>>>
>>>>> thanks for useful hints. Good day!
>>>>>
>>>>> Hui
>>>>>
>>>>> On May 11, 2012, at 7:17 PM, Dmitry Karpeev wrote:
>>>>>
>>>>>> You can call PCSetUp(pc) on either ASM or GASM, and that will destroy
>>>>>> and recreate the matrices (including calling
>>>>>> your modification subroutine), but not the subdomains or the subdomain
>>>>>> solvers.
>>>>>> If you just want to modify the submatrices, you can call
>>>>>> PC(G)ASMGetSubmatrices() and modify the matrices it returns
>>>>>> (in the same order as the subdomains were set). That's a bit of a hack,
>>>>>> since you will essentially be modifying the PC's internal data
>>>>>> structures. As long as you are careful, you should be okay, since you
>>>>>> already effectively have the same type of access to the submatrices
>>>>>> through the Modify callback.
>>>>>>
>>>>>> Dmitry.
>>>>>>
>>>>>> On Fri, May 11, 2012 at 11:52 AM, Hui Zhang <mike.hui.zhang at
>>>>>> hotmail.com> wrote:
>>>>>> I just have a question about reuse of PCASM or PCGASM.
>>>>>> Suppose I have seted up the PCASM and related KSP and I solved one time.
>>>>>> Next for the same linear system (matrix and RHS), I just want PCASM
>>>>>> modify the submatrices (PCSetModifySubmatrices) in a different way,
>>>>>> using the same routine for modifying but with
>>>>>> different user context for the modifying routine.
>>>>>>
>>>>>> What can I do for this task? Currently, I destroy the KSP and
>>>>>> re-construct it. I guess
>>>>>> even for PCASM I can re-use it because the partition of subdomains
>>>>>> remain the same.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On May 10, 2012, at 6:37 PM, Dmitry Karpeev wrote:
>>>>>>
>>>>>>> Hui,
>>>>>>> There've been several changes to PCGASM ahead of the new release.
>>>>>>> Let me go back and see if it affected the convergence problem.
>>>>>>> Dmitry.
>>>>>>>
>>>>>>> On Thu, May 10, 2012 at 4:16 AM, Hui Zhang <mike.hui.zhang at
>>>>>>> hotmail.com> wrote:
>>>>>>> Hi Dmitry,
>>>>>>>
>>>>>>> is there any news about PCGASM?
>>>>>>>
>>>>>>> thanks,
>>>>>>> Hui
>>>>>>>
>>>>>>> On Feb 20, 2012, at 6:38 PM, Dmitry Karpeev wrote:
>>>>>>>
>>>>>>>> Okay, thanks.
>>>>>>>> I'll take a look.
>>>>>>>>
>>>>>>>> Dmitry.
>>>>>>>>
>>>>>>>> On Mon, Feb 20, 2012 at 11:30 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>> hotmail.com> wrote:
>>>>>>>> For reference, my results are attached.
>>>>>>>>
>>>>>>>> asm1.txt for asm with 1 process,
>>>>>>>> asm2.txt for asm with 2 processes,
>>>>>>>> gasm1.txt for gasm with 1 process, (with the iteration numbers
>>>>>>>> different from others)
>>>>>>>> gasm2.txt for gasm with 2 processes
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> thank you,
>>>>>>>> Hui
>>>>>>>>
>>>>>>>> On Feb 20, 2012, at 3:06 PM, Dmitry Karpeev wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Feb 20, 2012 at 12:59 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>
>>>>>>>>> On Feb 20, 2012, at 12:41 AM, Dmitry Karpeev wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Feb 19, 2012 at 3:08 PM, Hui Zhang <mike.hui.zhang at
>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>> I have a new problem: the results from ASM and GASM are different
>>>>>>>>>> and it seems
>>>>>>>>>> GASM has something wrong with SetModifySubMatrices. Numerical tests
>>>>>>>>>> are with
>>>>>>>>>> each subdomain supported only by one subdomain. There are no
>>>>>>>>>> problems when
>>>>>>>>>> I did not modify submatrices. But when I modify submatrices, there
>>>>>>>>>> are problems
>>>>>>>>>> with GASM but no problems with ASM.
>>>>>>>>>>
>>>>>>>>>> For example, I use two subdomains. In the first case each subdomain
>>>>>>>>>> is supported by
>>>>>>>>>> one processor and there seems no problem with GASM. But when I use
>>>>>>>>>> run my program
>>>>>>>>>> with only one proc. so that it supports both of the two subdomains,
>>>>>>>>>> the iteration
>>>>>>>>>> number is different from the first case and is much larger. On the
>>>>>>>>>> other hand
>>>>>>>>>> ASM has no such problem.
>>>>>>>>>>
>>>>>>>>>> Are the solutions the same?
>>>>>>>>>> What problem are you solving?
>>>>>>>>>
>>>>>>>>> Yes, the solutions are the same. That's why ASM gives the same
>>>>>>>>> results with one or
>>>>>>>>> two processors. But GASM did not.
>>>>>>>>> Sorry, I wasn't clear: ASM and GASM produced different solutions in
>>>>>>>>> the case of two domains per processor?
>>>>>>>>> I'm solving the Helmholtz equation. Maybe
>>>>>>>>> I can prepare a simpler example to show this difference.
>>>>>>>>> That would be helpful.
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> Dmitry.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dmitry.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Feb 15, 2012, at 6:46 PM, Dmitry Karpeev wrote:
>>>>>>>>>>
>>>>>>>>>>> You should be able to.
>>>>>>>>>>> This behavior is the same as in PCASM,
>>>>>>>>>>> except in GASM the matrices live on subcommunicators.
>>>>>>>>>>> I am in transit right now, but I can take a closer look in Friday.
>>>>>>>>>>>
>>>>>>>>>>> Dmitry
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Feb 15, 2012, at 8:07, Hui Zhang <mike.hui.zhang at hotmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Feb 15, 2012, at 11:19 AM, Hui Zhang wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Dmitry,
>>>>>>>>>>>>>
>>>>>>>>>>>>> thanks a lot! Currently, I'm not using ISColoring. Just comes
>>>>>>>>>>>>> another question
>>>>>>>>>>>>> on PCGASMSetModifySubMatrices(). The user provided function has
>>>>>>>>>>>>> the prototype
>>>>>>>>>>>>>
>>>>>>>>>>>>> func (PC pc,PetscInt nsub,IS *row,IS *col,Mat *submat,void
>>>>>>>>>>>>> *ctx);
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think the coloumns from the parameter 'col' are always the same
>>>>>>>>>>>>> as the rows
>>>>>>>>>>>>> from the parameter 'row'. Because PCGASMSetLocalSubdomains() only
>>>>>>>>>>>>> accepts
>>>>>>>>>>>>> index sets but not rows and columns. Has I misunderstood
>>>>>>>>>>>>> something?
>>>>>>>>>>>>
>>>>>>>>>>>> As I tested, the row and col are always the same.
>>>>>>>>>>>>
>>>>>>>>>>>> I have a new question. Am I allowed to SetLocalToGlobalMapping()
>>>>>>>>>>>> for the submat's
>>>>>>>>>>>> in the above func()?
>>>>>>>>>>>>
>>>>>>>>>>>> thanks,
>>>>>>>>>>>> Hui
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>> Hui
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Feb 11, 2012, at 3:36 PM, Dmitry Karpeev wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, that's right.
>>>>>>>>>>>>>> There is no good way to help the user assemble the subdomains at
>>>>>>>>>>>>>> the moment beyond the 2D stuff.
>>>>>>>>>>>>>> It is expected that they are generated from mesh subdomains.
>>>>>>>>>>>>>> Each IS does carry the subdomains subcomm.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is ISColoringToList() that is supposed to convert a
>>>>>>>>>>>>>> "coloring" of indices to an array of ISs,
>>>>>>>>>>>>>> each having the indices with the same color and the subcomm that
>>>>>>>>>>>>>> supports that color. It is
>>>>>>>>>>>>>> largely untested, though. You could try using it and give us
>>>>>>>>>>>>>> feedback on any problems you encounter.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dmitry.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Feb 11, 2012 at 6:06 AM, Hui Zhang <mike.hui.zhang at
>>>>>>>>>>>>>> hotmail.com> wrote:
>>>>>>>>>>>>>> About PCGASMSetLocalSubdomains(), in the case of one subdomain
>>>>>>>>>>>>>> supported by
>>>>>>>>>>>>>> multiple processors, shall I always create the arguments 'is[s]'
>>>>>>>>>>>>>> and 'is_local[s]'
>>>>>>>>>>>>>> in a subcommunicator consisting of processors supporting the
>>>>>>>>>>>>>> subdomain 's'?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The source code of PCGASMCreateSubdomains2D() seemingly does so.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Hui
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> <gasm_test.c>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120515/3cf175e4/attachment-0001.htm>
