I was unable to do quad precision or even with 64 bit integers because my data files rely on intricate binary files that have been written in 32 bit.
However, I noticed a couple things which are puzzling to me: 1) I am solving a transient problem using my own backward euler function. Basically I call TaoSolve at each time level. What I find strange is that the number of TAO solve iterations vary at each time level for a given number of processors. The solution is roughly the same when I change the number of processors. Any idea why this is happening, or might this have more to do with the job scheduling/compute nodes on my HPC machine? 2) Sometimes, I get Tao Termination reason of -5, and from what I see from the online documentation, it means the number of function evaluations exceeds the maximum number of function evaluations. I only get this at certain time levels, and it also varies when I change the number of processors. I can understand the number of iterations going down the further in time i go (this is due to the nature of my problem), but I am not sure why the above two observations are happening. Any thoughts? Thanks, Justin On Fri, Jun 19, 2015 at 11:52 AM, Justin Chang <[email protected]> wrote: > My code sort of requires HDF5 so installing quad precision might be a > little difficult. I could try to work around this but that might take some > effort. > > In the mean time, is there any other potential explanation or alternative > to figuring this out? > > Thanks, > Justin > > > On Thursday, June 18, 2015, Matthew Knepley <[email protected]> wrote: > >> On Thu, Jun 18, 2015 at 1:52 PM, Jason Sarich <[email protected]> >> wrote: >> >>> BLMVM doesn't use a KSP or preconditioner, it updates using the L-BFGS-B >>> formula >>> >> >> Then this sounds like a bug, unless one of the constants is partition >> dependent. >> >> Matt >> >> >>> On Thu, Jun 18, 2015 at 1:45 PM, Matthew Knepley <[email protected]> >>> wrote: >>> >>>> On Thu, Jun 18, 2015 at 12:15 PM, Jason Sarich < >>>> [email protected]> wrote: >>>> >>>>> Hi Justin, >>>>> >>>>> I can't tell for sure why this is happening, have you tried using >>>>> quad precision to make sure that numerical cutoffs isn't the problem? >>>>> >>>>> 1 The Hessian being approximate and the resulting implicit >>>>> computation is the source of the cutoff, but would not be causing >>>>> different >>>>> convergence rates in infinite precision. >>>>> >>>>> 2 the local size may affect load balancing but not the resulting >>>>> norms/convergence rate. >>>>> >>>> >>>> This sounds to be like the preconditioner is dependent on the >>>> partition. Can you send -tao_view -snes_view >>>> >>>> Matt >>>> >>>> >>>>> Jason >>>>> >>>>> >>>>> On Thu, Jun 18, 2015 at 10:44 AM, Justin Chang <[email protected]> >>>>> wrote: >>>>> >>>>>> I solved a transient diffusion across multiple cores using TAO >>>>>> BLMVM. When I simulate the same problem but on different numbers of >>>>>> processing cores, the number of solve iterations change quite >>>>>> drastically. >>>>>> The numerical solution is the same, but these changes are quite vast. I >>>>>> attached a PDF showing a comparison between KSP and TAO. KSP remains >>>>>> largely invariant with number of processors but TAO (with bounded >>>>>> constraints) fluctuates. >>>>>> >>>>>> My question is, why is this happening? I understand that accumulation >>>>>> of numerical round-offs may attribute to this, but the differences seem >>>>>> quite vast to me. My initial thought was that >>>>>> >>>>>> 1) the Hessian is only projected and not explicitly computed, which >>>>>> may have something to do with the rate of convergence >>>>>> >>>>>> 2) local problem size. Certain regions of my domain have different >>>>>> number of "violations" which need to be corrected by the bounded >>>>>> constraints so the rate of convergence depends on how these regions are >>>>>> partitioned? >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> Thanks, >>>>>> Justin >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >
