Forgot to mention something: This run has only 7 refinement levels in total (i.e., the coarsest level, plus 6 additional levels of refinement), so every 64 timesteps, all levels should be updated.
-Zach * * * Zachariah Etienne Assistant Professor of Mathematics West Virginia University On Tue, Jun 2, 2015 at 11:11 AM, Zach Etienne <[email protected]> wrote: > Thanks for your quick feedback, Ian! > > > Think about the contents of the three timelevels of myadmbaselapse. > > myadmbaselapse has only one timelevel, and tags are set > 'InterpNumTimelevels=1 prolongation="none" Checkpoint="no"'. Thus > prolongation should be disabled on myadmbaselapse... > > Just prior to the output at, e.g., iteration 192 (the iteration when the > grid movement occurs), myadmbaselapse is set to ADMBase::alp, at all > refinement levels (due to GLOBAL,LOOP-LOCAL). Therefore, it should not > matter how often I choose to update myadmbaselapse, since the value just > prior to the file output (at iteration 192, for instance) is updated in a > GLOBAL,LOOP-LOCAL mode. So why do I see vastly different results when I > choose to update myadmbaselapse every cctk_iteration versus every 64? > > > -Zach > > * * * > Zachariah Etienne > Assistant Professor of Mathematics > West Virginia University > > On Tue, Jun 2, 2015 at 11:02 AM, Ian Hinder <[email protected]> wrote: > >> >> On 2 Jun 2015, at 16:27, Zach Etienne <[email protected]> wrote: >> >> > Hello. >> > >> > I am working on writing a diagnostic that reads in ADMBase variables >> (e.g., alp, betax, etc.) from a binary black hole simulation using >> McLachlan. The diagnostic is computed at CCTK_ANALYSIS, in a >> GLOBAL,LOOP-LOCAL context. >> > >> > Call the gridfunction quantity computed by this diagnostic, >> diag_data_gf. diag_data_gf is rather expensive to compute in general, so I >> only want it computed and output (to, e.g., IOASCII 2D data files) every 64 >> iterations. >> > >> > Setting regrid_every=64, I found that the diagnostic outputs >> reasonable-looking data every 64 iterations until the first actual AMR grid >> movement (at iteration 192). At this iteration, a large number of zones >> near AMR refinement boundaries in the IOASCII 2D output of diag_data_gf are >> set to undefined values (I have set IOASCII::out3D_ghosts=no). This is >> undesirable behavior! >> > >> > Here is what I want to happen: >> > 1) every 64 iterations, the diagnostic computes diag_data_gf at all >> gridpoints on all refinement levels at CCTK_ANALYSIS (I believe >> GLOBAL,LOOP-LOCAL should do this). >> > 2) After CCTK_ANALYSIS, diag_data_gf should be output to files >> > >> > So why does my scheduling choice yield obviously wrong values near AMR >> boundaries after the grids move. >> > ******** >> > Now here is where the situation becomes very weird: >> > ******** >> > When I set diag_data_gf to be computed at *every* iteration, the >> undefined value problem disappears! Remember, it is being called at >> CCTK_ANALYSIS in a GLOBAL,LOOP-LOCAL context, so diag_data_gf should be >> recomputed at all points on all levels prior to file output. Where are >> these mysterious undefined values coming from?! >> >> > Upon further analysis, even at iteration 64 the diagnostic yields >> inconsistent results at all refinement levels except the finest one. >> >> > >> > ******** >> > I have created a very simple thorn called ADMBaseMcLachlanTester ( >> math.wvu.edu/~zetienne/ADMBaseMcLachlanTester.tar.gz) that reproduces >> this problem (in both ET 2014 11 and ET 2015 05 releases) with a minimum of >> coding. All the thorn does is set a gridfunction called "myadmbaselapse", >> which appropriately enough, is set to ADMBase::alp at all gridpoints. >> > >> > In the thorn's par/ subdirectory, you'll find 2 parfiles: >> qc0-mclachlan-setlapseevery1.par and qc0-mclachlan-setlapseevery64.par. The >> former sets myadmbaselapse=alp at every cctk_iteration in CCTK_ANALYSIS, >> and the latter every 64 cctk_iteration's. You will notice that the former >> parfile yields reasonable data at iteration 192 in the >> admbasemclachlantester::admbasemclachlantestergfs.*.asc files. However, >> many nan's are produced at iteration 192 (corresponding to the first AMR >> grid movement) when using the latter parfile. >> > >> > What is causing this weird behavior? Have I uncovered a bug? >> >> If you regrid level L, Carpet will do time prolongation to fill the >> points new to the level if level L-1 (the coarser one) does not exist at >> the regridding time. It needs to do this, since it doesn't have any grid >> points to do a purely-spatial interpolation. Think about the contents of >> the three timelevels of myadmbaselapse. Timelevels are cycled every >> iteration, whether you have set them or not. Since you are setting >> timelevel 0 only every 64 iterations, the past two timelevels do not >> contain valid data for the previous two timesteps, so the points which need >> to be filled by time prolongation will contain bad data. If you can >> partition your grids into "moving" and "fixed", and then ensure that you >> only regrid when the fixed grids exist, then you can avoid time >> interpolation when regridding, and you won't need valid data on the past >> timelevels. There is also a parameter >> >> Carpet::time_interpolation_during_regridding ( >> https://bitbucket.org/eschnett/carpet/src/59e2074462e38e9a1574d1479d96b2701cf8d1c5/Carpet/param.ccl?at=master#cl-586 >> ) >> >> but I'm not sure exactly what it does. A quick grep of the Carpet source >> should reveal this. It might be helpful. But the best option is probably >> to just arrange your regridding frequency so that you don't need to >> interpolate in time. i.e., if your first fixed grid exists every 32 >> iterations, then only regrid on multiples of 32 iterations. >> >> There is another parameter, if you are using CarpetRegrid2, >> >> CarpetRegrid2::freeze_unaligned_parent_levels >> Do not change refinement levels where the parent does not exist >> at this time >> >> ( >> https://bitbucket.org/eschnett/carpet/src/59e2074462e38e9a1574d1479d96b2701cf8d1c5/CarpetRegrid2/param.ccl?at=master#cl-26 >> ) >> >> but I think this will stop the regridding from actually happening, so you >> would have to be careful that you do eventually regrid on an iteration when >> the grid is aligned with its parent. >> >> -- >> Ian Hinder >> http://members.aei.mpg.de/ianhin >> >> >
_______________________________________________ Users mailing list [email protected] http://lists.einsteintoolkit.org/mailman/listinfo/users
