[deal.II] Parallel distributed hp solution transfer with FE_nothing

2020-12-07 Thread Kaushik Das
Hi all:

I modified the test  tests/mpi/solution_transfer_05.cc to add a FE_Nohting 
element to the FECollection. I also modified the other elements to FE_Q. 

When I run the test, it's aborting in solution transfer. 
Is there any limitations in using FE_Nothing with parallel solution 
transfer? 
The modified test is attached.
Thank you very much.
Kaushik 

 
An error occurred in line <1167> of file 
 in 
function
Number& 
dealii::Vector::operator()(dealii::Vector::size_type) [with 
Number = double; dealii::Vector::size_type = unsigned int]
The violated condition was:
static_cast::type)>:: type>(i) < 
static_cast::type)>:: type>(size())
Additional information:
Index 0 is not in the half-open range [0,0). In the current case, this 
half-open range is in fact empty, suggesting that you are accessing an 
element of an empty collection such as a vector that has not been set to 
the correct size.


-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/124ed0f1-8ae6-4222-b6dd-badebc278c78n%40googlegroups.com.
// -
//
// Copyright (C) 2019 - 2020 by the deal.II authors
//
// This file is part of the deal.II library.
//
// The deal.II library is free software; you can use it, redistribute
// it, and/or modify it under the terms of the GNU Lesser General
// Public License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
// The full text of the license can be found in the file LICENSE.md at
// the top level directory of deal.II.
//
// -



// This test crashed at some point: We have set and sent active_fe_indices based
// on the refinement flags on the p::d::Triangulation object. However, p4est has
// the last word on deciding which cells will be refined -- and p4est makes use
// of it in the specific scenario provided as a test. A fix has been introduced
// along with this test.


#include 

#include 
#include 

#include 
#include 

#include 
#include 

#include 

#include 
#include 

#include 

#include 

#include "tests.h"


template 
void
test()
{
  // setup
  parallel::distributed::Triangulation tria(MPI_COMM_WORLD);
  GridGenerator::hyper_cube(tria);
  tria.refine_global(2);

  const unsigned intmax_degree = 6 - dim;
  hp::FECollection fe_dgq;
  
  fe_dgq.push_back(FE_Nothing());
  for (unsigned int deg = 1; deg <= max_degree - 1; ++deg)
fe_dgq.push_back(FE_Q(deg));

  hp::DoFHandler dgq_dof_handler(tria);

  // randomly assign fes
  for (const auto  : dgq_dof_handler.active_cell_iterators())
if (cell->is_locally_owned())
  cell->set_active_fe_index(Testing::rand() % max_degree);
  dgq_dof_handler.distribute_dofs(fe_dgq);

  // prepare index sets
  IndexSet dgq_locally_owned_dofs = dgq_dof_handler.locally_owned_dofs();
  IndexSet dgq_locally_relevant_dofs;
  DoFTools::extract_locally_relevant_dofs(dgq_dof_handler,
  dgq_locally_relevant_dofs);
  IndexSet dgq_ghost_dofs = dgq_locally_relevant_dofs;
  dgq_ghost_dofs.subtract_set(dgq_locally_owned_dofs);

  // prepare dof_values
  LinearAlgebra::distributed::Vector dgq_solution;
  dgq_solution.reinit(dgq_locally_owned_dofs, dgq_ghost_dofs, MPI_COMM_WORLD);
  VectorTools::interpolate(dgq_dof_handler,
   Functions::ZeroFunction(),
   dgq_solution);
  dgq_solution.update_ghost_values();

  parallel::distributed::SolutionTransfer<
dim,
LinearAlgebra::distributed::Vector,
hp::DoFHandler>
dgq_soltrans(dgq_dof_handler);
  dgq_soltrans.prepare_for_coarsening_and_refinement(dgq_solution);

  // refine and transfer
  {
unsigned int counter = 0;
for (auto cell = tria.begin_active(); cell != tria.end(); ++cell, ++counter)
  if (cell->is_locally_owned())
{
  if (counter > ((dim == 2) ? 4 : 8))
cell->set_coarsen_flag();
  else
cell->set_refine_flag();
}
  }

  tria.execute_coarsening_and_refinement();
  dgq_dof_handler.distribute_dofs(fe_dgq);

  // prepare index sets
  dgq_locally_owned_dofs = dgq_dof_handler.locally_owned_dofs();
  DoFTools::extract_locally_relevant_dofs(dgq_dof_handler,
  dgq_locally_relevant_dofs);
  dgq_ghost_dofs = dgq_locally_relevant_dofs;
  dgq_ghost_dofs.subtract_set(dgq_locally_owned_dofs);

  // unpack dof_values
  dgq_solution.reinit(dgq_locally_owned_dofs, dgq_ghost_dofs, MPI_COMM_WORLD);
  

Re: [deal.II] Parallel distributed hp solution transfer

2019-08-31 Thread Doug

On Tuesday, August 27, 2019 at 3:51:20 AM UTC-4, Marc Fehling wrote:
>
>
> The patch has been merged. Let me know if this fix does the trick for you.
>

It does! Thank you again for the quick support.

Doug 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/909139d6-c061-48c7-9211-2c862baa22cf%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-27 Thread Marc Fehling
Hi Doug!

On Tuesday, August 27, 2019 at 3:41:11 AM UTC+2, Doug wrote:
>
> Thank you very much for the quick fix! Looking forward to pull this once 
> it goes through all the checks.
>

The patch has been merged. Let me know if this fix does the trick for you.

We introduced a test named `tests/mpi/solution_transfer_05.cc` that may be 
in a name conflict with the test you were preparing. I'm sorry about that 
if this is the case. Please adjust the filenames of your test accordingly.

Marc

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/695dd024-1069-4ac3-8d4b-eeab9daebb83%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-26 Thread Doug
Thank you very much for the quick fix! Looking forward to pull this once it 
goes through all the checks.

Doug

On Sunday, August 25, 2019 at 9:44:29 AM UTC-4, Marc Fehling wrote:
>
>
> I came up with a fix for this issue in the following PR #8637 
>  that uses the CellStatus 
> flags to determine active_fe_indices while coarsening.
>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/8fba971e-54c9-4820-9795-707f0deb584d%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-25 Thread Marc Fehling
Hi Doug! Hi Wolfgang!

On Sunday, August 25, 2019 at 3:25:06 AM UTC+2, Wolfgang Bangerth wrote:
>
> On 8/23/19 6:32 PM, Marc Fehling wrote: 
> > 
> > Your scenario indeed revealed a bug: Currently, we set and send 
> > `active_fe_indices` based on the refinement flags on the Triangulation 
> object. 
> > However, p4est has the last word on deciding which cells will be refined 
>
> That's ultimately true, but we try to match what p4est expects in the 
> Triangulation::prepare_coarsening_and_refinement() function. Are you 
> calling 
> this function after you decide which cells *you* want to refine/coarsen 
> and 
> before you execute the refinement/coarsening? 
>

We've set all refinement and coarsening flags on the p::d::Triangulation, 
called prepare_coarsening_and_refinement(), and then executed refinement. 
The transfer of active_fe_indices will be prepared during the 
pre_distributed_refinement signal, but is executed later after p4est 
performed refinement on its forest. We utilize the CellStatus flags for 
transfer.

In the scenario provided by Doug, there are neighboring cells that are 
either flagged for coarsening or for refinement. I would guess that there 
are differences in enforcing 2:1 hanging node conditions between p4est and 
deal.II.

I came up with a fix for this issue in the following PR #8637 
 that uses the CellStatus flags 
to determine active_fe_indices while coarsening.

Marc

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/02c3cb9e-fc13-47d7-bd16-1505afc7eac8%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-24 Thread Wolfgang Bangerth
On 8/23/19 6:32 PM, Marc Fehling wrote:
> 
> Your scenario indeed revealed a bug: Currently, we set and send 
> `active_fe_indices` based on the refinement flags on the Triangulation 
> object. 
> However, p4est has the last word on deciding which cells will be refined

That's ultimately true, but we try to match what p4est expects in the 
Triangulation::prepare_coarsening_and_refinement() function. Are you calling 
this function after you decide which cells *you* want to refine/coarsen and 
before you execute the refinement/coarsening?

Best
  W.

-- 

Wolfgang Bangerth  email: bange...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/97f76da8-4624-4bda-ed7d-9a9ce41f9b81%40colostate.edu.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-23 Thread Marc Fehling
Hi Doug!

Your scenario indeed revealed a bug: Currently, we set and send 
`active_fe_indices` based on the refinement flags on the Triangulation 
object. However, p4est has the last word on deciding which cells will be 
refined -- and in your specific scenario p4est makes use of it. I came up 
with a fix that should resolve this issue. Thank you for providing us with 
this example!

I'll open a pull request once my local testsuite passes.

Marc

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/89662064-f19f-4747-a851-b751abc782e1%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-21 Thread Doug
I am using the master version from 2 days ago. Commit c4c4e5209a. Actually, 
it also fails for the 2D, but in a different scenario. I run this for 
mpirun=1 and mpirun=4. 

It fails in 2D for mpirun=1.debug, with the same error. It fails for 3D for 
mpirun=4.debug, mpirun=1.release. And fully succeeds for mpirun=4.release...

I have attached the output from ctest.

On Wednesday, August 21, 2019 at 5:39:11 AM UTC-4, Marc Fehling wrote:
>
> Hi Doug!
>
> On Wednesday, August 21, 2019 at 4:00:49 AM UTC+2, Doug wrote:
>>
>> 8134: void dealii::parallel::distributed::SolutionTransfer> VectorType, DoFHandlerType>::unpack_callback(const typename 
>> dealii::parallel::distributed::Triangulation> space_dimension>::cell_iterator&, typename 
>> dealii::parallel::distributed::Triangulation> space_dimension>::CellStatus, const 
>> boost::iterator_range<__gnu_cxx::__normal_iterator> std::vector > > >&, std::vector&) 
>> [with int dim = 3; VectorType = 
>> dealii::LinearAlgebra::distributed::Vector; DoFHandlerType = 
>> dealii::hp::DoFHandler<3, 3>; typename 
>> dealii::parallel::distributed::Triangulation> space_dimension>::cell_iterator = 
>> dealii::TriaIterator >; typename 
>> dealii::parallel::distributed::Triangulation> space_dimension>::CellStatus = dealii::Triangulation<3, 3>::CellStatus]
>>
>
> It seems that the assertion fails only in the 3D case, which is quite 
> interesting. Are you using the deal.II-9.1 release version, or do you work 
> with the current master branch? I'll try to reproduce the error once I'm at 
> my desk.
>
> It looks like either p::d::SolutionTransfer picks the wrong finite element 
> while packing the solution on cells to be coarsened, or hp::DoFHandler 
> chooses the wrong finite element for the parent cell in case of coarsening. 
> I'll investigate.
>
> Marc
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/ebdf5f1a-6e10-45e1-ad66-6cbc589cfd05%40googlegroups.com.
UpdateCTestConfiguration  from 
:/home/ddong/Libraries/dealii/build/DartConfiguration.tcl
UpdateCTestConfiguration  from 
:/home/ddong/Libraries/dealii/build/DartConfiguration.tcl
Test project /home/ddong/Libraries/dealii/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 8132
Start 8132: mpi/solution_transfer_05.mpirun=1.debug

8132: Test command: /usr/bin/cmake 
"-DTRGT=solution_transfer_05.mpirun1.debug.diff" 
"-DTEST=mpi/solution_transfer_05.mpirun=1.debug" "-DEXPECT=PASSED" 
"-DBINARY_DIR=/home/ddong/Libraries/dealii/build/tests/mpi" 
"-DGUARD_FILE=/home/ddong/Libraries/dealii/build/tests/mpi/solution_transfer_05.debug/interrupt_guard.cc"
 "-P" "/home/ddong/Libraries/dealii/build/share/deal.II/scripts/run_test.cmake"
8132: Test timeout computed to be: 600
8132: make[3]: *** [solution_transfer_05.debug/mpirun=1/output] Error 1
8132: make[2]: *** [CMakeFiles/solution_transfer_05.mpirun1.debug.diff.dir/all] 
Error 2
8132: make[1]: *** 
[CMakeFiles/solution_transfer_05.mpirun1.debug.diff.dir/rule] Error 2
8132: make: *** [solution_transfer_05.mpirun1.debug.diff] Error 2
8132: Test mpi/solution_transfer_05.mpirun=1.debug: RUN
8132: ===   OUTPUT BEGIN  
===
8132: Scanning dependencies of target solution_transfer_05.debug
8132: Building CXX object 
CMakeFiles/solution_transfer_05.debug.dir/solution_transfer_05.cc.o
8132: Linking CXX executable 
solution_transfer_05.debug/solution_transfer_05.debug
8132: Built target solution_transfer_05.debug
8132: Generating solution_transfer_05.debug/mpirun=1/output
8132: mpi/solution_transfer_05.mpirun=1.debug: BUILD successful.
8132: mpi/solution_transfer_05.mpirun=1.debug: RUN failed. -- Return code 
134
8132: mpi/solution_transfer_05.mpirun=1.debug: RUN failed. -- Result: 
/home/ddong/Libraries/dealii/build/tests/mpi/solution_transfer_05.debug/mpirun=1/failing_output
8132: mpi/solution_transfer_05.mpirun=1.debug: RUN failed. -- Partial 
output:
8132: JobId ddong-XPS-13-9360 Tue Aug 20 21:50:34 2019
8132: DEAL::   2D solution transfer
8132: 
8132: mpi/solution_transfer_05.mpirun=1.debug: RUN failed. -- Additional 
output on stdout/stderr:
8132: 
8132:  written to before.eps
8132: 
8132:  written to after.eps
8132: 
8132: 
8132: 
8132: An error occurred in line <405> of file 
 in 
function
8132: void dealii::parallel::distributed::SolutionTransfer::unpack_callback(const typename 

Re: [deal.II] Parallel distributed hp solution transfer

2019-08-21 Thread Marc Fehling
Hi Doug!

On Wednesday, August 21, 2019 at 4:00:49 AM UTC+2, Doug wrote:
>
> 8134: void dealii::parallel::distributed::SolutionTransfer VectorType, DoFHandlerType>::unpack_callback(const typename 
> dealii::parallel::distributed::Triangulation space_dimension>::cell_iterator&, typename 
> dealii::parallel::distributed::Triangulation space_dimension>::CellStatus, const 
> boost::iterator_range<__gnu_cxx::__normal_iterator std::vector > > >&, std::vector&) 
> [with int dim = 3; VectorType = 
> dealii::LinearAlgebra::distributed::Vector; DoFHandlerType = 
> dealii::hp::DoFHandler<3, 3>; typename 
> dealii::parallel::distributed::Triangulation space_dimension>::cell_iterator = 
> dealii::TriaIterator >; typename 
> dealii::parallel::distributed::Triangulation space_dimension>::CellStatus = dealii::Triangulation<3, 3>::CellStatus]
>

It seems that the assertion fails only in the 3D case, which is quite 
interesting. Are you using the deal.II-9.1 release version, or do you work 
with the current master branch? I'll try to reproduce the error once I'm at 
my desk.

It looks like either p::d::SolutionTransfer picks the wrong finite element 
while packing the solution on cells to be coarsened, or hp::DoFHandler 
chooses the wrong finite element for the parent cell in case of coarsening. 
I'll investigate.

Marc

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/294c934e-5582-4c4a-a9e9-f895fa8639bb%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-20 Thread Doug
Hello Marc,

I am trying to get the test working and unfortunately it is going a lot 
less smoothly than expected. Please find attached the test 
as solution_transfer_05.cc. Only h-refinement occurs. I have attached the 
before and after grids, which look OK. It is still that error

8134: 
8134: An error occurred in line <405> of file 
 in 
function
8134: void dealii::parallel::distributed::SolutionTransfer::unpack_callback(const typename 
dealii::parallel::distributed::Triangulation::cell_iterator&, typename 
dealii::parallel::distributed::Triangulation::CellStatus, const 
boost::iterator_range<__gnu_cxx::__normal_iterator > > >&, std::vector&) 
[with int dim = 3; VectorType = 
dealii::LinearAlgebra::distributed::Vector; DoFHandlerType = 
dealii::hp::DoFHandler<3, 3>; typename 
dealii::parallel::distributed::Triangulation::cell_iterator = 
dealii::TriaIterator >; typename 
dealii::parallel::distributed::Triangulation::CellStatus = dealii::Triangulation<3, 3>::CellStatus]
8134: The violated condition was: 
8134: dof_handler->get_fe(fe_index).dofs_per_cell == 
it_dofvalues->size()
8134: Additional information: 
8134: The transferred data was packed with a different number of dofs 
than the currently registered FE object assigned to the DoFHandler has.
8134: 

Note that if I decide to refine the cells (uncomment line 150) instead of 
coarsening them (line 149), the solution transfer is successful. Playing 
around with various refinement/coarsening, it seems a bit arbitrary when it 
does work and when it won't. Am I calling anything out of order?

Doug

On Sunday, August 18, 2019 at 9:44:50 PM UTC-4, Marc Fehling wrote:
>
> Hi Doug,
>
> when dealing with distributed meshes, ownership of cells change and we may 
> not know which finite element lives on cells that the process got recently 
> assigned to. Thus, we need to transfer each cell's `active_fe_index`, which 
> we do automatically during coarsening and refinement. However, you set 
> `active_fe_indices` after refinement happened, which works in the serial 
> case, but no longer in the parallel one. Before executing refinement, you 
> need to set `future_fe_indices` that describe to which finite element your 
> cell will be assigned to, and you need to do that before refinement 
> happened! This should resolve both issues.
>
> Further, you initialize `LinearAlgebra::distrtibuted::Vector` objects 
> without any parallel distribution by using this constructor. 
> 
>  
> Try using one a different one.
>
> Please see `tests/mpi/solution_transfer_04.cc 
> `
>  
> and `tests/mpi/p_coarsening_and_refinement.cc 
> `
>  
> for working examples (I guess we should provide one using an actual 
> `SolutionTransfer` object as well), that should hopefully be applicable to 
> your problem. This is a recently added feature: If you have any suggestions 
> for improvement or encounter more problems, feel free to message us!
>
> Marc
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/8601d9b7-76f3-42c6-8972-d91ff3b640bc%40googlegroups.com.
// -
//
// Copyright (C) 1998 - 2018 by the deal.II authors
//
// This file is part of the deal.II library.
//
// The deal.II library is free software; you can use it, redistribute
// it, and/or modify it under the terms of the GNU Lesser General
// Public License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
// The full text of the license can be found in the file LICENSE.md at
// the top level directory of deal.II.
//
// -



#include 

#include 
#include 

#include 
#include 

#include 
#include 
#include 
#include 

#include 
#include 
#include 

#include 

#include 

#include 
#include 

#include 

#include 
#include 

#include "../tests.h"

// a linear function that should be transferred exactly with Q1 and Q2
// elements
template 
class MyFunction : public Function
{
public:
  MyFunction()
: Function(){};

  virtual double
  value(const Point , const unsigned int) const
  {
double f = 0.25 + 2 * 

Re: [deal.II] Parallel distributed hp solution transfer

2019-08-19 Thread Wolfgang Bangerth
On 8/19/19 11:40 AM, Doug wrote:
> Yes, will do. I have already fixed that example into a working code. I 
> just need to clean it up a little and I'll submit a pull request.
> 
> Also, I'm encountering this issue where tests fail because a zero error 
> ends up being around 1e-13 due to round-off errors. This not only 
> happens for my test, but for a lot of other deal.II tests. I see that 
> most of deal.II's testing framework relies on picking up tests based on 
> a .cc + .output file, making all the tests "regression"-like. I usually 
> have some tolerance for comparing doubles, which I thought would be the 
> common way to do this.
> 
> How do you provide tests in the deal.II testsuite to account for 
> different round-off errors that might occur?

We use a program called 'numdiff'. If cmake finds it, it uses that 
instead of regular 'diff', and then these small differences are ignored.

If you grep for 'numdiff' in the doc/ directory, there will be mention 
of this program somewhere.

Best
  W.


-- 

Wolfgang Bangerth  email: bange...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/487fe8da-bce5-f4ec-d526-3c4bb65fd5db%40colostate.edu.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-19 Thread Doug
Yes, will do. I have already fixed that example into a working code. I just 
need to clean it up a little and I'll submit a pull request.

Also, I'm encountering this issue where tests fail because a zero error 
ends up being around 1e-13 due to round-off errors. This not only happens 
for my test, but for a lot of other deal.II tests. I see that most of 
deal.II's testing framework relies on picking up tests based on a .cc + 
.output file, making all the tests "regression"-like. I usually have some 
tolerance for comparing doubles, which I thought would be the common way to 
do this. 

How do you provide tests in the deal.II testsuite to account for different 
round-off errors that might occur?

On Monday, August 19, 2019 at 1:17:10 PM UTC-4, Wolfgang Bangerth wrote:
>
> On 8/18/19 10:44 PM, Doug wrote: 
> > 
> > I'd be happy to add this test to the list if that's something you are 
> > interested in. I just have to take the time to read through how to. 
>
> I think this would be fantastic -- this area is new, so there aren't 
> that many tests yet. Anything that does what it is supposed to do will 
> therefore make for a good test, and since you already know how tests 
> look, it shouldn't even be very difficult to convert your code example 
> into a working test. Want to give this a try? 
>
> Best 
>   W. 
>
>
> -- 
>  
> Wolfgang Bangerth  email: bang...@colostate.edu 
>  
> www: http://www.math.colostate.edu/~bangerth/ 
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/a797b28a-9b74-49d2-aa74-ebcb25b02103%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-19 Thread Wolfgang Bangerth
On 8/18/19 10:44 PM, Doug wrote:
> 
> I'd be happy to add this test to the list if that's something you are 
> interested in. I just have to take the time to read through how to.

I think this would be fantastic -- this area is new, so there aren't 
that many tests yet. Anything that does what it is supposed to do will 
therefore make for a good test, and since you already know how tests 
look, it shouldn't even be very difficult to convert your code example 
into a working test. Want to give this a try?

Best
  W.


-- 

Wolfgang Bangerth  email: bange...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/d6852260-7e1b-643b-81c1-eb289fd64491%40colostate.edu.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-18 Thread Doug
Wow, I did not expect to get such a quick fix of an answer over the 
week-end. Thank you both for taking the time to answer.

The key thing here really was to use set_future_fe_index() instead of 
set_active_fe_index(). Would it make sense to discontinue one of them in 
the future? Otherwise, I would suggest adding the first paragraph you typed 
up in the documentation of the hp finite element module. Test 
tests/mpi/p_coarsening_and_refinement.cc 

 was 
indeed the perfect example to showcase the p-refinement.

Regarding the Vector constructor, I had replicated the bug even with a 
single core, so any bug related to this did not show up. In my code, I do 
give an MPI communicator.

I have modified the example to make it work. Note that in my original test, 
I was replicating a test where both DG and CG elements are refinement 
simultaneously for two different solutions. Since this 
set_future_fe_index() now occurs on grid refinement instead of occuring at 
the exact moment of set_active_fe_index(), I wouldn't expect to do the 
refinement for both DoF handlers at the same time.

I'd be happy to add this test to the list if that's something you are 
interested in. I just have to take the time to read through how to.

Anyways, it all works as expected right now. Thank you again.

Doug

On Sunday, August 18, 2019 at 9:44:50 PM UTC-4, Marc Fehling wrote:
>
> Hi Doug,
>
> when dealing with distributed meshes, ownership of cells change and we may 
> not know which finite element lives on cells that the process got recently 
> assigned to. Thus, we need to transfer each cell's `active_fe_index`, which 
> we do automatically during coarsening and refinement. However, you set 
> `active_fe_indices` after refinement happened, which works in the serial 
> case, but no longer in the parallel one. Before executing refinement, you 
> need to set `future_fe_indices` that describe to which finite element your 
> cell will be assigned to, and you need to do that before refinement 
> happened! This should resolve both issues.
>
> Further, you initialize `LinearAlgebra::distrtibuted::Vector` objects 
> without any parallel distribution by using this constructor. 
> 
>  
> Try using one a different one.
>
> Please see `tests/mpi/solution_transfer_04.cc 
> `
>  
> and `tests/mpi/p_coarsening_and_refinement.cc 
> `
>  
> for working examples (I guess we should provide one using an actual 
> `SolutionTransfer` object as well), that should hopefully be applicable to 
> your problem. This is a recently added feature: If you have any suggestions 
> for improvement or encounter more problems, feel free to message us!
>
> Marc
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/82d7a96c-221d-4771-b79f-45f34c4bc8e0%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-18 Thread Marc Fehling
Hi Doug,

when dealing with distributed meshes, ownership of cells change and we may 
not know which finite element lives on cells that the process got recently 
assigned to. Thus, we need to transfer each cell's `active_fe_index`, which 
we do automatically during coarsening and refinement. However, you set 
`active_fe_indices` after refinement happened, which works in the serial 
case, but no longer in the parallel one. Before executing refinement, you 
need to set `future_fe_indices` that describe to which finite element your 
cell will be assigned to, and you need to do that before refinement 
happened! This should resolve both issues.

Further, you initialize `LinearAlgebra::distrtibuted::Vector` objects 
without any parallel distribution by using this constructor. 

 
Try using one a different one.

Please see `tests/mpi/solution_transfer_04.cc 
`
 
and `tests/mpi/p_coarsening_and_refinement.cc 
`
 
for working examples (I guess we should provide one using an actual 
`SolutionTransfer` object as well), that should hopefully be applicable to 
your problem. This is a recently added feature: If you have any suggestions 
for improvement or encounter more problems, feel free to message us!

Marc

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/f78c5eae-4b73-4188-935f-1dd9a5f1009d%40googlegroups.com.


Re: [deal.II] Parallel distributed hp solution transfer

2019-08-17 Thread Wolfgang Bangerth


Doug,

> I am trying to use the parallel::distributed::SolutionTransfer LinearAlgebra::distributed::Vector, hp::DoFHandler> class and I 
> can't seem to use it similarly to the serial version.
> 
> I looked through the tests/distributed_grids/solution_transfer_0*.cc tests 
> and 
> none of them seem to be testing for the hp refinement.

Correct. The hp case wasn't implemented so far, but most of it made it into 
the 9.1 release. Some of it only happened on the master branch after the 
release, so I would suggest that that's what you should be working with.

I don't recall which tests you should look at, but this functionality was 
implemented by Marc Fehling, so I'm CC:ing him here.

Best
  W.

-- 

Wolfgang Bangerth  email: bange...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/56801645-f1ae-c0dc-6a74-a1ac4fea8f0c%40colostate.edu.


[deal.II] Parallel distributed hp solution transfer

2019-08-17 Thread Doug
Hello,

I am trying to use the parallel::distributed::SolutionTransfer, hp::DoFHandler> class and 
I can't seem to use it similarly to the serial version.

I looked through the tests/distributed_grids/solution_transfer_0*.cc tests 
and none of them seem to be testing for the hp refinement. The code I'm 
working on might be a bit large. Instead, I recreated the errors by copying 
the tests/hp/solution_transfer.cc, which contains SolutionTransfer for hp, 
and modified it to use the parallel distributed version of it.

Please find attached a proposed test. Line 160-175 are commented out and 
does the p-refinement. Therefore, the error below is for h-refinement only, 
but with an hp object. Oddly enough, my code works fine for h-refinement, 
but I get the same error below when I do p-refinement.

110: 
110: An error occurred in line <401> of file 
 in 
function
110: void dealii::parallel::distributed::SolutionTransfer::unpack_callback(const typename 
dealii::parallel::distributed::Triangulation::cell_iterator&, typename 
dealii::parallel::distributed::Triangulation::CellStatus, const 
boost::iterator_range<__gnu_cxx::__normal_iterator > > >&, std::vector&) 
[with int dim = 2; VectorType = 
dealii::LinearAlgebra::distributed::Vector; DoFHandlerType = 
dealii::hp::DoFHandler<2, 2>; typename 
dealii::parallel::distributed::Triangulation::cell_iterator = 
dealii::TriaIterator >; typename 
dealii::parallel::distributed::Triangulation::CellStatus = dealii::Triangulation<2, 2>::CellStatus]
110: The violated condition was: 
110: dof_handler->get_fe(fe_index).dofs_per_cell == it_dofvalues->size()
110: Additional information: 
110: The transferred data was packed with a different number of dofs 
than thecurrently registered FE object assigned to the DoFHandler has.
110: 

Now, if I uncomment those lines, and therefore do p-refinement in the 
attached test, I get

110: 
110: An error occurred in line <382> of file 
 in 
function
110: void dealii::parallel::distributed::SolutionTransfer::unpack_callback(const typename 
dealii::parallel::distributed::Triangulation::cell_iterator&, typename 
dealii::parallel::distributed::Triangulation::CellStatus, const 
boost::iterator_range<__gnu_cxx::__normal_iterator > > >&, std::vector&) 
[with int dim = 2; VectorType = 
dealii::LinearAlgebra::distributed::Vector; DoFHandlerType = 
dealii::hp::DoFHandler<2, 2>; typename 
dealii::parallel::distributed::Triangulation::cell_iterator = 
dealii::TriaIterator >; typename 
dealii::parallel::distributed::Triangulation::CellStatus = dealii::Triangulation<2, 2>::CellStatus]
110: The violated condition was: 
110: cell->child(child_index)->active_fe_index() == fe_index
110: Additional information: 
110: This exception -- which is used in many places in the library -- 
usually indicates that some condition which the author of the code thought 
must be satisfied at a certain point in an algorithm, is not fulfilled. An 
example would be that the first part of an algorithm sorts elements of an 
array in ascending order, and a second part of the algorithm later 
encounters an element that is not larger than the previous one.
110: 
110: There is usually not very much you can do if you encounter such an 
exception since it indicates an error in deal.II, not in your own program. 
Try to come up with the smallest possible program that still demonstrates 
the error and contact the deal.II mailing lists with it to obtain help.
110: 

I went through some of the deal.II source code and it's a bit out of my 
depth. Please let me know if I am somehow not p-refining correctly, or if I 
can be of any help to fix this.

Best regards,

Doug

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/1836ac96-4f6c-4d90-ad2f-5e68cbabe623%40googlegroups.com.
// -
//
// Copyright (C) 1998 - 2018 by the deal.II authors
//
// This file is part of the deal.II library.
//
// The deal.II library is free software; you can use it, redistribute
// it, and/or modify it under the terms of the GNU Lesser General
// Public License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
// The full text of the license can be found in the file LICENSE.md at
// the top level directory of deal.II.
//
//