Re: [deal.II] constraining dofs across distributed mesh

[Daniel Arndt]

> [...]
> It works in the serial case. However, it doesn't work when I use multiple 
> MPI processes. For my actual mesh, it breaks when n_processes >= 5. 
> Basically I have to make sure that dof 41 is included in the relevant dofs 
> of all the processes. How to I do that? I think this type of constraint is 
> different than a periodic boundary condition and I could not find a 
> dedicated function that takes care of it.
>
No, there is no function for that. It would of course be pretty simple if 
you just knew the value all these dofsshould be set to, but I assume that's 
not the case.

I didn't mean to say that you have to modify the locally relevant dofs, 
i.e. the IndexSet you obtain from DoFtools::locally_relevant_dofs, but tha 
tyou need to use an extended IndexSet in all the places you would normally 
use that IndexSet.
There is IndexSet::add_index that you can use to add entries to an existing 
IndexSet object.

Best,
Daniel

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/ba25e22c-428b-4073-b0a6-f485b4fba92f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] constraining dofs across distributed mesh

[Reza Rastak]

The picture I mentioned can we viewed at this link.

https://ibb.co/1nrRvfr

Reza

On Wednesday, July 10, 2019 at 8:09:51 PM UTC-7, Reza Rastak wrote:
>
> Thank you Daniel,
>
> I think the way I described the question was a little vague, so I like to 
> clarify it. Consider the simplified mesh (shown below) with 30 cells, 42 
> vertices, and 42 dofs. My desired constraint specifies that all the dofs 
> corresponding to the vertices on the right edge must have the same value. 
> So if I call the unknown field 'u', then I like to enforce u6 = u13 = u20 = 
> u27 = u34 = u41. In my current code, I do something similar to:
>
> constraint_matrix.reinit(locally_relevant_dofs);
> constraint_matrix.add_entry(34, 41, 1.0); 
> constraint_matrix.add_entry(27, 41, 1.0); 
> constraint_matrix.add_entry(20, 41, 1.0); 
> constraint_matrix.add_entry(13, 41, 1.0); 
> constraint_matrix.add_entry(6, 41, 1.0); 
>
> It works in the serial case. However, it doesn't work when I use multiple 
> MPI processes. For my actual mesh, it breaks when n_processes >= 5. 
> Basically I have to make sure that dof 41 is included in the relevant dofs 
> of all the processes. How to I do that? I think this type of constraint is 
> different than a periodic boundary condition and I could not find a 
> dedicated function that takes care of it.
>
> Thanks,
> Reza
>
>
>
>
> On Wednesday, July 10, 2019 at 7:24:47 PM UTC-7, Daniel Arndt wrote:
>>
>> Reza,
>>
>> What exactly do you mean by "link all the dofs of all the nodes along an 
>> edge of the body together"?
>> From the error message above, I assume that you want to identify certain 
>> degrees of freedom via a ConstraintMatrix (or AffineConstraints) object.
>> That is basically what DoFTools::make_periodicity_constraints is doing as 
>> well. Are you trying to do something similar?
>> In any way, you need to make sure that the ConstraintMatrix (or 
>> AffineConstraints) object can store all the degrees of freedom you want to 
>> constrain.
>> This is normally the set of locally relevant degrees and freedoms and the 
>> reason why you normally do
>>
>> constraints.reinit 
>> 
>>  
>> (locally_relevant_dofs);
>>
>> In your case, you need to make sure that you extend this IndexSet by the 
>> degrees of freedom you want to constrain. Furthermore, all processes need 
>> to know about all
>> the constraints related to these constraints. There is 
>> AffineConstraints::is_consistent_in_parallel (
>> https://www.dealii.org/current/doxygen/deal.II/classAffineConstraints.html#ad5fb8d3ca686a76ba33eb82d7f52b5fd
>> )
>> that checks that this is the case. Of course, you need to pass the 
>> modified IndexSet there as well instead of just the locally relevant dofs.
>>
>> Best,
>> Daniel
>>
>>
>> Am Mi., 10. Juli 2019 um 20:48 Uhr schrieb Reza Rastak <
>> reza@gmail.com>:
>>
>>> Hi,
>>>
>>>
>>> I like to link all the dofs of all the nodes along an edge of the body 
>>> together. So I find the first dof (dof i) located on the edge and then I 
>>> try to link all the dofs on that edge (except i) to dof i. It does not work 
>>> if I have sufficiently large number of processes where that edge is chopped 
>>> by different MPI processes. I know why it does not work, but I don't know 
>>> any methods that can provide my desired dof constraints. Anyone has 
>>> attempted to solve something similar?
>>>
>>>
>>> This is the error that I get with 8 MPI processes.
>>>
>>> 
>>>
>>> An error occurred in line <1607> of file 
>>> 
>>>  
>>> in function
>>>
>>> void 
>>> dealii::ConstraintMatrix::add_entry(dealii::ConstraintMatrix::size_type, 
>>> dealii::ConstraintMatrix::size_type, double)
>>>
>>> The violated condition was: 
>>>
>>> !local_lines.size() || local_lines.is_element(column)
>>>
>>> Additional information: 
>>>
>>> The index set given to this constraint matrix indicates constraints 
>>> using degree of freedom 580 should not be stored by this object, but a 
>>> constraint for degree of freedom 736 uses it.
>>>
>>> 
>>>
>>>
>>> Regards,
>>>
>>>
>>> Reza
>>>
>>>
>>> -- 
>>> The deal.II project is located at http://www.dealii.org/
>>> For mailing list/forum options, see 
>>> https://groups.google.com/d/forum/dealii?hl=en
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "deal.II User Group" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to dea...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/dealii/31e7aedd-b087-41d4-a412-0e211bcb27bd%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://group

Re: [deal.II] constraining dofs across distributed mesh

[Reza Rastak]

Thank you Daniel,

I think the way I described the question was a little vague, so I like to 
clarify it. Consider the simplified mesh (shown below) with 30 cells, 42 
vertices, and 42 dofs. My desired constraint specifies that all the dofs 
corresponding to the vertices on the right edge must have the same value. 
So if I call the unknown field 'u', then I like to enforce u6 = u13 = u20 = 
u27 = u34 = u41. In my current code, I do something similar to:

constraint_matrix.reinit(locally_relevant_dofs);
constraint_matrix.add_entry(34, 41, 1.0); 
constraint_matrix.add_entry(27, 41, 1.0); 
constraint_matrix.add_entry(20, 41, 1.0); 
constraint_matrix.add_entry(13, 41, 1.0); 
constraint_matrix.add_entry(6, 41, 1.0); 

It works in the serial case. However, it doesn't work when I use multiple 
MPI processes. For my actual mesh, it breaks when n_processes >= 5. 
Basically I have to make sure that dof 41 is included in the relevant dofs 
of all the processes. How to I do that? I think this type of constraint is 
different than a periodic boundary condition and I could not find a 
dedicated function that takes care of it.

Thanks,
Reza




On Wednesday, July 10, 2019 at 7:24:47 PM UTC-7, Daniel Arndt wrote:
>
> Reza,
>
> What exactly do you mean by "link all the dofs of all the nodes along an 
> edge of the body together"?
> From the error message above, I assume that you want to identify certain 
> degrees of freedom via a ConstraintMatrix (or AffineConstraints) object.
> That is basically what DoFTools::make_periodicity_constraints is doing as 
> well. Are you trying to do something similar?
> In any way, you need to make sure that the ConstraintMatrix (or 
> AffineConstraints) object can store all the degrees of freedom you want to 
> constrain.
> This is normally the set of locally relevant degrees and freedoms and the 
> reason why you normally do
>
> constraints.reinit 
> 
>  
> (locally_relevant_dofs);
>
> In your case, you need to make sure that you extend this IndexSet by the 
> degrees of freedom you want to constrain. Furthermore, all processes need 
> to know about all
> the constraints related to these constraints. There is 
> AffineConstraints::is_consistent_in_parallel (
> https://www.dealii.org/current/doxygen/deal.II/classAffineConstraints.html#ad5fb8d3ca686a76ba33eb82d7f52b5fd
> )
> that checks that this is the case. Of course, you need to pass the 
> modified IndexSet there as well instead of just the locally relevant dofs.
>
> Best,
> Daniel
>
>
> Am Mi., 10. Juli 2019 um 20:48 Uhr schrieb Reza Rastak  >:
>
>> Hi,
>>
>>
>> I like to link all the dofs of all the nodes along an edge of the body 
>> together. So I find the first dof (dof i) located on the edge and then I 
>> try to link all the dofs on that edge (except i) to dof i. It does not work 
>> if I have sufficiently large number of processes where that edge is chopped 
>> by different MPI processes. I know why it does not work, but I don't know 
>> any methods that can provide my desired dof constraints. Anyone has 
>> attempted to solve something similar?
>>
>>
>> This is the error that I get with 8 MPI processes.
>>
>> 
>>
>> An error occurred in line <1607> of file 
>>  
>> in function
>>
>> void 
>> dealii::ConstraintMatrix::add_entry(dealii::ConstraintMatrix::size_type, 
>> dealii::ConstraintMatrix::size_type, double)
>>
>> The violated condition was: 
>>
>> !local_lines.size() || local_lines.is_element(column)
>>
>> Additional information: 
>>
>> The index set given to this constraint matrix indicates constraints 
>> using degree of freedom 580 should not be stored by this object, but a 
>> constraint for degree of freedom 736 uses it.
>>
>> 
>>
>>
>> Regards,
>>
>>
>> Reza
>>
>>
>> -- 
>> The deal.II project is located at http://www.dealii.org/
>> For mailing list/forum options, see 
>> https://groups.google.com/d/forum/dealii?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "deal.II User Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to dea...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/dealii/31e7aedd-b087-41d4-a412-0e211bcb27bd%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop rec

Re: [deal.II] constraining dofs across distributed mesh

[Daniel Arndt]

Reza,

What exactly do you mean by "link all the dofs of all the nodes along an
edge of the body together"?
>From the error message above, I assume that you want to identify certain
degrees of freedom via a ConstraintMatrix (or AffineConstraints) object.
That is basically what DoFTools::make_periodicity_constraints is doing as
well. Are you trying to do something similar?
In any way, you need to make sure that the ConstraintMatrix (or
AffineConstraints) object can store all the degrees of freedom you want to
constrain.
This is normally the set of locally relevant degrees and freedoms and the
reason why you normally do

constraints.reinit

(locally_relevant_dofs);

In your case, you need to make sure that you extend this IndexSet by the
degrees of freedom you want to constrain. Furthermore, all processes need
to know about all
the constraints related to these constraints. There is
AffineConstraints::is_consistent_in_parallel (
https://www.dealii.org/current/doxygen/deal.II/classAffineConstraints.html#ad5fb8d3ca686a76ba33eb82d7f52b5fd
)
that checks that this is the case. Of course, you need to pass the modified
IndexSet there as well instead of just the locally relevant dofs.

Best,
Daniel


Am Mi., 10. Juli 2019 um 20:48 Uhr schrieb Reza Rastak <
reza.ras...@gmail.com>:

> Hi,
>
>
> I like to link all the dofs of all the nodes along an edge of the body
> together. So I find the first dof (dof i) located on the edge and then I
> try to link all the dofs on that edge (except i) to dof i. It does not work
> if I have sufficiently large number of processes where that edge is chopped
> by different MPI processes. I know why it does not work, but I don't know
> any methods that can provide my desired dof constraints. Anyone has
> attempted to solve something similar?
>
>
> This is the error that I get with 8 MPI processes.
>
> 
>
> An error occurred in line <1607> of file
> 
> in function
>
> void
> dealii::ConstraintMatrix::add_entry(dealii::ConstraintMatrix::size_type,
> dealii::ConstraintMatrix::size_type, double)
>
> The violated condition was:
>
> !local_lines.size() || local_lines.is_element(column)
>
> Additional information:
>
> The index set given to this constraint matrix indicates constraints
> using degree of freedom 580 should not be stored by this object, but a
> constraint for degree of freedom 736 uses it.
>
> 
>
>
> Regards,
>
>
> Reza
>
>
> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see
> https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dealii+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dealii/31e7aedd-b087-41d4-a412-0e211bcb27bd%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/CAOYDWb%2Be-4Xi1ESmUb64USo3RarK%3DVmhvPYof6yqcidnPqcv5A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Equivalent option for local_range() for Trilinos vectors

[Daniel Arndt]

Vivek,

just remove the line
if (i->first >= range.first && i->first < range.second)
VectorTools::interpolate_boundary_values should only set values for locally
active degrees of freedom anyway.
As long as the values are consistent between different processes, this
should work just fine.

Best,
Daniel


Am Mi., 10. Juli 2019 um 20:02 Uhr schrieb Vivek Kumar <
vivekkr1...@gmail.com>:

> Hi all,
>
> I had a legacy code were PETScWrappers::MPI::Vectors were used to for
> parallel computing. For very large problems, I was told to move to
> Trilinos. It was fairly easy to do so for most part of the code but I am
> stuck in the implementation of boundary values. Currently the boundary
> values are implemented as follows:
>
> template
> void ElasticityEquation::apply_boundary_conditions(const double time,
> const double velocity)
> {
> // Begin timer
> TimerOutput::Scope t(computing_timer, "Apply Boundary Condition");
>
> boundary_values.clear();
> VectorTools::interpolate_boundary_values
> (dof_handler,2,ZeroFunction(dim),boundary_values);
>
> VectorTools::interpolate_boundary_values
> (dof_handler,3,BoundaryValues(time, velocity) , boundary_values);
> // The range of owned dofs
> std::pair range = solution.local_range();
>
> // Set the boundary conditions
> for (typename std::map::const_iterator i =
> boundary_values.begin(); i != boundary_values.end(); ++i){
> if (i->first >= range.first && i->first < range.second){
> completely_distributed_solution(i->first) = i->second;
> }
> }
>
> completely_distributed_solution.compress(VectorOperation::insert);
>
> hanging_constraints.distribute( completely_distributed_solution);
>
> solution = completely_distributed_solution;
> }
>
>
> This used to work fine with PETSc but not with Trilinos. The issue seems
> to be the fact the Trilinos does not arrange the vector contiguous ranges
> of elements (
> https://www.dealii.org/current/doxygen/deal.II/classTrilinosWrappers_1_1MPI_1_1Vector.html#abcc22742227d63950cb0995223e6ee32)
> . Is there a standard workaround?
>
> Thanks
> Vivek
>
> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see
> https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dealii+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dealii/73ea3ef9-b5f8-461d-81c0-e43c0b4d8fbb%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/CAOYDWbK98dz_690UiyRpPdT77Lo%3D_%3Djb2Rr8ydfr-Nqy6dMBRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] constraining dofs across distributed mesh

[Reza Rastak]

Hi,


I like to link all the dofs of all the nodes along an edge of the body 
together. So I find the first dof (dof i) located on the edge and then I 
try to link all the dofs on that edge (except i) to dof i. It does not work 
if I have sufficiently large number of processes where that edge is chopped 
by different MPI processes. I know why it does not work, but I don't know 
any methods that can provide my desired dof constraints. Anyone has 
attempted to solve something similar?


This is the error that I get with 8 MPI processes.



An error occurred in line <1607> of file 
 
in function

void 
dealii::ConstraintMatrix::add_entry(dealii::ConstraintMatrix::size_type, 
dealii::ConstraintMatrix::size_type, double)

The violated condition was: 

!local_lines.size() || local_lines.is_element(column)

Additional information: 

The index set given to this constraint matrix indicates constraints 
using degree of freedom 580 should not be stored by this object, but a 
constraint for degree of freedom 736 uses it.




Regards,


Reza


-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/31e7aedd-b087-41d4-a412-0e211bcb27bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Equivalent option for local_range() for Trilinos vectors

[Vivek Kumar]

Hi all,

I had a legacy code were PETScWrappers::MPI::Vectors were used to for 
parallel computing. For very large problems, I was told to move to 
Trilinos. It was fairly easy to do so for most part of the code but I am 
stuck in the implementation of boundary values. Currently the boundary 
values are implemented as follows:

template
void ElasticityEquation::apply_boundary_conditions(const double time, 
const double velocity)
{
// Begin timer
TimerOutput::Scope t(computing_timer, "Apply Boundary Condition");

boundary_values.clear();
VectorTools::interpolate_boundary_values 
(dof_handler,2,ZeroFunction(dim),boundary_values);

VectorTools::interpolate_boundary_values 
(dof_handler,3,BoundaryValues(time, velocity) , boundary_values);
// The range of owned dofs
std::pair range = solution.local_range();

// Set the boundary conditions
for (typename std::map::const_iterator i = 
boundary_values.begin(); i != boundary_values.end(); ++i){
if (i->first >= range.first && i->first < range.second){
completely_distributed_solution(i->first) = i->second;
}
}

completely_distributed_solution.compress(VectorOperation::insert);

hanging_constraints.distribute( completely_distributed_solution);

solution = completely_distributed_solution;
}


This used to work fine with PETSc but not with Trilinos. The issue seems to 
be the fact the Trilinos does not arrange the vector contiguous ranges of 
elements (
https://www.dealii.org/current/doxygen/deal.II/classTrilinosWrappers_1_1MPI_1_1Vector.html#abcc22742227d63950cb0995223e6ee32)
 
. Is there a standard workaround?

Thanks
Vivek

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/73ea3ef9-b5f8-461d-81c0-e43c0b4d8fbb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Can I get the cell numbers of the cells neighbouring a face?

[Bruno Turcksin]

Stephen

On Wednesday, July 10, 2019 at 3:56:02 PM UTC-4, Stephen wrote:
>
> I have an error estimator which I need to calculate between edges and I'd 
> like to loop over all edges in the triangulation. I know that I can do this 
> the "traditional" way via looping over all cells and then looping over all 
> faces in the cell but, if I do this, I end up calculating the same thing 
> twice since I then run over non-boundary faces once from one cell and then 
> again from it's neighbour; this is not acceptable since efficiency is 
> really of paramount importance in the code I'm currently writing. Is it 
> possible to instead loop over all active edges in the triangulation and 
> then find the cell IDs of the cells which neighbour this face? This way I 
> should be able to calculate the terms directly on the edge instead of 
> trying to go via the cell first. Thanks!
>
What you can do is to check that the face is not on the boundary and in 
that case only compute your error on the left and the bottom edges (for 
example). I think you could also use this function 

 
to do all the heavy lifting for you.

Best,

Bruno

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/a9f98552-5ca3-4bf6-a54e-01fed8f46ade%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Can I get the cell numbers of the cells neighbouring a face?

[Stephen]

I have an error estimator which I need to calculate between edges and I'd 
like to loop over all edges in the triangulation. I know that I can do this 
the "traditional" way via looping over all cells and then looping over all 
faces in the cell but, if I do this, I end up calculating the same thing 
twice since I then run over non-boundary faces once from one cell and then 
again from it's neighbour; this is not acceptable since efficiency is 
really of paramount importance in the code I'm currently writing. Is it 
possible to instead loop over all active edges in the triangulation and 
then find the cell IDs of the cells which neighbour this face? This way I 
should be able to calculate the terms directly on the edge instead of 
trying to go via the cell first. Thanks!

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/bcbfcaf2-25ef-43d0-8bed-4ea7c2fe0088%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Installation error on Haswell nodes on Cori at NERSC (failed AVX512 support)

[Martin Kronbichler]

Dear Steve,

>From what I can see the failure is for the expand_instantiations script
of deal.II, which is compiled as part of deal.II. It uses slightly
different flags as the full install, but assuming that you either passed
-xHASWELL or -xCORE-AVX2 to CMAKE_CXX_FLAGS it should not generate that
code. Before we look into the flags used for compilation, a basic
question: Did you start with a clean build directory without any file
left over from a more advanced architecture with AVX512 support?

Best,
Martin

On 10.07.19 15:56, Stephen DeWitt wrote:
> Hello,
> I'm trying to install deal.II on the Haswell nodes on Cori at NERSC
> using the Intel compiler. I'm using deal.II version 9.0, because
> support for a few of the function calls I make were dropped in v9.1
> and I haven't had a chance to modify the sections of the code. In my
> CMake command, I'm adding either the -xHASWELL or -xCORE-AVX2 flags to
> get the right level of vectorization for this architecture (AVX). The
> CMake output is what I expect:
>
> |
> -- Performing Test DEAL_II_HAVE_SSE2
> -- Performing Test DEAL_II_HAVE_SSE2 - Success
> -- Performing Test DEAL_II_HAVE_AVX
> -- Performing Test DEAL_II_HAVE_AVX - Success
> -- Performing Test DEAL_II_HAVE_AVX512
> -- Performing Test DEAL_II_HAVE_AVX512 - Failed
> -- Performing Test DEAL_II_HAVE_OPENMP_SIMD
> -- Performing Test DEAL_II_HAVE_OPENMP_SIMD - Success
> |
>
> However, when I try to compile I get this error:
>
> |
> [ 32%] Built target obj_umfpack_DL_STORE_release
> [ 34%] Built target obj_amd_global_release
> [ 35%] Built target obj_amd_long_release
> [ 36%] Built target obj_amd_int_release
> [ 37%] Built target obj_muparser_release
> [ 37%] Built target obj_sundials_inst
> Scanning dependencies of target obj_sundials_release
> [ 37%] Building CXX object
> source/sundials/CMakeFiles/obj_sundials_release.dir/arkode.cc.o
> [ 37%] Building CXX object
> source/sundials/CMakeFiles/obj_sundials_release.dir/ida.cc.o
> [ 37%] Building CXX object
> source/sundials/CMakeFiles/obj_sundials_release.dir/copy.cc.o
> [ 38%] Building CXX object
> source/sundials/CMakeFiles/obj_sundials_release.dir/kinsol.cc.o
> [ 38%] Built target obj_sundials_release
> [ 38%] Generating data_out_dof_data.inst
>
> Please verify that both the operating system and the processor support
> Intel(R) AVX512F, ADX, AVX512ER, AVX512PF and AVX512CD instructions.
>
> source/numerics/CMakeFiles/obj_numerics_inst.dir/build.make:91: recipe
> for target 'source/numerics/data_out_dof_data.inst' failed
> make[2]: *** [source/numerics/data_out_dof_data.inst] Error 1
> CMakeFiles/Makefile2:1860: recipe for target
> 'source/numerics/CMakeFiles/obj_numerics_inst.dir/all' failed
> make[1]: *** [source/numerics/CMakeFiles/obj_numerics_inst.dir/all]
> Error 2
> Makefile:129: recipe for target 'all' failed
> make: *** [all] Error 2
> |
>
> It seems to me that it is looking for AVX512 support and generating an
> error when it doesn't find it. But why would it look for AVX512
> support if DEAL_II_HAVE_AVX512 failed? I haven't had this issue when
> installing on other machines that support AVX but not AVX512.
>
> Thanks for any insight you have,
> Steve
> -- 
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see
> https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dealii+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dealii/b36fd46d-2dff-41f4-8762-8ffad6ad4da1%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/daa0d6eb-090a-9876-bd94-c70876378752%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Installation error on Haswell nodes on Cori at NERSC (failed AVX512 support)

[Stephen DeWitt]

Hello,
I'm trying to install deal.II on the Haswell nodes on Cori at NERSC using 
the Intel compiler. I'm using deal.II version 9.0, because support for a 
few of the function calls I make were dropped in v9.1 and I haven't had a 
chance to modify the sections of the code. In my CMake command, I'm adding 
either the -xHASWELL or -xCORE-AVX2 flags to get the right level of 
vectorization for this architecture (AVX). The CMake output is what I 
expect:

-- Performing Test DEAL_II_HAVE_SSE2
-- Performing Test DEAL_II_HAVE_SSE2 - Success
-- Performing Test DEAL_II_HAVE_AVX
-- Performing Test DEAL_II_HAVE_AVX - Success
-- Performing Test DEAL_II_HAVE_AVX512
-- Performing Test DEAL_II_HAVE_AVX512 - Failed
-- Performing Test DEAL_II_HAVE_OPENMP_SIMD
-- Performing Test DEAL_II_HAVE_OPENMP_SIMD - Success

However, when I try to compile I get this error:

[ 32%] Built target obj_umfpack_DL_STORE_release
[ 34%] Built target obj_amd_global_release
[ 35%] Built target obj_amd_long_release
[ 36%] Built target obj_amd_int_release
[ 37%] Built target obj_muparser_release
[ 37%] Built target obj_sundials_inst
Scanning dependencies of target obj_sundials_release
[ 37%] Building CXX object 
source/sundials/CMakeFiles/obj_sundials_release.dir/arkode.cc.o
[ 37%] Building CXX object 
source/sundials/CMakeFiles/obj_sundials_release.dir/ida.cc.o
[ 37%] Building CXX object 
source/sundials/CMakeFiles/obj_sundials_release.dir/copy.cc.o
[ 38%] Building CXX object 
source/sundials/CMakeFiles/obj_sundials_release.dir/kinsol.cc.o
[ 38%] Built target obj_sundials_release
[ 38%] Generating data_out_dof_data.inst

Please verify that both the operating system and the processor support 
Intel(R) AVX512F, ADX, AVX512ER, AVX512PF and AVX512CD instructions.

source/numerics/CMakeFiles/obj_numerics_inst.dir/build.make:91: recipe for 
target 'source/numerics/data_out_dof_data.inst' failed
make[2]: *** [source/numerics/data_out_dof_data.inst] Error 1
CMakeFiles/Makefile2:1860: recipe for target 
'source/numerics/CMakeFiles/obj_numerics_inst.dir/all' failed
make[1]: *** [source/numerics/CMakeFiles/obj_numerics_inst.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

It seems to me that it is looking for AVX512 support and generating an 
error when it doesn't find it. But why would it look for AVX512 support if 
DEAL_II_HAVE_AVX512 failed? I haven't had this issue when installing on 
other machines that support AVX but not AVX512.

Thanks for any insight you have,
Steve

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/b36fd46d-2dff-41f4-8762-8ffad6ad4da1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Output subset of cells

[Alexander]

Thanks, Daniel, looks appropriate! 

On Friday, July 5, 2019 at 8:05:42 PM UTC+2, Daniel Arndt wrote:
>
> Alexander,
>
> have a look at 
> https://www.dealii.org/developer/doxygen/deal.II/classDataOut.html#afdefbb966f2053f4c4fc52293d2339ce
> .
> You just need to overload DataOut::next_cell().
>
> Best,
> Daniel
>
> Am Fr., 5. Juli 2019 um 10:34 Uhr schrieb Alexander  >:
>
>> I am starting to hit a space/memory boundary since my VTU output becomes 
>> > 100 GB. Partial mitigation to this would be to output (DataOut) only some 
>> cells in VTU, which are of interest. In my applications, I can easily 
>> ignore at least half of cells. 
>>
>> Is there a possibility to do that within the existing interface?
>>
>> Best
>> Alexander
>>
>>
>> -- 
>> The deal.II project is located at http://www.dealii.org/
>> For mailing list/forum options, see 
>> https://groups.google.com/d/forum/dealii?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "deal.II User Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to dea...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/dealii/0464e017-bada-410b-a1af-ea51637d7eee%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/4bd3ae72-40e3-40cb-b53e-4aedab064beb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.