[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear Bruno,

Thanks. That's even better and even in just a single line of code. ;)

MPI_Bcast(, 1, MPI_DOUBLE, max_rank, mpi_com);

Best,
Seyed Ali

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear Bruno,

Thanks. That's even better and even in just a single line of code. ;)

MPI_Bcast(, 1, MPI_DOUBLE, max_rank, mpi_com);



Best,
Seyed Ali

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Bruno Turcksin]

Seyed,

you probably want to use MPI_Bcast instead of your code (see 
http://mpitutorial.com/tutorials/mpi-broadcast-and-collective-communication/)

Best,

Bruno

On Wednesday, March 8, 2017 at 9:51:50 AM UTC-5, Seyed Ali Mohseni wrote:
>
> Dear all,
>
> I was able to solve the MPI variable problem. 
>
> For the users who may have the same issue once:
>
> if ( this_mpi_process == max_rank )
> {
>  for (unsigned int i = 0; i < n_mpi_processes; ++i)
>  {
>   if ( i != max_rank )
>MPI_Send(, 1, MPI_DOUBLE, i, 0, mpi_com);
>  }
> }
> else
> {
>  MPI_Recv(, 1, MPI_DOUBLE, max_rank, 0, mpi_com, 
> MPI_STATUS_IGNORE);
>  printf("Process %d received number from process %d\n", 
> this_mpi_process, max_rank);
> }
>
> A double variable called position (just as an example here, you have to 
> pass x,y,z, so 3 variables) can be defined earlier which has then to be 
> filled on the rank equal to max_rank. 
>
> Finally, I can proceed. The code was tested for ranks 1 to 16 and works 
> perfectly. Every time the maximum value and position remains correct and 
> unchanged, but the maximum rank (max_rank) changes dependent on the core 
> numbers.
>
> Thank you a lot. 
> The community here is extraordinary. ;)
>
> Best regards,
> S. A. Mohseni
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear all,

I was able to solve the MPI variable problem. 

For the users who may have the same issue once:

if ( this_mpi_process == max_rank )
{
 for (unsigned int i = 0; i < n_mpi_processes; ++i)
 {
  if ( i != max_rank )
   MPI_Send(, 1, MPI_DOUBLE, i, 0, mpi_com);
 }
}
else
{
 MPI_Recv(, 1, MPI_DOUBLE, max_rank, 0, mpi_com, 
MPI_STATUS_IGNORE);
 printf("Process %d received number from process %d\n", 
this_mpi_process, max_rank);
}

A double variable called position (just as an example here, you have to 
pass x,y,z, so 3 variables) can be defined earlier which has then to be 
filled on the rank equal to max_rank. 

Finally, I can proceed. The code was tested for ranks 1 to 16 and works 
perfectly. Every time the maximum value and position remains correct and 
unchanged, but the maximum rank (max_rank) changes dependent on the core 
numbers.

Thank you a lot. 
The community here is extraordinary. ;)

Best regards,
S. A. Mohseni

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear all,

I was able to solve the MPI variable problem. 

For the users who may have the same issue once:

if ( this_mpi_process == max_rank )
{
 for (unsigned int i = 0; i < n_mpi_processes; ++i)
 {
  if ( i != max_rank )
   MPI_Send(, 1, MPI_DOUBLE, i, 0, mpi_com);
 }
}
else
{
 MPI_Recv(, 1, MPI_DOUBLE, max_rank, 0, mpi_com, 
MPI_STATUS_IGNORE);
 printf("Process %d received number from process %d\n", 
this_mpi_process, max_rank);
}

A double variable called position (just as an example here, you have to 
pass x,y,z, so 3 variables) has been defined earlier which was filled on 
the rank equal to max_rank. 

Finally, I can proceed. The code was tested for ranks 1 to 16 and works 
perfectly. Every time the maximum value and position remains correct and 
unchanged, but the maximum rank (max_rank) changes dependent on the core 
numbers.

Thank you a lot. 
The community here is extraordinary. ;)

Best regards,
S. A. Mohseni

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Jean-Paul Pelteret]

Dear Seyed,
 

> Extra question: Can we store variables or copy them to all processors? 
> Since I am filling a variable in a locally owned cell, currently on rank 3. 
> Then my other ranks, especially the root rank 0 has no clue of the values 
> which are set. Hence, there has to be a possibility to copy data to other 
> processors or let them know about the content?
>
 
This falls into the category of standard, but very much nontrivial, MPI 
questions that are unrelated to deal.II itself. How you would implement the 
point-to-point 
communication 
 of data 
depends very much on, amongst other things, the type of data that is being 
transferred, and how/to which processes the information is being broadcast 
and received. 

So I would highly recommend that you take a look at the documentation 
 for the MPI library that you are using 
and perhaps also go through a couple of tutorials that focus on MPI and 
its communication models itself. Otherwise, there are examples 
 in the 
source code and tests 
 
that you can use to better understand how you might implement whatever it 
is that you're trying to do.

Regards,
Jean-Paul

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

@Prof. Bangerth: I will try to learn debugging parallel code and also to 
follow your words of advice. Please do not apologize since I don't think 
you ever became rude. I mean, after 1 emails I would definitely not be 
as calm as you. Thank you.  :) 

@Daniel: Thumbs up. From your words I realize, there is still a lot to 
learn about deal.II yet. To be honest, I always try to solve it in a simple 
way than I extend it to the general case such as 3D, DG elements, etc. 
Thank you for all the help.

Extra question: Can we store variables or copy them to all processors? 
Since I am filling a variable in a locally owned cell, currently on rank 3. 
Then my other ranks, especially the root rank 0 has no clue of the values 
which are set. Hence, there has to be a possibility to copy data to other 
processors or let them know about the content?

Kind regards,
Seyed Ali
 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Daniel Arndt]

Seyed,

after having just a quick glance over your approach there seem to be some 
more issues you can easily stumble upon:
- n_vertices() only gives you the number of vertices for one process, not 
the global one. In particular, you can't rely on the fact that this is the 
same for all processes. Furthermore, you (probably) don't initialize all 
the values of your struct. This might be the reason for the large numbers 
you are observing.
- You seem to rely on a specific numbering of the degrees of freedom in 
FEValues. In particular, you are assuming that you have only nodal values 
at the cell vertices. This can only be true for Q1 elements. Furthermore, 
you rely on the fact that after a local dof for the x-component, a local 
dof for the y-component is stored. This is an unsafe assumption. The proper 
way of doing this is to use an appropriate Quadrature object that is 
initialized with support points at the points you are interested in. Have a 
look at the FAQ [1,2].
- You seem to first calculate the maximum for each vertex across all the 
processes and after that the maximum over all vertices. It would probably 
be easier to first find the maximum value and location on each process and 
after that just compare this one value across all the processes. This would 
probably also avoid your problems with large numbers.

Else I can just agree with Wolfgang, learning how to debug is a crucial 
point if you ever want to build working software.

Best,
Daniel

[1] 
https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#how-do-i-access-values-of-discontinuous-elements-at-vertices
 

[2] 
https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#how-to-get-the-mapped-position-of-support-points-of-my-element

Am Dienstag, 7. März 2017 16:55:47 UTC+1 schrieb Seyed Ali Mohseni:
>
> Dear all,
>
> With MPI_Allreduce it works like a charm. Thank you very much everyone.
>
> Especially Prof. Bangerth and Daniel.
>
> Kind regards,
> S. A. Mohseni
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear all,

With MPI_Allreduce it works like a charm. Thank you very much everyone.

Especially Prof. Bangerth and Daniel.

Kind regards,
S. A. Mohseni

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Re: Access specific element within a distributed triangulation

[Wolfgang Bangerth]

On 03/07/2017 07:46 AM, 'Seyed Ali Mohseni' via deal.II User Group wrote:


Now my question how can I store the data on all processors?
Or how am I able to at least store my max_rank variable on all processors?


The function you're looking for is called MPI_Allreduce.
Best
 W.


--

Wolfgang Bangerth  email: bange...@colostate.edu
   www: http://www.math.colostate.edu/~bangerth/

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

I think I figured it out ;) 
After thinking about your suggestion, Prof. Bangerth.
It came to my mind, that the result of my max_rank variable is stored on a 
specific processor due to MPI_Reduce.
That means I cannot access these data parts without being currently at the 
correspinding processor rank.
For instance I store my results on rank 0, if I want to check the max_rank 
data on processor rank 3, it is empty.
That is why I cannot get any output, because when rank 3 owns everything 
output works fine.

Now my question how can I store the data on all processors?
Or how am I able to at least store my max_rank variable on all processors?

BR,
Seyed Ali 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Re: Access specific element within a distributed triangulation

[Wolfgang Bangerth]

On 03/07/2017 05:31 AM, 'Seyed Ali Mohseni' via deal.II User Group wrote:

Now the funny part is, if I set max_rank manually such as max_rank = 3 for
instance, it works and for the currently owned rank I receive an output within
terminal. Another thing is that MPI_Reduce creates somewhere some big values,
which is why I fixed it by erasing values greater for 1e10.


Seyed -- you need to learn to debug these things. Just ignoring values greater 
than 1e10 means that you don't understand why these values are there -- but 
then how can you be sure that the *other* values are correct?


You need to learn strategies to figure these things out. Run the program in a 
debugger. Print the values that you send to MPI_Reduce and compare what that 
function returns with what you *expect* it to return, etc. You cannot write 
software that does what it is supposed to do if you don't understand what it 
computes. Learning strategies to debug software is the only way you can learn 
to write good software.


Best
 WB

--

Wolfgang Bangerth  email: bange...@colostate.edu
   www: http://www.math.colostate.edu/~bangerth/

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Daniel Arndt]

Seyed,
 

> But is it necessary to #include  ?
> Since deal.II won't recognize functions such as MPI_Comm_rank 
> (MPI_COMM_WORLD, 
> ) for instance or MPI_Reduce.
> Is it implemented in Utilities::MPI::something ? 
>
If you are using a parallel Triangulation you automatically include 
"". You can be sure that the comiler will tell you if you need to 
include that file explicitly.

Best,
Daniel

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear Daniel,

Thanks a lot. I understand now.
Example 3 is exactly what I need, I try the MPI_MAXLOC approach.

But is it necessary to #include  ?
Since deal.II won't recognize functions such as MPI_Comm_rank 
(MPI_COMM_WORLD, 
) for instance or MPI_Reduce.
Is it implemented in Utilities::MPI::something ? 

Kind regards,
Seyed Ali

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear Daniel,

Thank you. Does your approach also apply to MPICH? Since I am using MPICH, 
I wonder, if this command can be used there also.
This struct variable, is there an example within deal.II steps? Or do you 
have an example in C++ I can learn from. 

To be honest I am not yet very familiar with MPI itself, barely 
understanding deal.II parallelism ;)

I have also another idea: How about I find the maximum value regardless of 
the position and processor core. Then just loop again over all cells and 
compare the maximum value with the existing value within each cell. Would 
this be a very inefficient approach or is looping over cells usually 
computationally cheap in deal.II.

Kind regards,
Seyed Ali 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Daniel Arndt]

Seyed,

What I understood so far is, that the Utilities::MPI::max() function 
> computes the maximum between all processors for each row of my vector. This 
> means if I have a vector of 20 entries, each entry has 8 values distributed 
> on 8 cores for instance. As a result, I get 20 maximum values again, but 
> this time they are maximal and processor independent. 
>
Correct.
 

> Hence, my question now is how am I able to check where the maximum value 
> comes from? Is it possible to output the processor number of the maximum 
> value for each row?
>
If you look at the implementation of Utilities::MPI::max() [2], you see 
that this is essentially a wrapper around MPI_Allreduce called in [3].
What you want is currently not implemented in deal.II, but MPI_MAXLOC is 
exactly doing this. Example 3 in [1] shows how to use this, i.e.
 - copy your std::vector into a struct containing a value and the MPI 
process number
 - call MPI_Allreduce( in_struct, out_struct, vector.size(), 
MPI_DOUBLE_INT, MPI_MAXLOC, mpi_communicator);
 - the maximum value and the number of the first process with that value 
for each entry of your vector is stored in out_struct

My aim is to find the node where the maximum value exists. This is somehow 
> cumbersome in parallel mode since I have to store information about the 
> current CPU core, cell ID and node number. Then somehow use my previously 
> shown distance function in C++ and check the position of the maximum value 
> with regard to the geometry structure I stored.
>
If you save the cells/positions containing the maximum values for each 
process before comparing these across processes, you should be fine with 
this approach.
 
Best,
Daniel 

[1] https://www.open-mpi.org/doc/v2.0/man3/MPI_Reduce.3.php
[2] https://www.dealii.org/8.4.0/doxygen/deal.II/mpi_8h_source.html#l00708
[3] https://www.dealii.org/8.4.0/doxygen/deal.II/mpi_8h_source.html#l00544 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Dear Prof. Bangerth and Jean-Paul,

What I understood so far is, that the Utilities::MPI::max() function 
computes the maximum between all processors for each row of my vector. This 
means if I have a vector of 20 entries, each entry has 8 values distributed 
on 8 cores for instance. As a result, I get 20 maximum values again, but 
this time they are maximal and processor independent. 

Hence, my question now is how am I able to check where the maximum value 
comes from? Is it possible to output the processor number of the maximum 
value for each row?
My aim is to find the node where the maximum value exists. This is somehow 
cumbersome in parallel mode since I have to store information about the 
current CPU core, cell ID and node number. Then somehow use my previously 
shown distance function in C++ and check the position of the maximum value 
with regard to the geometry structure I stored.

Is there a more elegant way to solve my problem than what I suggested in 
deal.II?

Thank you for your kind assistance so far :)
And hopefully you overlook my style of approaching things in this rather 
silly and questionable way.

Kind regards,
Seyed Ali

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Thank you everyone. I understand your suggestions. From what I learned now 
in deal.II is, that each cell is owned by a specific processor which means 
I cannot access the information within a locally owned cell. That's why I 
have to loop over all cells and work with distributed data without using 
locally owned entities. 

Maybe I describe my aim more properly. I like to find the maximum value 
within a MPI vector such as our locally_relevant_solution in step-40. 
Hence, I have to loop over all cells and all nodes to find where the 
geometrical position and value of the displacement lies. How is this 
possible within a parallely distributed environment?

My approach using local_dof_indices is somehow impossible. It only works 
for 1 core according to the below code:

std::vector displacement_norms, loaded_nodes, loaded_elements;
double max_displacement;
int max_loaded_node, max_loaded_element;

typename DoFHandler::active_cell_iterator cell = 
dof_handler.begin_active(), endc = dof_handler.end();
std::vector local_dof_indices(fe.dofs_per_cell);
unsigned int cell_number = 0;

for (; cell != endc; ++cell, ++cell_number)
{
 pcout << "CELL ID: " << cell_number << std::endl; // this works 
surprisingly.

 cell->get_dof_indices(local_dof_indices);// Output or accessing 
this won't work I assume.

 for (unsigned int i = 0, j = 1; i < fe.dofs_per_cell; i += 2, j += 2)
 {
  double displacement_x = locally_relevant_solution(local_dof_indices[i]);
  double displacement_y = locally_relevant_solution(local_dof_indices[j]);

  double displacement_norm = sqrt(displacement_x * displacement_x + 
displacement_y * displacement_y);

  loaded_nodes.push_back(cell->vertex_index(v));
  loaded_elements.push_back(cell_number);
  displacement_norms.push_back(displacement_norm);
 }
}

max_displacement = *max_element(displacement_norms.begin(), 
displacement_norms.end());

double max_index = distance(displacement_norms.begin(), max_element(
displacement_norms.begin(), displacement_norms.end()));

// Node and element number of maximal displacements
max_loaded_node = loaded_nodes[max_index];
max_loaded_element = loaded_elements[max_index];


Hopefully someone has an idea to help me out. Thank you :)


Kind regards,
S. A. Mohseni

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Thank you everyone. I understand your suggestions. From what I learned now 
in deal.II is, that each cell is owned by a specific processor which means 
I cannot access the information within a locally owned cell. That's why I 
have to loop over all cells and work with distributed data without using 
locally owned entities. 

Maybe I describe my aim more properly. I like to find the maximum value 
within a MPI vector such as our locally_relevant_solution in step-40. 
Hence, I have to loop over all cells and all nodes to find where the 
geometrical position and value of the displacement lies. How is this 
possible within a parallely distributed environment?

My approach using local_dof_indices is somehow impossible:

for (; cell != endc; ++cell, ++cell_number)
{
 pcout << "CELL ID: " << cell_number << std::endl; // this works 
surprisingly.

 cell->get_dof_indices(local_dof_indices);// Output or accessing 
this won't work I assume.

 for (unsigned int i = 0, j = 1; i < fe.dofs_per_cell; i += 2, j += 2)
 {
  double displacement_x = locally_relevant_solution(local_dof_indices[i]);
  double displacement_y = locally_relevant_solution(local_dof_indices[j]);

double config_forces_norm = sqrt(config_forces_x * config_forces_x + 
config_forces_y * config_forces_y);
 }
}


-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Re: Access specific element within a distributed triangulation

[Wolfgang Bangerth]

Seyed,


Generally, in FEM you number elements like in the below picture:




No, that is patently wrong. First, you are only considering rectangular 
domains subdivided into rectangular cells. Second, you think that 
because there is a "simple" numbering that that is what codes "should 
do". Both of these assumptions are wrong. How, for example, should a 
code number these cells:


https://raw.githubusercontent.com/dealii/dealii/master/doc/doxygen/images/distributed_mesh_1.png

https://raw.githubusercontent.com/dealii/dealii/master/doc/doxygen/images/hyper_shell_96_cut.png

Starting a sentence with "Generally, in FEM you number elements..." is 
generally wrong.




I am a bit confused since it is a rather easy task which should not be
such a big "deal" in deal.II.


Because it's *not* easy. It's only easy if you consider simple situation 
like yours. But if you want to do complicated things, they generally 
turn out to be complicated.


Best
 W.

--

Wolfgang Bangerth  email: bange...@colostate.edu
   www: http://www.math.colostate.edu/~bangerth/

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Jean-Paul Pelteret]

Hi Seyed,

Nope, like I said, there's no such thing as a cell number in deal.II (at 
least in the sense that a cell knows its own "id" number). The cell->id() 
function 

 
returns a CellId 
 object but 
it does not provide an index/counter in the sense that you think it does. 
One possible reason for this is that (refined) deal.II triangulations are 
stored in levels, and the numbering would have to be rebuilt each time you 
change the mesh. This becomes tricky for distributed triangulations where 
the actual mesh stored on each processor is different. Also, does one want 
to count inactive cells or not? Sometimes yes, sometimes no.

So, if for example I want to store the material ID for each cell in a 
vector I would have to do it like this:
Vector 
 material_id;
material_id.reinit 

(triangulation.n_active_cells 

());

unsigned int c = 0;
typename DoFHandler::active_cell_iterator
cell = dof_handler.begin_active(),
endc = dof_handler.end();
for (; cell!=endc; ++cell, ++c)
 {
   material_id(c) = static_cast(cell->material_id());
 }


Notice that I create and increment the cell counter "c". Notice as well 
that there is no presupposition of the order that deal.II traverses its 
cells.

So dependent on what you're wanting to achieve, you probably wish to 
implement a permutation of the above. Picking out the cell that you are 
wanting to query is probably best achieved using a geometric query or by 
premarking it (e.g. by setting its user_index).

Best,
Jean-Paul

On Friday, February 24, 2017 at 3:45:16 PM UTC+1, Seyed Ali Mohseni wrote:
>
> Thanks for your suggestions.
>
>
> Generally, in FEM you number elements like in the below picture: 
>
>
> 
>
>
> I understand of course, that the cells (elements) in deal.II are parallely 
> distributed, but it should be possible to identify the geometrical position 
> and ID of the elements and their corresponding nodes. 
>
>
> For instance the following works:
>
>
> typename parallel::distributed::Triangulation::active_cell_iterator 
> cell = triangulation.begin_active(), endc = triangulation.end(); 
> for (; cell != endc; ++cell)
> {
>  std::cout << "CELL NUMBER: " << cell->id() << std::endl;
> }
>
>
>
> Of course here the locally owned term is missing, but is it correct to 
> access the cells like this?
>
>
> I am a bit confused since it is a rather easy task which should not be 
> such a big "deal" in deal.II.
>
>
> Best regards,
>
> S. A. Mohseni
>
>
>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

['Seyed Ali Mohseni' via deal.II User Group]

Thanks for your suggestions.


Generally, in FEM you number elements like in the below picture: 




I understand of course, that the cells (elements) in deal.II are parallely 
distributed, but it should be possible to identify the geometrical position 
and ID of the elements and their corresponding nodes. 


For instance the following works:


typename parallel::distributed::Triangulation::active_cell_iterator 
cell = triangulation.begin_active(), endc = triangulation.end(); 
for (; cell != endc; ++cell)
{
 std::cout << "CELL NUMBER: " << cell->id() << std::endl;
}



Of course here the locally owned term is missing, but is it correct to 
access the cells like this?


I am a bit confused since it is a rather easy task which should not be such 
a big "deal" in deal.II.


Best regards,

S. A. Mohseni



-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Re: Access specific element within a distributed triangulation

[Timo Heister]

To add to this, there might also be no reason for cell "4" (whatever
that means) to be available on the current processor. The mesh in a
parallel computation is "distributed" after all.

On Fri, Feb 24, 2017 at 8:01 AM, Jean-Paul Pelteret
 wrote:
> Dear Seyed,
>
> There are no cell "numbers" in deal.II, only CellIDs to give some unique
> identifier to each cell. So its unclear to me as to exactly what you're
> trying to achieve here.
>
> Are you wanting the fourth cell iterator for the entire triangulation, or
> the fourth locally owned cell? Either way, you need to increment your cell
> iterator by however many times you want, taking note that dependent on how
> you do this you may not end up with a locally owned cell. You can increment
> iterators by a given number of times using std::advance, and you may wish to
> consider using FilteredIterators with the LocallyOwned predicate to ensure
> that you always end up with a cell thats valid on the given process.
>
> I hope that this helps,
> Jean-Paul
>
> On Friday, February 24, 2017 at 11:12:59 AM UTC+1, Seyed Ali Mohseni wrote:
>>
>> Hi,
>>
>> Is it possible to access a specific element within a parallel distributed
>> triangulation by means of the following procedure for instance:
>>
>> typename DoFHandler::active_cell_iterator cell =
>> dof_handler.begin_active(), endc = dof_handler.end();
>>
>> for (; cell != endc; ++cell)
>> {
>>  if (cell->is_locally_owned())
>>  {
>>  std::cout << "CELL NUMBER 4 VERTEX 1: " << cell(4)->vertex(0) <<
>> std::endl;
>>  }
>> }
>>
>> The above won't compile.
>>
>> How am I able to access for example element number 4 in order to obtain
>> the coordinates of point 1 within the element.
>>
>> Kind regards,
>> S. A. Mohseni
>>
> --
> The deal.II project is located at 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.dealii.org_=DwIBaQ=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4=4k7iKXbjGC8LfYxVJJXiaYVu6FRWmEjX38S7JmlS9Vw=fmFEEI3Kq19LfHmpN-c61No1dYXeRRPQnotNi7PL4Po=kB0zzWe3cVJaujXkNnKZL0WfssR0O3dkB99s0iqyBFs=
>  
> For mailing list/forum options, see
> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_forum_dealii-3Fhl-3Den=DwIBaQ=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4=4k7iKXbjGC8LfYxVJJXiaYVu6FRWmEjX38S7JmlS9Vw=fmFEEI3Kq19LfHmpN-c61No1dYXeRRPQnotNi7PL4Po=TkHRRnwbkhay8yAMwTsfS0HHpyj3IqU-AHr0x54Dw2k=
>  
> ---
> You received this message because you are subscribed to the Google Groups
> "deal.II User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dealii+unsubscr...@googlegroups.com.
> For more options, visit 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout=DwIBaQ=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4=4k7iKXbjGC8LfYxVJJXiaYVu6FRWmEjX38S7JmlS9Vw=fmFEEI3Kq19LfHmpN-c61No1dYXeRRPQnotNi7PL4Po=3z0U2_GKFXAOvrHFb6Q6f62Nq_MTxwhLVG7vR48J2yg=
>  .



-- 
Timo Heister
http://www.math.clemson.edu/~heister/

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: Access specific element within a distributed triangulation

[Jean-Paul Pelteret]

Dear Seyed,

There are no cell "numbers" in deal.II, only CellIDs 
 to give 
some unique identifier to each cell. So its unclear to me as to exactly 
what you're trying to achieve here. 

Are you wanting the fourth cell iterator for the entire triangulation, or 
the fourth locally owned cell? Either way, you need to increment your cell 
iterator by however many times you want, taking note that dependent on how 
you do this you may not end up with a locally owned cell. You can increment 
iterators by a given number of times using std::advance 
, and you may wish to 
consider using FilteredIterators 
 with 
the LocallyOwned predicate 

 
to ensure that you always end up with a cell thats valid on the given 
process.

I hope that this helps,
Jean-Paul

On Friday, February 24, 2017 at 11:12:59 AM UTC+1, Seyed Ali Mohseni wrote:
>
> Hi,
>
> Is it possible to access a specific element within a parallel distributed 
> triangulation by means of the following procedure for instance:
>
> typename DoFHandler::active_cell_iterator cell = 
> dof_handler.begin_active(), endc = dof_handler.end();
>
> for (; cell != endc; ++cell)
> {
>  if (cell->is_locally_owned())
>  {
>  std::cout << "CELL NUMBER 4 VERTEX 1: " << cell(4)->vertex(0) << 
> std::endl;
>  }
> }
>
> The above won't compile.
>
> How am I able to access for example element number 4 in order to obtain 
> the coordinates of point 1 within the element.
>
> Kind regards,
> S. A. Mohseni
>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.