Re: [deal.II] deal-II tests fail due to ginkgo

[Wolfgang Bangerth]

On 2/14/20 12:49 AM, Thodoros Katsaounis wrote:


I am on a linux workstation with ubuntu 18.04. I have successfully installed 
deal-II (9.1.1)
including ginkgo. When I tried to run the deal-II tests with 'make test' ALL 
of them failed.

The reason of failing in ALL tests is due to the following error

../../lib/libdeal_II.so.9.1.1: error: undefined reference to 
'gko::OmpExecutor::raw_copy_to(gko::HipExecutor const*, unsigned long, void 
const*, void*) const'
../../lib/libdeal_II.so.9.1.1: error: undefined reference to 'typeinfo for 
gko::HipExecutor'

collect2: error: ld returned 1 exit status

I build ginkgo the way it's described in 
https://www.dealii.org/current/external-libs/ginkgo.html

the installation was successful and all the tests of ginkgo passed.


any suggestions on how to resolve this issue?


Good question. I think you probably want to ask the gingko people on their 
mailing list why that function is not available. It's not a template function, 
so it's not a question of instantiation -- it seems to simply not be available 
in the gingko library.


Best
 W.

--

Wolfgang Bangerth  email: bange...@colostate.edu
   www: http://www.math.colostate.edu/~bangerth/

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/f847150d-e1ae-adbf-581c-3a1e4b5ea8f9%40colostate.edu.

Re: [deal.II] Shared Memory Parallel Linear Solver

[Wolfgang Bangerth]

While going through the documentation, I came across options for DMP linear 
solvers but couldn't find any with SMP.


Could somenone please guide me to any such possibilities within the library or 
through interface to some external library.


You could also go through PETSc. You can build the PETSc matrix on a single 
node and then see whether PETSc supports any of their sparse direct solvers 
using multiple threads.


Best
 W.

--

Wolfgang Bangerth  email: bange...@colostate.edu
   www: http://www.math.colostate.edu/~bangerth/

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/718f6cf6-e91f-0bf5-1b05-3a0702078f72%40colostate.edu.

[deal.II] Re: Shared Memory Parallel Linear Solver

[Bruno Turcksin]

Paras,

You could try to use SuperLU_MT (see 
https://portal.nersc.gov/project/sparse/superlu/) but we don't have wrapper 
for it. The Krylov solver in deal.II are multithreaded but the 
preconditioners are not. What you can try is to use deal.II solvers and 
STRUMPACK (https://github.com/pghysels/STRUMPACK) for the preconditioner 
but again we don't have wrapper for it. 

Best,

Bruno

On Friday, February 14, 2020 at 7:22:15 AM UTC-5, Paras Kumar wrote:
>
> Dear deal.ii Community,
>
> I am working  on finite deformation hyperelasticity problem which is 
> essentially a nonlinear-vector-valued problem with displacement as the 
> unknown at each support point(dim=2,3).  *With regards to parallelism, we 
> currently restrict ourselves to shared memory parallel (SMP) only.* The 
> FE assembly process has been paralleized using the workstream function.
>
> This question pertains to (possibly faster) linear solvers. Currently. For 
> the current 2D problem, we use the SparseDirectUMFPACK solver. As can been 
> seen in the time log below (ths example was computed on a 36 core machine 
> with 64 threads), the linear solver consumes the most time. I also tried 
> using the CG solver, but it was much much slower, probably due to the large 
> condition number stemming out of  highly refined mesh in regions of 
> interest.
>
> Triangulation:
>  Number of active cells: 2059646
>  Number of Degrees of Freedom: 4119454
>
> +-+++
> | Total wallclock time elapsed since start|  7.81e+03s ||
> | |||
> | Section | no. calls |  wall time | % of total |
> +-+---+++
> | Assemble Linear System  |50 |   875s |11% |
> | Linear Solver   |40 |  5.55e+03s |71% |
> | Make Constraints|50 |   255s |   3.3% |
> | PBC setup   | 1 |  1.02s | 0% |
> | Set Boundary IDs| 1 | 0.109s | 0% |
> | Stress Output Computation   |11 |   464s |   5.9% |
> | Stress Output Overall   |12 | 1e+03s |13% |
> | Stress Output Writing   |11 |   379s |   4.9% |
> | Stress output initialize| 1 |   158s | 2% |
> | System Setup| 1 |   260s |   3.3% |
> +-+---+++
>
> While going through the documentation, I came across options for DMP 
> linear solvers but couldn't find any with SMP. 
>
> Could somenone please guide me to any such possibilities within the 
> library or through interface to some external library.
>
> Best regards,
> Paras
>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/2a2ce382-894d-42ff-a9ea-d1c15612a309%40googlegroups.com.

[deal.II] solving stabilized Stokes

[Richard Schussnig]

Hi everyone!
I am currently implementing some stationary Stokes solvers based on step-55.
Therein, Taylor-Hood elements are being used. One can check the optimal 
order of 
convergence easily comparing to the Kovasznay or Poisuille flow solution, 
the first one being already implemented in this step.
I went ahead and added PSPG and Bochev-Dohrmann stabilizations to check the 
errors using those.

For the latter, i get the expected suboptimal (!) convergence rates (vel:2 
and p~1.5-2).
Using PSPG, one adds 
tau * residual momentum dot gradient(q)
(q being the pressure test function)
per element like:

local_matrix (i,j) += fe_values.JxW(q) * tau_sups * phi_grads_p [j] * 
phi_grads_p [i];

if (ADD_THE_LAPLACIAN)
{
local_matrix (i,j) += fe_values.JxW(q) * tau * (
  - viscosity * phi_laplacians_v [j]
  ) * phi_grads_p [i];
}

Using the laplacian of the velocity u defined as below:

if (get_laplace_residual)
{
phi_hessians_v[k] = fe_hessians[velocities].hessian(k,q);
for (int d = 0; d < dim; ++d)
{ phi_laplacians_v[k][d] = trace(phi_hessians_v[k][d]);
}

// Check laplacian, both give zero for rectangular grid and identical 
values for distorted grids.
unsigned int comp_k = fe.system_to_component_index(k).first;

Tensor<1,dim> vector_laplace_v;
vector_laplace_v = 0;
if (comp_k nonzero_hessian_k = 
fe_hessians.shape_hessian_component(k,q,comp_k);
vector_laplace_v [comp_k] += trace(nonzero_hessian_k);
}
Tensor<1,dim> diff;
diff = phi_laplacians_v[k] - vector_laplace_v;

if (diff.norm() > 1e-6 || vector_laplace_v.norm() > 1e-16)
std::cout << "|divgrad_v| = " << std::setw(15) << vector_laplace_v.norm() 
<< " , diff = " << diff.norm() << std::endl;

}

Which also includes a second option to compute the wanted laplacian.

Using a perfectly rectangular grid, the laplacian is actually zero for all 
elements, thus everything reduces to just adding a pressure stiffness in 
the pressure-pressure block.
Nonetheless, i observe convergence rates of vel:2 and p:1.5-1.8 which is 
suboptimal and not(!) expected.
I implemented the same methods in a matlab inhouse code & observed perfect 
convergence rates of 2&2 for v&p using a direct solver.

The stabilization parameter is in both cases (matlab or deal.ii) chosen as 
tau = 0.1*h^2/viscosity 
with 
double h = std::pow(cell->measure(), 1.0/ static_cast(dim)) 
or h = cell->diameter() giving similar results.

Could the observed behaviour be caused by applying an iterative solver*? 
(using ReductionControl and a reduction factor of 1e-13 which is really low)
* Block-triangular Schur-complement-based approach, similar as in step-55 
suggested in the further possibilities for extension, basically using 
S~-(1/viscosity)*Mass_pressure giving 20-40 iterations up to 3mio dofs 
tested.

I have been checking the code for a week now, and i really doubt, that the 
integration of just tau*grad(p)*grad(p) is wrong (again, the laplace drops 
out for a rectangular grid).
Just to check, i used a manufactured solution from Poisuille flow and added 
the laplacian from PSPG to the rhs, giving me the exact (linear) solution 
in the pressure and quadratic convergence in the velocity, thus I assume, 
that the grad(p)grad(q) term added is correctly implemeted. (exact solution 
meaning, that i get an L2 error of err<1e-9 for any number of elements used)

Anyone has ever experienced this or has anyone some tipps for further 
debugging?
Thanks for taking the time to read the post, any help or comment would be 
greatly appreciated!

Greetings from Austria,
Richard

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/2ded867b-a2e7-4041-b90c-2dd774f1f5dd%40googlegroups.com.

[deal.II] Shared Memory Parallel Linear Solver

[Paras Kumar]

Dear deal.ii Community,

I am working  on finite deformation hyperelasticity problem which is 
essentially a nonlinear-vector-valued problem with displacement as the 
unknown at each support point(dim=2,3).  *With regards to parallelism, we 
currently restrict ourselves to shared memory parallel (SMP) only.* The FE 
assembly process has been paralleized using the workstream function.

This question pertains to (possibly faster) linear solvers. Currently. For 
the current 2D problem, we use the SparseDirectUMFPACK solver. As can been 
seen in the time log below (ths example was computed on a 36 core machine 
with 64 threads), the linear solver consumes the most time. I also tried 
using the CG solver, but it was much much slower, probably due to the large 
condition number stemming out of  highly refined mesh in regions of 
interest.

Triangulation:
 Number of active cells: 2059646
 Number of Degrees of Freedom: 4119454

+-+++
| Total wallclock time elapsed since start|  7.81e+03s ||
| |||
| Section | no. calls |  wall time | % of total |
+-+---+++
| Assemble Linear System  |50 |   875s |11% |
| Linear Solver   |40 |  5.55e+03s |71% |
| Make Constraints|50 |   255s |   3.3% |
| PBC setup   | 1 |  1.02s | 0% |
| Set Boundary IDs| 1 | 0.109s | 0% |
| Stress Output Computation   |11 |   464s |   5.9% |
| Stress Output Overall   |12 | 1e+03s |13% |
| Stress Output Writing   |11 |   379s |   4.9% |
| Stress output initialize| 1 |   158s | 2% |
| System Setup| 1 |   260s |   3.3% |
+-+---+++

While going through the documentation, I came across options for DMP linear 
solvers but couldn't find any with SMP. 

Could somenone please guide me to any such possibilities within the library 
or through interface to some external library.

Best regards,
Paras

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dealii/b5508fa4-6a2c-4368-b727-52f5dd47f393%40googlegroups.com.

[deal.II] Re: Installation on cray XC50 | linking to petsc, lapack and blas libraries with different names

[vachan potluri]

Here is a summary of the installation process on Cray XC50.

I have configured deal.II with MPI, LAPACK, SCALAPACK, PETSc and p4est. Our 
system didn't have p4est so I started with installing it. All cray 
libraries are in /opt/cray/pe/lib64/ in out system.

*Installing p4est*
1. Download source files and setup script from here 
.
2. By default, the setup script searches for mpicxx compilers. Instead, 
explicitly specifiy cray compilers. The configure command will look as 
follows.
"$SRCDIR/configure" CXX=/opt/cray/pe/craype/2.5.13/bin/CC \
CC=/opt/cray/pe/craype/2.5.13/bin/cc \
F77=/opt/cray/pe/craype/2.5.13/bin/ftn \
FC=/opt/cray/pe/craype/2.5.13/bin/ftn \
--enable-mpi --enable-shared \
--disable-vtk-binary --without-blas \
--prefix="$INSTALL_DEBUG" CFLAGS="$CFLAGS_DEBUG" \
CPPFLAGS="-DSC_LOG_PRIORITY=SC_LP_ESSENTIAL" \
"$@" > config.output || bdie "Error in configure"
Make this change for both FAST and DEBUG versions.
3. By default cray assumes static linking. Change them:
export XTPE_LINK_TYPE=dynamic
export CRAYPE_LINK_TYPE=dynamic
This will be required in subsequent steps also.
4. The makefile generated uses flags corresponding to GNU compilers. Switch 
module PrgEnv-cray with PrgEnv-gnu.

*Configuring with LAPACK and SCALAPACK*
1. For LAPACK, deal.II's find module calls cmake's corresponding find 
module. For this to work on cray systems, cmake version >=3.16 is required. 
So I installed a new version in my home directory and used this cmake 
version. See this 

 
and this .
2. For Cray environments, lapack libraries are linked directly to cray 
compiler without requiring any other flags. So the _lapack_libraries 
variable in deal.II's FindLAPACK.cmake will be empty. This is okay. So set 
this as OPTIONAL in the end of this file.
3. For SCALAPACK, the library name in FindLAPACK.cmake should be changed to 
sci_gnu_61_mpi_mp (or whatever is the name of libsci library on your 
system) since on cray, SCALAPACK is a part of this library.

*Configuring with MPI and PETSc*
1. For MPI, simply specify the compilers explicitly.
2. For PETSc, the library name must be changed to craypetsc_gnu_real-64 
(depending on your system).
3. The additional libraries PETSc interfaces to are read from linker line 
of $PETSC_DIR/lib/petsc/conf/petscvariables. Make a copy of this file and 
modify the linker line so that the library names are correct (if they are 
not already, as was the case with me). Change the hint to petscvariables 
file in FindPETSC.cmake.
4. Also, add the correct hint to these library paths in the following 
portion of the aforementioned file.
DEAL_II_FIND_LIBRARY(PETSC_LIBRARY_${_token}
NAMES ${_token}
#HINTS ${_hints}
HINTS ${_hints} ${CMAKE_PREFIX_PATH}
)
In my case, I set CMAKE_PREFIX_PATH in configure script to 
/opt/cray/pe/lib64.
5. If your system has PETSc libraries with ".so.mpi" extensions, you 
must enable find those in dealii-9.1.1/CMakeLists.txt (the top most one)
SET(CMAKE_FIND_LIBRARY_SUFFIXES ${CMAKE_FIND_LIBRARY_SUFFIXES}
".so.0" ".so.5" ".so.mpi31.2" ".so.mpi31.4" ".so.mpi31.5" 
".so.mpi31.6" ".so.mpi31.12"
)
6. If you are using 64-bit versions of PETSc libraries, you must enable 
this for deal.II too (see below).



You must unload the atp module before configuring (see here 
). For 
cross-compilation (see here 
),
 
you can just add -DCMAKE_SYSTEM_NAME=CrayLinuxEnvironment without requiring 
a Toolchain file in newer cmake versions. The configure script is

cmake_new=~/bin/cmake-3.16.4/usr/local/bin/cmake # from bashrc, shell 
scripts can't use aliases
$cmake_new -DCMAKE_INSTALL_PREFIX=~/bin/dealii-9.1.1/ \
-DWITH_64BIT_INDICES=ON \
-DCMAKE_PREFIX_PATH=/opt/cray/pe/lib64 \
-DWITH_MPI=ON \
-DMPI_DIR=/opt/cray/pe/mpt/default/gni/mpich-gnu/5.1/ \

-DMPI_CXX_INCLUDE_PATH=/opt/cray/pe/mpt/default/gni/mpich-gnu/5.1/include/ \
-DCMAKE_CXX_COMPILER=/opt/cray/pe/craype/2.5.13/bin/CC \
-DCMAKE_C_COMPILER=/opt/cray/pe/craype/2.5.13/bin/cc \
-DCMAKE_Fortran_COMPILER=/opt/cray/pe/craype/2.5.13/bin/ftn 
\
-DWITH_BLAS=ON \
-DWITH_LAPACK=ON \
-DWITH_SCALAPACK=ON \
-DWITH_PETSC=ON \
-DWITH_P4EST=ON -DP4EST_DIR=~/bin/p4est-2.2/ \
-DCMAKE_SYSTEM_NAME=CrayLinuxEnvironment \
~/source/dealii-9.1.1

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you ar