Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Jed Brown]

Edoardo alinovi  writes:

> Hello Barry,
>
> Welcome to the party! Thank you guys for your precious suggestions, they
> are really helpful!
>
> It's been a while since I am messing around and I have tested many
> combinations. Schur + selfp is the best preconditioner, it converges within
> 5 iters using gmres for inner solvers but it is not very fast and sometimes
> multiplicative it's a better option as the inner iterations looks lighter.
> If I tune the relative tolerance I get a huge speed up, but I am a bit less
> confident about the results. The funny thing is that the default relative
> tolerance if commercial CFD solver is huge, very often 0.1 -.-

This depends on time step size and tolerances for quantities of interest.

> If I may borrow your brain guys for a while, I would like to ask your
> opinion about the multigrid they use in this paper:
> https://www.aub.edu.lb/msfea/research/Documents/CFD-P18.pdf.
> At some point they say: " The algorithm used in this work is a combination
> of the ILU(0) [28] algorithm with an additive corrective multigrid method
> [29] ", 29: https://www.tandfonline.com/doi/abs/10.1080/10407788608913491

It looks like this is a coupled solver with collocated dofs and thus I don't 
know how they stabilize in the incompressible limit (an inf-sup requirement is 
needed for stability). (This is based on a cursory read.)

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Hello Barry,

Welcome to the party! Thank you guys for your precious suggestions, they
are really helpful!

It's been a while since I am messing around and I have tested many
combinations. Schur + selfp is the best preconditioner, it converges within
5 iters using gmres for inner solvers but it is not very fast and sometimes
multiplicative it's a better option as the inner iterations looks lighter.
If I tune the relative tolerance I get a huge speed up, but I am a bit less
confident about the results. The funny thing is that the default relative
tolerance if commercial CFD solver is huge, very often 0.1 -.-

If I may borrow your brain guys for a while, I would like to ask your
opinion about the multigrid they use in this paper:
https://www.aub.edu.lb/msfea/research/Documents/CFD-P18.pdf.
At some point they say: " The algorithm used in this work is a combination
of the ILU(0) [28] algorithm with an additive corrective multigrid method
[29] ", 29: https://www.tandfonline.com/doi/abs/10.1080/10407788608913491

Is that something similar to fieldsplit additive?

Many thanks

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Barry Smith]

  To expand on what Matt said slightly. 

When you have a preconditioner based on (possibly nested) sub solves one 
generally "tunes" the solves to minimize time to solution. We recommend doing 
this by first using very accurate subsolves (when possible using direct solves 
inside); this tells us the "best reasonably expected" convergence rate of the 
outermost solve. One then "backs off" on the accuracy of the inner solves 
(making them less accurate), monitoring how this "backing off" affects the 
convergence rate of the outermost solve (one will often allow the convergence 
of the outer-most solve to get "a bit worse" in the hope that the since the 
cost of the inner solves will be much cheaper (with the less accuracy) the cost 
of the extra outer iterations will not make the entire solve time longer. 
Deciding how much to "back-off" on each inner solve is mostly determined by 
monitoring the time for convergence of the entire problem (but monitoring 
number of iterations also provides important information, for example if the 
number of outer iterations suddenly shots up you know you are probably 
undersolving  a subproblem.

While doing this process, it is important to work with "production" size 
problems since for PDEs the convergence rates are often very affected by the 
problem size. One should start with smaller problems to get a feeling for the 
tolerances needed but usually final tuning needs to be done for large problems. 
And, sadly, if one changes some parameters in the system a very different 
tuning may be optimal.

  Barry




> On May 5, 2023, at 5:13 AM, Edoardo alinovi  wrote:
> 
> Hi Matt, 
> 
> I have some more questions on the fieldsplit saga :)
> 
> I am running a 1M cell ahmed body case using  the following options:
> 
> "solver": "fgmres",
> "preconditioner": "fieldsplit",
>  "absTol": 1e-6,
>  "relTol": 0.0,
> 
> "options":{
>"pc_fieldsplit_type": "multiplicative",
>"fieldsplit_u_pc_type": "ml",
>"fieldsplit_p_pc_type": "ml",
>"fieldsplit_u_ksp_type": "preonly",
>"fieldsplit_p_ksp_type": "preonly",
>"fieldsplit_u_ksp_rtol": 1e-2,
>"fieldsplit_p_ksp_rtol": 1e-2,
> }
> 
> 
> I have run the case using -ksp_monitor_true_residual and this is the results 
> I am getting:
>  
>   Residual norms for UPeqn_ solve.
>   0 KSP unpreconditioned resid norm 5.003190920461e+00 true resid norm 
> 5.003190920461e+00 ||r(i)||/||b|| 4.993739374163e-03
>   1 KSP unpreconditioned resid norm 4.959389845148e+00 true resid norm 
> 4.959389845148e+00 ||r(i)||/||b|| 4.950021043622e-03
>   2 KSP unpreconditioned resid norm 4.793097370730e+00 true resid norm 
> 4.793097370730e+00 ||r(i)||/||b|| 4.784042712927e-03
>   3 KSP unpreconditioned resid norm 4.187770916162e+00 true resid norm 
> 4.187770916162e+00 ||r(i)||/||b|| 4.179859782782e-03
>   4 KSP unpreconditioned resid norm 3.099045576565e+00 true resid norm 
> 3.099045576565e+00 ||r(i)||/||b|| 3.093191158212e-03
>   5 KSP unpreconditioned resid norm 2.072551338956e+00 true resid norm 
> 2.072551338956e+00 ||r(i)||/||b|| 2.068636074628e-03
>   6 KSP unpreconditioned resid norm 1.414678932482e+00 true resid norm 
> 1.414678932482e+00 ||r(i)||/||b|| 1.412006457328e-03
>   7 KSP unpreconditioned resid norm 1.006854855789e+00 true resid norm 
> 1.006854855789e+00 ||r(i)||/||b|| 1.004952802592e-03
>   8 KSP unpreconditioned resid norm 7.332800358083e-01 true resid norm 
> 7.332800358084e-01 ||r(i)||/||b|| 7.318947938062e-04
>   9 KSP unpreconditioned resid norm 5.406076142092e-01 true resid norm 
> 5.406076142093e-01 ||r(i)||/||b|| 5.395863503846e-04
>  10 KSP unpreconditioned resid norm 4.037336888099e-01 true resid norm 
> 4.037336888100e-01 ||r(i)||/||b|| 4.029709940193e-04
>  11 KSP unpreconditioned resid norm 3.041388930530e-01 true resid norm 
> 3.041388930530e-01 ||r(i)||/||b|| 3.035643431559e-04
>  12 KSP unpreconditioned resid norm 2.299815364065e-01 true resid norm 
> 2.299815364066e-01 ||r(i)||/||b|| 2.295470774436e-04
>  13 KSP unpreconditioned resid norm 1.739866268817e-01 true resid norm 
> 1.739866268817e-01 ||r(i)||/||b|| 1.736579481074e-04
>  14 KSP unpreconditioned resid norm 1.317133652074e-01 true resid norm 
> 1.317133652074e-01 ||r(i)||/||b|| 1.314645450067e-04
>  15 KSP unpreconditioned resid norm 9.966247212017e-02 true resid norm 
> 9.966247212019e-02 ||r(i)||/||b|| 9.947419937897e-05
>  16 KSP unpreconditioned resid norm 7.531138284402e-02 true resid norm 
> 7.531138284404e-02 ||r(i)||/||b|| 7.516911183479e-05
>  17 KSP unpreconditioned resid norm 5.646770286889e-02 true resid norm 
> 5.646770286889e-02 ||r(i)||/||b|| 5.636102952452e-05
>  18 KSP unpreconditioned resid norm 4.22511838e-02 true resid norm 
> 4.22511838e-02 ||r(i)||/||b|| 4.217132765661e-05
>  19 KSP unpreconditioned resid norm 3.160393382046e-02 true resid norm 
> 3.160393382046e-02 ||r(i)||/||b|| 3.154423071329e-05
>  20 KSP unpreconditioned resid norm 2.366499890335e-02 true 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Fri, May 5, 2023 at 5:13 AM Edoardo alinovi 
wrote:

> Hi Matt,
>
> I have some more questions on the fieldsplit saga :)
>
> I am running a 1M cell ahmed body case using  the following options:
>
> "solver": "fgmres",
> "preconditioner": "fieldsplit",
>  "absTol": 1e-6,
>  "relTol": 0.0,
>
> "options":{
>"pc_fieldsplit_type": "multiplicative",
>"fieldsplit_u_pc_type": "ml",
>"fieldsplit_p_pc_type": "ml",
>"fieldsplit_u_ksp_type": "preonly",
>"fieldsplit_p_ksp_type": "preonly",
>"fieldsplit_u_ksp_rtol": 1e-2,
>"fieldsplit_p_ksp_rtol": 1e-2,
> }
>
>
> I have run the case using -ksp_monitor_true_residual and this is the
> results I am getting:
>
>
>   Residual norms for UPeqn_ solve.
>   0 KSP unpreconditioned resid norm 5.003190920461e+00 true resid norm
> 5.003190920461e+00 ||r(i)||/||b|| 4.993739374163e-03
>   1 KSP unpreconditioned resid norm 4.959389845148e+00 true resid norm
> 4.959389845148e+00 ||r(i)||/||b|| 4.950021043622e-03
>   2 KSP unpreconditioned resid norm 4.793097370730e+00 true resid norm
> 4.793097370730e+00 ||r(i)||/||b|| 4.784042712927e-03
>   3 KSP unpreconditioned resid norm 4.187770916162e+00 true resid norm
> 4.187770916162e+00 ||r(i)||/||b|| 4.179859782782e-03
>   4 KSP unpreconditioned resid norm 3.099045576565e+00 true resid norm
> 3.099045576565e+00 ||r(i)||/||b|| 3.093191158212e-03
>   5 KSP unpreconditioned resid norm 2.072551338956e+00 true resid norm
> 2.072551338956e+00 ||r(i)||/||b|| 2.068636074628e-03
>   6 KSP unpreconditioned resid norm 1.414678932482e+00 true resid norm
> 1.414678932482e+00 ||r(i)||/||b|| 1.412006457328e-03
>   7 KSP unpreconditioned resid norm 1.006854855789e+00 true resid norm
> 1.006854855789e+00 ||r(i)||/||b|| 1.004952802592e-03
>   8 KSP unpreconditioned resid norm 7.332800358083e-01 true resid norm
> 7.332800358084e-01 ||r(i)||/||b|| 7.318947938062e-04
>   9 KSP unpreconditioned resid norm 5.406076142092e-01 true resid norm
> 5.406076142093e-01 ||r(i)||/||b|| 5.395863503846e-04
>  10 KSP unpreconditioned resid norm 4.037336888099e-01 true resid norm
> 4.037336888100e-01 ||r(i)||/||b|| 4.029709940193e-04
>  11 KSP unpreconditioned resid norm 3.041388930530e-01 true resid norm
> 3.041388930530e-01 ||r(i)||/||b|| 3.035643431559e-04
>  12 KSP unpreconditioned resid norm 2.299815364065e-01 true resid norm
> 2.299815364066e-01 ||r(i)||/||b|| 2.295470774436e-04
>  13 KSP unpreconditioned resid norm 1.739866268817e-01 true resid norm
> 1.739866268817e-01 ||r(i)||/||b|| 1.736579481074e-04
>  14 KSP unpreconditioned resid norm 1.317133652074e-01 true resid norm
> 1.317133652074e-01 ||r(i)||/||b|| 1.314645450067e-04
>  15 KSP unpreconditioned resid norm 9.966247212017e-02 true resid norm
> 9.966247212019e-02 ||r(i)||/||b|| 9.947419937897e-05
>  16 KSP unpreconditioned resid norm 7.531138284402e-02 true resid norm
> 7.531138284404e-02 ||r(i)||/||b|| 7.516911183479e-05
>  17 KSP unpreconditioned resid norm 5.646770286889e-02 true resid norm
> 5.646770286889e-02 ||r(i)||/||b|| 5.636102952452e-05
>  18 KSP unpreconditioned resid norm 4.22511838e-02 true resid norm
> 4.22511838e-02 ||r(i)||/||b|| 4.217132765661e-05
>  19 KSP unpreconditioned resid norm 3.160393382046e-02 true resid norm
> 3.160393382046e-02 ||r(i)||/||b|| 3.154423071329e-05
>  20 KSP unpreconditioned resid norm 2.366499890335e-02 true resid norm
> 2.366499890334e-02 ||r(i)||/||b|| 2.362029326721e-05
>  21 KSP unpreconditioned resid norm 1.759504138840e-02 true resid norm
> 1.759504138839e-02 ||r(i)||/||b|| 1.756180253124e-05
>  22 KSP unpreconditioned resid norm 1.309326628086e-02 true resid norm
> 1.309326628085e-02 ||r(i)||/||b|| 1.306853174355e-05
>  23 KSP unpreconditioned resid norm 9.710513089816e-03 true resid norm
> 9.710513089808e-03 ||r(i)||/||b|| 9.692168923954e-06
>  24 KSP unpreconditioned resid norm 7.171039760236e-03 true resid norm
> 7.171039760227e-03 ||r(i)||/||b|| 7.157492922743e-06
>  25 KSP unpreconditioned resid norm 5.277221846963e-03 true resid norm
> 5.277221846949e-03 ||r(i)||/||b|| 5.267252627824e-06
>  26 KSP unpreconditioned resid norm 3.906960734588e-03 true resid norm
> 3.906960734575e-03 ||r(i)||/||b|| 3.899580080737e-06
>  27 KSP unpreconditioned resid norm 2.896843259283e-03 true resid norm
> 2.896843259273e-03 ||r(i)||/||b|| 2.891370822059e-06
>  28 KSP unpreconditioned resid norm 2.140269358580e-03 true resid norm
> 2.140269358568e-03 ||r(i)||/||b|| 2.136226167881e-06
>  29 KSP unpreconditioned resid norm 1.585513255966e-03 true resid norm
> 1.585513255956e-03 ||r(i)||/||b|| 1.582518057055e-06
>  30 KSP unpreconditioned resid norm 1.173839272299e-03 true resid norm
> 1.173839272299e-03 ||r(i)||/||b|| 1.171621768229e-06
>  31 KSP unpreconditioned resid norm 8.777233482545e-04 true resid norm
> 8.777233482544e-04 ||r(i)||/||b|| 8.760652378615e-07
>  32 KSP unpreconditioned resid norm 6.546689191353e-04 true resid norm
> 6.546689191370e-04 ||r(i)||/||b|| 6.534321816833e-07
>  33 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Hi Matt,

I have some more questions on the fieldsplit saga :)

I am running a 1M cell ahmed body case using  the following options:

"solver": "fgmres",
"preconditioner": "fieldsplit",
 "absTol": 1e-6,
 "relTol": 0.0,

"options":{
   "pc_fieldsplit_type": "multiplicative",
   "fieldsplit_u_pc_type": "ml",
   "fieldsplit_p_pc_type": "ml",
   "fieldsplit_u_ksp_type": "preonly",
   "fieldsplit_p_ksp_type": "preonly",
   "fieldsplit_u_ksp_rtol": 1e-2,
   "fieldsplit_p_ksp_rtol": 1e-2,
}


I have run the case using -ksp_monitor_true_residual and this is the
results I am getting:


  Residual norms for UPeqn_ solve.
  0 KSP unpreconditioned resid norm 5.003190920461e+00 true resid norm
5.003190920461e+00 ||r(i)||/||b|| 4.993739374163e-03
  1 KSP unpreconditioned resid norm 4.959389845148e+00 true resid norm
4.959389845148e+00 ||r(i)||/||b|| 4.950021043622e-03
  2 KSP unpreconditioned resid norm 4.793097370730e+00 true resid norm
4.793097370730e+00 ||r(i)||/||b|| 4.784042712927e-03
  3 KSP unpreconditioned resid norm 4.187770916162e+00 true resid norm
4.187770916162e+00 ||r(i)||/||b|| 4.179859782782e-03
  4 KSP unpreconditioned resid norm 3.099045576565e+00 true resid norm
3.099045576565e+00 ||r(i)||/||b|| 3.093191158212e-03
  5 KSP unpreconditioned resid norm 2.072551338956e+00 true resid norm
2.072551338956e+00 ||r(i)||/||b|| 2.068636074628e-03
  6 KSP unpreconditioned resid norm 1.414678932482e+00 true resid norm
1.414678932482e+00 ||r(i)||/||b|| 1.412006457328e-03
  7 KSP unpreconditioned resid norm 1.006854855789e+00 true resid norm
1.006854855789e+00 ||r(i)||/||b|| 1.004952802592e-03
  8 KSP unpreconditioned resid norm 7.332800358083e-01 true resid norm
7.332800358084e-01 ||r(i)||/||b|| 7.318947938062e-04
  9 KSP unpreconditioned resid norm 5.406076142092e-01 true resid norm
5.406076142093e-01 ||r(i)||/||b|| 5.395863503846e-04
 10 KSP unpreconditioned resid norm 4.037336888099e-01 true resid norm
4.037336888100e-01 ||r(i)||/||b|| 4.029709940193e-04
 11 KSP unpreconditioned resid norm 3.041388930530e-01 true resid norm
3.041388930530e-01 ||r(i)||/||b|| 3.035643431559e-04
 12 KSP unpreconditioned resid norm 2.299815364065e-01 true resid norm
2.299815364066e-01 ||r(i)||/||b|| 2.295470774436e-04
 13 KSP unpreconditioned resid norm 1.739866268817e-01 true resid norm
1.739866268817e-01 ||r(i)||/||b|| 1.736579481074e-04
 14 KSP unpreconditioned resid norm 1.317133652074e-01 true resid norm
1.317133652074e-01 ||r(i)||/||b|| 1.314645450067e-04
 15 KSP unpreconditioned resid norm 9.966247212017e-02 true resid norm
9.966247212019e-02 ||r(i)||/||b|| 9.947419937897e-05
 16 KSP unpreconditioned resid norm 7.531138284402e-02 true resid norm
7.531138284404e-02 ||r(i)||/||b|| 7.516911183479e-05
 17 KSP unpreconditioned resid norm 5.646770286889e-02 true resid norm
5.646770286889e-02 ||r(i)||/||b|| 5.636102952452e-05
 18 KSP unpreconditioned resid norm 4.22511838e-02 true resid norm
4.22511838e-02 ||r(i)||/||b|| 4.217132765661e-05
 19 KSP unpreconditioned resid norm 3.160393382046e-02 true resid norm
3.160393382046e-02 ||r(i)||/||b|| 3.154423071329e-05
 20 KSP unpreconditioned resid norm 2.366499890335e-02 true resid norm
2.366499890334e-02 ||r(i)||/||b|| 2.362029326721e-05
 21 KSP unpreconditioned resid norm 1.759504138840e-02 true resid norm
1.759504138839e-02 ||r(i)||/||b|| 1.756180253124e-05
 22 KSP unpreconditioned resid norm 1.309326628086e-02 true resid norm
1.309326628085e-02 ||r(i)||/||b|| 1.306853174355e-05
 23 KSP unpreconditioned resid norm 9.710513089816e-03 true resid norm
9.710513089808e-03 ||r(i)||/||b|| 9.692168923954e-06
 24 KSP unpreconditioned resid norm 7.171039760236e-03 true resid norm
7.171039760227e-03 ||r(i)||/||b|| 7.157492922743e-06
 25 KSP unpreconditioned resid norm 5.277221846963e-03 true resid norm
5.277221846949e-03 ||r(i)||/||b|| 5.267252627824e-06
 26 KSP unpreconditioned resid norm 3.906960734588e-03 true resid norm
3.906960734575e-03 ||r(i)||/||b|| 3.899580080737e-06
 27 KSP unpreconditioned resid norm 2.896843259283e-03 true resid norm
2.896843259273e-03 ||r(i)||/||b|| 2.891370822059e-06
 28 KSP unpreconditioned resid norm 2.140269358580e-03 true resid norm
2.140269358568e-03 ||r(i)||/||b|| 2.136226167881e-06
 29 KSP unpreconditioned resid norm 1.585513255966e-03 true resid norm
1.585513255956e-03 ||r(i)||/||b|| 1.582518057055e-06
 30 KSP unpreconditioned resid norm 1.173839272299e-03 true resid norm
1.173839272299e-03 ||r(i)||/||b|| 1.171621768229e-06
 31 KSP unpreconditioned resid norm 8.777233482545e-04 true resid norm
8.777233482544e-04 ||r(i)||/||b|| 8.760652378615e-07
 32 KSP unpreconditioned resid norm 6.546689191353e-04 true resid norm
6.546689191370e-04 ||r(i)||/||b|| 6.534321816833e-07
 33 KSP unpreconditioned resid norm 4.973281362004e-04 true resid norm
4.973281362015e-04 ||r(i)||/||b|| 4.963886317973e-07
 34 KSP unpreconditioned resid norm 3.775325682448e-04 true resid norm
3.775325682480e-04 ||r(i)||/||b|| 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Thu, Apr 20, 2023 at 11:07 AM Edoardo alinovi 
wrote:

> Hi Matt,
>
> thanks for sharing the literature.
>
> Would you suggest any monolitic approach for the mpiaij/mpibaij matrix
> instead of fieldsplit?
>

I have not seen them be better, but Vanka-type smoothers can work for this
system.


> I did some blind search using gamg/hypre and they look terribile. I guess
> i am missing a trick, probaly they are not the way to go?
>

I believe that trick is that the patches you use have to very specific.

  Thanks,

 Matt


> Thanks!
>
> Il Lun 17 Apr 2023, 13:50 Matthew Knepley  ha scritto:
>
>> On Mon, Apr 17, 2023 at 6:37 AM Edoardo alinovi <
>> edoardo.alin...@gmail.com> wrote:
>>
>>> Sure thing, the solver I am working on is this one:
>>> https://gitlab.com/alie89/flubio-code-fvm.
>>>
>>> It is a 3D, collocated, unstructured, finite volume solver for
>>> incompressibility NS. I can run steady, unsteady and I can use SIMPLE, PISO
>>> and Factional step method (both explicit and fully implicit momentum). I
>>> can also solve for turbulence (k-omega, BSL, SST, Spalart-Allmars, LES). I
>>> have also implemented some kind of Immersed boundary (2D/3D) that I need to
>>> resume at some point.
>>>
>>> Hot topic of the moment, I am developing a fully coupled pressure based
>>> solver using field-split. What I have now is working ok, I have validated
>>> it on a lot of 2D problems and going on with 3D right now.  If all the
>>> tests are passed, I'll focus on tuning the field splitting which looks to
>>> be a quite interesting topic!
>>>
>>
>> I think a very good discussion of the issues from the point of view of
>> FEM is
>>
>>https://arxiv.org/abs/1810.03315
>>
>> There should be a similar analysis from the FVM side, although it might
>> not be possible to
>> find a pressure discretization compatible with the FVM velocity for this
>> purpose.
>>
>>   Thanks,
>>
>>  Matt
>>
>>
>>> Flubio is a project I have been carrying on since PhD days. The
>>> implementation is 99% on my shoulders, despite the fact I am
>>> collaborating with some people around. I am coding evenings and
>>> weekends/free time, it gives me a lot of satisfaction and also a lot of
>>> insights!
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> 
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Hi Matt,

thanks for sharing the literature.

Would you suggest any monolitic approach for the mpiaij/mpibaij matrix
instead of fieldsplit?

I did some blind search using gamg/hypre and they look terribile. I guess i
am missing a trick, probaly they are not the way to go?

Thanks!

Il Lun 17 Apr 2023, 13:50 Matthew Knepley  ha scritto:

> On Mon, Apr 17, 2023 at 6:37 AM Edoardo alinovi 
> wrote:
>
>> Sure thing, the solver I am working on is this one:
>> https://gitlab.com/alie89/flubio-code-fvm.
>>
>> It is a 3D, collocated, unstructured, finite volume solver for
>> incompressibility NS. I can run steady, unsteady and I can use SIMPLE, PISO
>> and Factional step method (both explicit and fully implicit momentum). I
>> can also solve for turbulence (k-omega, BSL, SST, Spalart-Allmars, LES). I
>> have also implemented some kind of Immersed boundary (2D/3D) that I need to
>> resume at some point.
>>
>> Hot topic of the moment, I am developing a fully coupled pressure based
>> solver using field-split. What I have now is working ok, I have validated
>> it on a lot of 2D problems and going on with 3D right now.  If all the
>> tests are passed, I'll focus on tuning the field splitting which looks to
>> be a quite interesting topic!
>>
>
> I think a very good discussion of the issues from the point of view of FEM
> is
>
>https://arxiv.org/abs/1810.03315
>
> There should be a similar analysis from the FVM side, although it might
> not be possible to
> find a pressure discretization compatible with the FVM velocity for this
> purpose.
>
>   Thanks,
>
>  Matt
>
>
>> Flubio is a project I have been carrying on since PhD days. The
>> implementation is 99% on my shoulders, despite the fact I am
>> collaborating with some people around. I am coding evenings and
>> weekends/free time, it gives me a lot of satisfaction and also a lot of
>> insights!
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> 
>

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Mon, Apr 17, 2023 at 6:37 AM Edoardo alinovi 
wrote:

> Sure thing, the solver I am working on is this one:
> https://gitlab.com/alie89/flubio-code-fvm.
>
> It is a 3D, collocated, unstructured, finite volume solver for
> incompressibility NS. I can run steady, unsteady and I can use SIMPLE, PISO
> and Factional step method (both explicit and fully implicit momentum). I
> can also solve for turbulence (k-omega, BSL, SST, Spalart-Allmars, LES). I
> have also implemented some kind of Immersed boundary (2D/3D) that I need to
> resume at some point.
>
> Hot topic of the moment, I am developing a fully coupled pressure based
> solver using field-split. What I have now is working ok, I have validated
> it on a lot of 2D problems and going on with 3D right now.  If all the
> tests are passed, I'll focus on tuning the field splitting which looks to
> be a quite interesting topic!
>

I think a very good discussion of the issues from the point of view of FEM
is

   https://arxiv.org/abs/1810.03315

There should be a similar analysis from the FVM side, although it might not
be possible to
find a pressure discretization compatible with the FVM velocity for this
purpose.

  Thanks,

 Matt


> Flubio is a project I have been carrying on since PhD days. The
> implementation is 99% on my shoulders, despite the fact I am
> collaborating with some people around. I am coding evenings and
> weekends/free time, it gives me a lot of satisfaction and also a lot of
> insights!
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Sure thing, the solver I am working on is this one:
https://gitlab.com/alie89/flubio-code-fvm.

It is a 3D, collocated, unstructured, finite volume solver for
incompressibility NS. I can run steady, unsteady and I can use SIMPLE, PISO
and Factional step method (both explicit and fully implicit momentum). I
can also solve for turbulence (k-omega, BSL, SST, Spalart-Allmars, LES). I
have also implemented some kind of Immersed boundary (2D/3D) that I need to
resume at some point.

Hot topic of the moment, I am developing a fully coupled pressure based
solver using field-split. What I have now is working ok, I have validated
it on a lot of 2D problems and going on with 3D right now.  If all the
tests are passed, I'll focus on tuning the field splitting which looks to
be a quite interesting topic!

Flubio is a project I have been carrying on since PhD days. The
implementation is 99% on my shoulders, despite the fact I am
collaborating with some people around. I am coding evenings and
weekends/free time, it gives me a lot of satisfaction and also a lot of
insights!

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Mon, Apr 17, 2023 at 6:16 AM Edoardo alinovi 
wrote:

> Do you mean the solver I am messing around? XD
>

Yes, and what physics it is targeting.

  THanks,

Matt
-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Do you mean the solver I am messing around? XD

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Mon, Apr 17, 2023 at 6:09 AM Edoardo alinovi 
wrote:

> Thanks Matt, your always there when you need <3
>

Glad it's working! Sometime you have to tell me what it is solving.

  Thanks,

 Matt

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Thanks Matt, your always there when you need <3

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Aaah yes you are right. Do not ask me why, but I was not getting this with
3.18.5, odd.

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Mon, Apr 17, 2023 at 6:00 AM Edoardo alinovi 
wrote:

> Hey Matt,
>
> Thanks for the help. Here is the error:
>
> [0]PETSC ERROR: - Error Message
> --
> [0]PETSC ERROR: Object is in wrong state
> [0]PETSC ERROR: Not for unassembled vector, did you call
> VecAssemblyBegin()/VecAssemblyEnd()?
> [0]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
> be the program crashed before they were used or a spelling mistake, etc!
> [0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
> 0.1E-0001 source: code
> [0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
> hypre source: code
> [0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
> 0.1E-0001 source: code
> [0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
> bjacobi source: code
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.19.0, unknown
> [0]PETSC ERROR: flubio_coupled on a arch-gnu named betelgeuse by edo Mon
> Apr 17 12:05:28 2023
> [0]PETSC ERROR: Configure options PETSC_ARCH=arch-gnu FOPTFLAGS=-O3
> COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no
> -with-mpi-dir=/home/edo/user_software_repository/openmpi-4.1.2/build
> -download-fblaslapack=1 -download-superlu_dist -download-mumps
> -download-hypre -download-metis -download-parmetis -download-scalapack
> --download-ml -download-slepc -download-spai -download-fftw
> [0]PETSC ERROR: #1 MatMult() at
> /home/edo/user_software_repository/petsc/src/mat/interface/matrix.c:2557
> [1]PETSC ERROR: - Error Message
> --
> [1]PETSC ERROR: Object is in wrong state
> [1]PETSC ERROR: Not for unassembled vector, did you call
> VecAssemblyBegin()/VecAssemblyEnd()?
> [1]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
> be the program crashed before they were used or a spelling mistake, etc!
> [1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
> 0.1E-0001 source: code
> [1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
> hypre source: code
> [1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
> 0.1E-0001 source: code
> [1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
> bjacobi source: code
> [1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> [1]PETSC ERROR: Petsc Release Version 3.19.0, unknown
> [1]PETSC ERROR: flubio_coupled on a arch-gnu named betelgeuse by edo Mon
> Apr 17 12:05:28 2023
> [1]PETSC ERROR: Configure options PETSC_ARCH=arch-gnu FOPTFLAGS=-O3
> COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no
> -with-mpi-dir=/home/edo/user_software_repository/openmpi-4.1.2/build
> -download-fblaslapack=1 -download-superlu_dist -download-mumps
> -download-hypre -download-metis -download-parmetis -download-scalapack
> --download-ml -download-slepc -download-spai -download-fftw
> [1]PETSC ERROR: #1 MatMult() at
> /home/edo/user_software_repository/petsc/src/mat/interface/matrix.c:2557
> [2]PETSC ERROR: - Error Message
> --
> [2]PETSC ERROR: Object is in wrong state
> [2]PETSC ERROR: Not for unassembled vector, did you call
> VecAssemblyBegin()/VecAssemblyEnd()?
> [2]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
> be the program crashed before they were used or a spelling mistake, etc!
> [2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
> 0.1E-0001 source: code
> [2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
> hypre source: code
> [2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
> 0.1E-0001 source: code
> [2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
> bjacobi source: code
> [2]PETSC ERROR: [3]PETSC ERROR: - Error Message
> --
> [3]PETSC ERROR: Object is in wrong state
> [3]PETSC ERROR: Not for unassembled vector, did you call
> VecAssemblyBegin()/VecAssemblyEnd()?
> [3]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
> be the program crashed before they were used or a spelling mistake, etc!
> [3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
> 0.1E-0001 source: code
> [3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
> hypre source: code
> [3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
> 0.1E-0001 source: code
> [3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
> bjacobi source: code
> See https://petsc.org/release/faq/ for trouble shooting.
> [2]PETSC ERROR: 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Hey Matt,

Thanks for the help. Here is the error:

[0]PETSC ERROR: - Error Message
--
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Not for unassembled vector, did you call
VecAssemblyBegin()/VecAssemblyEnd()?
[0]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
be the program crashed before they were used or a spelling mistake, etc!
[0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
0.1E-0001 source: code
[0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
hypre source: code
[0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
0.1E-0001 source: code
[0]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
bjacobi source: code
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.19.0, unknown
[0]PETSC ERROR: flubio_coupled on a arch-gnu named betelgeuse by edo Mon
Apr 17 12:05:28 2023
[0]PETSC ERROR: Configure options PETSC_ARCH=arch-gnu FOPTFLAGS=-O3
COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no
-with-mpi-dir=/home/edo/user_software_repository/openmpi-4.1.2/build
-download-fblaslapack=1 -download-superlu_dist -download-mumps
-download-hypre -download-metis -download-parmetis -download-scalapack
--download-ml -download-slepc -download-spai -download-fftw
[0]PETSC ERROR: #1 MatMult() at
/home/edo/user_software_repository/petsc/src/mat/interface/matrix.c:2557
[1]PETSC ERROR: - Error Message
--
[1]PETSC ERROR: Object is in wrong state
[1]PETSC ERROR: Not for unassembled vector, did you call
VecAssemblyBegin()/VecAssemblyEnd()?
[1]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
be the program crashed before they were used or a spelling mistake, etc!
[1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
0.1E-0001 source: code
[1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
hypre source: code
[1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
0.1E-0001 source: code
[1]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
bjacobi source: code
[1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[1]PETSC ERROR: Petsc Release Version 3.19.0, unknown
[1]PETSC ERROR: flubio_coupled on a arch-gnu named betelgeuse by edo Mon
Apr 17 12:05:28 2023
[1]PETSC ERROR: Configure options PETSC_ARCH=arch-gnu FOPTFLAGS=-O3
COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no
-with-mpi-dir=/home/edo/user_software_repository/openmpi-4.1.2/build
-download-fblaslapack=1 -download-superlu_dist -download-mumps
-download-hypre -download-metis -download-parmetis -download-scalapack
--download-ml -download-slepc -download-spai -download-fftw
[1]PETSC ERROR: #1 MatMult() at
/home/edo/user_software_repository/petsc/src/mat/interface/matrix.c:2557
[2]PETSC ERROR: - Error Message
--
[2]PETSC ERROR: Object is in wrong state
[2]PETSC ERROR: Not for unassembled vector, did you call
VecAssemblyBegin()/VecAssemblyEnd()?
[2]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
be the program crashed before they were used or a spelling mistake, etc!
[2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
0.1E-0001 source: code
[2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
hypre source: code
[2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
0.1E-0001 source: code
[2]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
bjacobi source: code
[2]PETSC ERROR: [3]PETSC ERROR: - Error Message
--
[3]PETSC ERROR: Object is in wrong state
[3]PETSC ERROR: Not for unassembled vector, did you call
VecAssemblyBegin()/VecAssemblyEnd()?
[3]PETSC ERROR: WARNING! There are option(s) set that were not used! Could
be the program crashed before they were used or a spelling mistake, etc!
[3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_ksp_rtol value:
0.1E-0001 source: code
[3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_p_pc_type value:
hypre source: code
[3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_ksp_rtol value:
0.1E-0001 source: code
[3]PETSC ERROR:   Option left: name:-UPeqn_fieldsplit_u_pc_type value:
bjacobi source: code
See https://petsc.org/release/faq/ for trouble shooting.
[2]PETSC ERROR: Petsc Release Version 3.19.0, unknown
[2]PETSC ERROR: flubio_coupled on a arch-gnu named betelgeuse by edo Mon
Apr 17 12:05:28 2023
[2]PETSC ERROR: Configure options PETSC_ARCH=arch-gnu FOPTFLAGS=-O3
COPTFLAGS=-O3 CXXOPTFLAGS=-O3 

Re: [petsc-users] issues with VecSetValues in petsc 3.19

[Matthew Knepley]

On Mon, Apr 17, 2023 at 5:36 AM Edoardo alinovi 
wrote:

> Hello Barry, Matt, Jed,
>
> I have just installed the latest and greatest version of petsc and I am
> hitting a problem I did not have in previous releases.
>
> Here is the error:
>
>
>
>
> *[1]PETSC ERROR: - Error Message
> --[1]PETSC
> ERROR: Object is in wrong state[1]PETSC ERROR: Not for unassembled vector,
> did you call VecAssemblyBegin()/VecAssemblyEnd()?[1]PETSC ERROR: WARNING!
> There are option(s) set that were not used! Could be the program crashed
> before they were used or a spelling mistake, etc!*
>
> the vector x I am filling is created with:
>
>- call VecDuplicate(this%rhs, x, ierr)
>
> rhs is allocated and I called VecAssemblyBegin()/VecAssemblyEnd() on, do I
> need to call it on duplicated vectors as well now on?
>

Only if you change the values. Can you show the entire stack from the error
message?

  Thanks,

 Matt


> Thank you!
>
-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

[petsc-users] issues with VecSetValues in petsc 3.19

[Edoardo alinovi]

Hello Barry, Matt, Jed,

I have just installed the latest and greatest version of petsc and I am
hitting a problem I did not have in previous releases.

Here is the error:




*[1]PETSC ERROR: - Error Message
--[1]PETSC
ERROR: Object is in wrong state[1]PETSC ERROR: Not for unassembled vector,
did you call VecAssemblyBegin()/VecAssemblyEnd()?[1]PETSC ERROR: WARNING!
There are option(s) set that were not used! Could be the program crashed
before they were used or a spelling mistake, etc!*

the vector x I am filling is created with:

   - call VecDuplicate(this%rhs, x, ierr)

rhs is allocated and I called VecAssemblyBegin()/VecAssemblyEnd() on, do I
need to call it on duplicated vectors as well now on?

Thank you!