Hi,
I fully agree with Timo.
We run some Dune models (shallow water) on our high performance cluster. From my
experiences with Dune I can say that models with more than a few Mio cells can
become very uncomfortable in the handling. Preprocessing and postprocesing can
get very complex and excution time may disturb your workflow.
Our models scales up to 10000-1000 cells per core. So a 1-2 hundred thousand
elements is a good size for your computer. This also means that you should think
about using a cluster. Not only for higher resolution, even if you end up with
a smaller model you will perform parameter, sensitivity or uncertainty studies.
This means that your will run you model many times.
If we think about performance, the time step size and convergence are the main
challange for implicit models. If you model some small scale phenomenas you
might be limited to the time scale of the physics. Time step size does not scale
on a cluster so you may have to simplify the model concept.
Best regards
Leo
> Timo Koch <[email protected]> hat am 4. April 2019 um 12:20
> geschrieben:
>
>
>
>
> On 04.04.19 10:05, Johannes Koestel wrote:
>
> > >
> > Hej Bernd,
> >
> >
> >
> > Thanks for the quick reply.
> >
> > Ah, I am wondering about the second case, of course.
> >
> > I am a bit unsure about the terminology. I was referring to what is
> > referred to as 2p2c in the attached paper.
> >
> > (no, there will not be roots in the models I want to run).
> >
> >
> >
> > The Stokes equation on an explicitly resolved pore geometry would be
> > ultimate goal, especially if it could be coupled to a
> >
> > Darcy-domain. But I had the impression that it is still unrealistic
> > without true supercomputers (with whatever software).
> >
> >
> >
> > As for setting up Dumux and trying out one time step, I surely would
> > appreciate help. Thank you.
> >
> > I will take the Dumux short course at the Interpore and plan to try it
> > out after that. I probably will again contact the forum by then.
> >
> >
> >
> > For now, I only need a rough estimate on what kind of discretization I
> > could run Dumux with on the specified computer.
> >
> > It would be for a grant application.
> >
> >
> >
> > It is planned to gather tomographies on water flow in soil. These 3-D
> > image will be approximately 250 megavoxels large (a cylinder with 600 vx
> > diameter and 900 vx height).
> >
> > Due to the dynamic meshing, the number of elements depends on the
> > permeability structures, saturations and potential fields, right?
> >
> > OK.. so it is not so easy to estimate how much computation time it
> > would need, correct?
> >
> > >
> Hi John,
>
>
>
> which spatial size do your tomography images cover? Will the resolution be
> above or below an REV-scale. That will also determine how many cells you want
> to have. And you are right, when you track a front, e.g., with adaptive grid
> refinement you may significantly reduce the number of cells. Also how detailed
> will your permeability/porosity data be?
>
>
>
> Anyway, just to come up with a very rough estimate: if you want 250 Mio cells
> at 40 cores, that's gonna be 6Mio cells per core. It could easily take
> 10-30min for a time step (fully-implicit, Newton), which would mean one
> simulation would probably be in the order of weeks. You might also get memory
> problems at that size, because we currently don't implement matrix-free
> solvers.
>
>
>
> Do you really need the same resolution in the model as in the tomography
> image? I.e. will there be interesting structures that are so small that this
> resolution is needed to capture the relevant effects?
>
>
>
> And to be fully honest with you, we don't have too much experience with
> simulations at that size.
>
>
>
> Cheers,
>
> Timo
>
>
>
> > >
> >
> >
> >
> >
> > But if I would reduce the resolution by a factor 2, i.e. app. 43
> > megavoxels (300 vx diameter and 450 height), this would put me on the safe
> > side?
> >
> >
> >
> > Cheers and thanks,
> >
> >
> >
> > /John
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > From: Flemisch, Bernd <[email protected]>
> > mailto:[email protected]
> > Sent: Thursday, April 4, 2019 8:59 AM
> > To: [email protected]
> > mailto:[email protected]
> > Cc: Johannes Koestel <[email protected]> mailto:[email protected]
> > Subject: AW: system requirements for Dumux with 2C2P in 3-D
> >
> >
> >
> > Hi Johannes,
> >
> >
> >
> > it sounds like you want to solve (Navier)Stokes on an explicitly
> > resolved pore geometry? Or do you want to use Darcy's law without resolving
> > the pores but using saturation, permeability, porosity and so on? The first
> > case would currently be impossible with Dumux since we don't have a
> > two-phase Stokes model. The second case would be feasible, we could look at
> > how much time it needs for one time step and then estimate the overall cost.
> >
> >
> >
> > Kind regards
> >
> > Bernd
> >
> >
> >
> >
> >
> > --
> > _______________________________________________________________
> >
> > Bernd Flemisch phone: +49 711 685 69162
> > IWS, Universität Stuttgart fax: +49 711 685 60430
> > Pfaffenwaldring 61 email: [email protected]
> > mailto:[email protected]
> > D-70569 Stuttgart url:http://www.hydrosys.uni-stuttgart.de
> > _______________________________________________________________
> >
> >
> > ---------------------------------------------
> >
> > Von: Dumux <[email protected]
> > mailto:[email protected] > im Auftrag von Johannes
> > Koestel <[email protected] mailto:[email protected] >
> > Gesendet: Mittwoch, 3. April 2019 14:59:24
> > An: [email protected]
> > mailto:[email protected]
> > Betreff: [DuMuX] system requirements for Dumux with 2C2P in 3-D
> >
> >
> >
> > Hej,
> >
> >
> >
> > I am interested in using Dumux to run flow and transport simulations in
> > soil with 3-D resolution.
> >
> >
> >
> > Could you help me with getting an idea about the computational demands?
> >
> >
> >
> > More specifically:
> >
> > How much days would I approximately need to …
> >
> > .. model water and air flows (2c2p) under transient conditions in soil
> > (without fissures or similar) ..
> >
> > .. and a perfectly conservative tracer transport with it (say D2O) ..
> >
> > .. in a 3-D domain with 250 megavoxels …
> >
> > .. using an Intel Xeon with 40 CPUs @2.4 GHz, 128 GB RAM?
> >
> >
> >
> > Will it be hours, days or years? Will it run at all on such a system?
> >
> >
> >
> > Thanks a lot for your help,
> >
> >
> >
> > Cheers from Sweden,
> >
> >
> >
> > /John
> >
> > ---
> > När du skickar e-post till SLU så innebär detta att SLU behandlar dina
> > personuppgifter. För att läsa mer om hur detta går till, klicka här
> > https://www.slu.se/om-slu/kontakta-slu/personuppgifter/
> > E-mailing SLU will result in SLU processing your personal data. For
> > more information on how this is done, click here
> > https://www.slu.se/en/about-slu/contact-slu/personal-data/
> >
> > ---
> > När du skickar e-post till SLU så innebär detta att SLU behandlar dina
> > personuppgifter. För att läsa mer om hur detta går till, klicka här
> > https://www.slu.se/om-slu/kontakta-slu/personuppgifter/
> > E-mailing SLU will result in SLU processing your personal data. For
> > more information on how this is done, click here
> > https://www.slu.se/en/about-slu/contact-slu/personal-data/
> >
> >
> > _______________________________________________
> > Dumux mailing list
> > [email protected] mailto:[email protected]
> > https://listserv.uni-stuttgart.de/mailman/listinfo/dumux
> >
> > >
> --
> _______________________________________________________________
>
> Timo Koch phone: +49 711 685 64676
> IWS, Universität Stuttgart fax: +49 711 685 60430
> Pfaffenwaldring 61 email: [email protected]
> mailto:[email protected]
> D-70569 Stuttgart url: http://www.hydrosys.uni-stuttgart.de
> _______________________________________________________________
>
> _______________________________________________
> Dumux mailing list
> [email protected]
> https://listserv.uni-stuttgart.de/mailman/listinfo/dumux
>
Im Auftrag
Dr.-Ing. Leopold Stadler
--
Referat Numerische Verfahren im Wasserbau
Abteilung Wasserbau im Binnenbereich
Bundesanstalt für Wasserbau
Federal Waterways Engineering and Research Institute
Kußmaulstraße 17 | 76187 Karlsruhe
E-Mail: [email protected]
Tel.: +49 721 9726-3525
Fax: +49 721 9726-4540
https://www.baw.de
Der Umwelt zuliebe:
Drucken Sie diese Mail nur aus, wenn dies unbedingt erforderlich ist!
_______________________________________________
Dumux mailing list
[email protected]
https://listserv.uni-stuttgart.de/mailman/listinfo/dumux
