Hi Mohamad, I just ran the first time steps of this case with Red Had 6 and OpenMPI, both with one and with four MPI processes. It works over here. I am running on a binary compiled from git yesterday, so I have not checked the release version of flow. So at this point, simply sharing that your problem does not happen on my set-up.
Generally speaking, SPE10 model 2 may not be the best model to use. It was created as an upscaling case with exaggerated heterogeneities to deliberately make it difficult to run. We currently do not include it in any of our integration testing, so the probability of having severe convergence issues with default tuning and solvers in flow is high. The SPE9 or Norne model is far better for benchmarking purposes. If you have particular interest in SPE10 model 2, there are a couple of steps you should take. One is to re-write it into a two-phase case (when it was created we did not provide support for running two-phase cases, so gas was added with zero saturation). Then you should test with the multigrid based solver provided through a command line switch (please check the manual for documentation). Lastly, you may also want to play with tolerances. Currently we have relatively strict tolerances by default, and time steps with convergence failure is not accepted by default. This means that hard-to-converge models may fail to run through. Hence, it may not be what you want for SPE10 model 2. Cheers, Alf From: Opm <[email protected]> On Behalf Of sindimo Sent: 26. september 2018 21:43 To: Markus Blatt <[email protected]>; [email protected] Subject: Re: [Opm] MPI Parallel OPM Flow - SPE 10 Model 2 Hangs Dear Dr. Blatt, Thank you for your reply, I really appreciate it. I am running that model as part of my PhD research work at MIT, it's one of the main models I am benchmarking. I was suspecting this might be an issue with Flow RedHat7/MPICH build but I am able to reproduce the same problem even on an Ubuntu machine with OpenMPI. Below are step by step instructions on how to reproduce the hang on a freshly installed AWS Ubuntu machine in case this helps with investigating it further. Similarly this is reproducible on a freshly installed AWS RedHat7 machine (but using yum to install flow). Thanks again. Sincerely, Mohamad Sindi #--------------Steps to reproduce problem------------------------------------- #Launch a fresh new machine instance on Amazon AWS for Ubuntu Server 16.04 (e.g. m4.2xlarge with 32 GB RAM and 4 cores) #Install OPM Flow using instructions from website (https://opm-project.org/?page_id=245): sudo apt-get update sudo apt-get install software-properties-common sudo apt-add-repository ppa:opm/ppa sudo apt-get update sudo apt-get install mpi-default-bin sudo apt-get install libopm-simulators-bin #Get spe10model2 model from https://github.com/OPM/opm-data/tree/master/spe10model2 #These are the md5sum for the files used just to make sure we are running the exact same model: ubuntu@ip-172-31-42-209:~/spe10model2$ ls SPE10_MODEL2.DATA SPE10MODEL2_PERM.INC SPE10MODEL2_PHI.INC SPE10MODEL2_TOPS.INC ubuntu@ip-172-31-42-209:~/spe10model2$ md5sum * 322c2d60c8f6a7982fc03a42630ba001 SPE10_MODEL2.DATA 72b5f0f1c8fe3ef133bd533c784ff479 SPE10MODEL2_PERM.INC 9bcc43dbaa2670b4ab2190fef3664310 SPE10MODEL2_PHI.INC 2a24fda84f073e96af295b8116400fd3 SPE10MODEL2_TOPS.INC #Run the model which hangs (seems during cell partitioning): ubuntu@ip-172-31-42-209:~/spe10model2$ mpirun --mca btl tcp,self -np 4 /usr/bin/flow SPE10_MODEL2.DATA output_dir=out_parallel ********************************************************************** * * * This is flow 2018.04 * * * * Flow is a simulator for fully implicit three-phase black-oil flow, * * including solvent and polymer capabilities. * * For more information, see https://opm-project.org * * * ********************************************************************** After loadbalancing process 0 has 322630 cells. On Wed, Sep 26, 2018 at 4:10 AM Markus Blatt <[email protected]<mailto:[email protected]>> wrote: Hi On Tue, Sep 25, 2018 at 09:30:26PM -0400, sindimo wrote: > I am running OPM Flow with MPICH MPI on RedHat 7 (installed via yum through > OPM repo, version 2018.04). > > I am able to successfully run in parallel some of the test models (norne, > spe5, spe9), however spe10model2 always hangs when I try to launch it. It > seems it hangs during the cell partitioning as below. It just partially > does the partitioning on a subset of the processes only (example below > where I have 4 processes and it hangs after doing 2 partitions, I've also > tried with 8 processors and it shows similar behavior). Any help with this > is much appreciated as I need to run SPE10 for some work I am doing, many > thanks! May I ask what work that is? So you are using a release (even of a target distribution). That is a bit weired. Unfortunately I do not have access to such a system and cannot be of much help here. Maybe somebody else can do a quick test? But I did a quick test with the current master on my system and it works with 4 processes. So if nobody else can help you, then you might want to checkout master and compile OPM yourself. Cheers, Markus -- Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de Pedettistr. 38, 85072 Eichstätt, Germany, USt-Id: DE279960836 Tel.: +49 (0) 160 97590858 _______________________________________________ Opm mailing list [email protected]<mailto:[email protected]> https://opm-project.org/cgi-bin/mailman/listinfo/opm ------------------------------------------------------------------- The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorized use, dissemination of the information or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Thank you
_______________________________________________ Opm mailing list [email protected] https://opm-project.org/cgi-bin/mailman/listinfo/opm
