This shows you the differences between two versions of the page.
| |
| software:openfoam:mills [2017-10-23 18:01] – created sraskar | software:openfoam:mills [2021-04-27 16:21] (current) – external edit 127.0.0.1 |
|---|
| | ==== Running your OpenFOAM application on Mills ==== |
| |
| | Using the ''openmpi-psm.qs'' job script template found in ''/opt/shared/templates/gridengine/openmpi'' provides the best performance for a parallel MPI run for OpenFOAM applications. Once you have copied the template for your job script (e.g. ''qsub_sowfa.qs''), then edit the following for your OpenFOAM application: |
| | |
| | - Modify NPROC to select the number of slots (cores); best results have been found specifying twice as many cores (a multiple of 24) and binding cores due to the shared FP unit (see steps 5 and 6). |
| | - Modify the email notification ''-m'' and ''-M'' according to the instructions if you want email to be sent at the end of the job, if the job is aborted, rescheduled, or suspended. |
| | - Setup your environment with your newly built OpenFOAM application by replacing the ''vpkg_require openmpi/1.4.4-gcc'' with your VALET package name (e.g. ''sowfa/2.0-OF2.1-gcc'') |
| | - Setup the MPI executable and options (arguments) for your OpenFOAM application (e.g. ''ABLSolver -parallel > log.ABLSolver.3.${JOB_ID} 2>err.ABLSolver.3'') |
| | - Remove ''#'' in front of ''#WANT_CPU_AFFINITY=YES'' |
| | - Remove ''#'' in front of ''#WANT_HALF_CORES_ONLY=YES'' |
| | |
| | Now submit your job script using [[general/jobsched/exclusive|exclusive access]] to prevent other jobs from running on the same node(s) |
| | |
| | <code> |
| | qsub -l exclusive=1 qsub_sowfa.qs |
| | </code> |
| | |
| | <note important> There is a 1.5.x Grid Engine integration error when **only** using OpenFOAM 2.1.1 compiled with ''openmpi-1.5.3'' which causes the following warning when initiating an MPI run |
| | |
| | <code> mca: base: component_find: unable to open /opt/shared/openfoam/2.1. |
| | 1-gcc/ThirdParty-2.1.1/platforms/linux64Gcc/openmpi-1.5.3/lib/openmpi/mca_plm_rs |
| | hd: /opt/shared/openfoam/2.1.1-gcc/ThirdParty-2.1.1/platforms/linux64Gcc/openmpi |
| | -1.5.3/lib/openmpi/mca_plm_rshd.so: cannot open shared object file: Permission d |
| | enied (ignored)</code> |
| | |
| | This is a warning and not an error. Please ignore.</note> |