Differences

This shows you the differences between two versions of the page.

--- technical:recipes:ls-dyna [2024-06-07 12:34] – [Open MPI install] frey
+++ technical:recipes:ls-dyna [2024-06-07 12:51] (current) – [VALET package definition] frey
@@ Line 21: / Line 21: @@
 For this example, LS-DYNA will be installed in a workgroup's storage.  The base path ''$WORKDIR'' will be referenced in this recipe, which implies that prior to installation the ''workgroup'' command was used to spawn a workgroup shell.  The recipe assumes version R15.0.2 and the variants mentioned above.
-===== Directory Organization =====
+===== Directory organization =====
 The LS-DYNA software packages must be downloaded to the cluster.  To facilitate that, a directory must be created to hold those files.  Rather than type-out the full path every time, the variable ''LS_DYNA_PREFIX'' will be set to the base directory containing LS-DYNA software:
@@ Line 203: / Line 203: @@
                 - intel/2019
 </file>
+===== VALET package definition =====
+With the necessary dependencies satisfied, a VALET package definition for LS-DYNA can be created.  Assuming this is the first time LS-DYNA is being installed, the file will need to be created as it appears below; if versions/variants are added in the future, that file should be augmented with just the new version/variant information.  The ''<LS_DYNA_PREFIX>'' text must be replaced in the package definition with the value of that environment variable that was assigned at the beginning of this recipe:
+<code bash>
+$ echo $LS_DYNA_PREFIX
+/work/group/sw/ls-dyna
+</code>
+The ''<LSTC-LICENSE-SERVER-NAME>'' text must be replaced with the IP address or DNS name of the computer that services the LSTC LS-DYNA license that was purchased.
+<file yaml ls-dyna.vpkg_yaml>
+ls-dyna:
+    prefix: <LS_DYNA_PREFIX>
+    url: http://www.lstc.com/
+    description: general-purpose finite element program simulating complex real world problems
+    actions:
+        - variable: LSTC_LICENSE
+          action: set
+          value: network
+        - variable: LSTC_LICENSE_SERVER
+          action: set
+          value: <LSTC-LICENSE-SERVER-NAME>
+        - variable: LSTC_INTERNAL_CLIENT
+          action: set
+          value: off
+        - variable: LSTC_ROOT
+          action: set
+          value: ${VALET_PATH_PREFIX}
+        - bindir: ${VALET_PATH_PREFIX}
+    default-version: "15.0.2:double,hybrid,avx2"
+    versions:
+        "15.0.2:single,hybrid,avx2":
+            description: 15.0.2, single-precision, Hybrid (SMP+MPP), AVX2
+            prefix: 15.0.2/avx2/hyb/single
+            dependencies:
+                - openmpi/4.0.5:intel19,ls-dyna
+        "15.0.2:double,hybrid,avx2":
+            description: 15.0.2, double-precision, Hybrid (SMP+MPP), AVX2
+            prefix: 15.0.2/avx2/hyb/double
+            dependencies:
+                - openmpi/4.0.5:intel19,ls-dyna
+</file>
+Loading one of these packages into the shell environment makes the ''ls-dyna'' command (the symlink created during install) available:
+<code bash>
+$ vpkg_require ls-dyna/single,hybrid,avx2
+Adding dependency `intel/2019u5` to your environment
+Adding dependency `libfabric/1.17.1` to your environment
+Adding dependency `openmpi/4.0.5:intel19,ls-dyna` to your environment
+Adding package `ls-dyna/15.0.2:single,avx2,hybrid` to your environment
+$ which ls-dyna
+/work/group/sw/ls-dyna/15.0.2/avx2/hyb/single/ls-dyna
+</code>
+===== Running jobs =====
+On the Caviness/DARWIN cluster, LS-DYNA computation must be submitted as a Slurm job.  The variants installed here use Open MPI, so the job script template at ''/opt/templates/slurm/generic/mpi/openmpi/openmpi.qs'' is a starting point for LS-DYNA jobs.  As in all cases for parallel jobs:
+  * The number of Slurm //tasks// represent the MPI ranks for a hybrid or MPP variant of LS-DYNA
+  * The number of CPUs per Slurm task represent the Open MP threads for an smp or hybrid variant of LS-DYNA
+Follow the comments in the script header to determine which flags to alter and how to do so.  The script template must be altered to add the desired version/variant of LS-DYNA to the job environment:
+<code>
+#
+# [EDIT] Do any pre-processing, staging, environment setup with VALET
+#        or explicit changes to PATH, LD_LIBRARY_PATH, etc.
+#
+vpkg_require ls-dyna/15.0.2:single,hybrid,avx2
+</code>
+Toward the end of the job script template, the LS-DYNA program is run:
+<code>
+#
+# [EDIT] Execute your MPI program
+#
+${UD_MPIRUN} ls-dyna i=my_model.k
+mpi_rc=$?
+</code>