technical:recipes:gromacs-plumed

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
technical:recipes:gromacs-plumed [2024-05-12 14:26] – bkangtechnical:recipes:gromacs-plumed [2024-05-12 15:03] (current) – [Scaling] bkang
Line 1: Line 1:
 +====== Building Gromacs-Plumed on Caviness/DARWIN ======
  
 +The build procedure outlined herein uses Open MPI on top of the Intel compiler suite.
 +
 +===== Directory Preparation =====
 +
 +To begin, choose a directory in which the Gromacs_Plumed version(s) will be built and installed.  To build in your home directory, for example:
 +<code bash>
 +[user@login00.darwin ~]$ GP_BASEDIR=~/sw/Gromacs_Plumed
 +[user@login00.darwin ~]$ G_BASEDIR="${GP_BASEDIR}/Gromacs"
 +[user@login00.darwin ~]$ P_BASEDIR="${GP_BASEDIR}/Plumed"
 +[user@login00.darwin ~]$ GP_BASEDIR_PRIVS=0700
 +</code>
 +If you are managing Gromacs_Plumed software for your entire workgroup, you could instead use
 +<code bash>
 +[user@login00.darwin ~]$ GP_BASEDIR="${WORKDIR}/sw/Gromacs_Plumed"
 +[user@login00.darwin ~]$ G_BASEDIR="${GP_BASEDIR}/Gromacs"
 +[user@login00.darwin ~]$ P_BASEDIR="${GP_BASEDIR}/Plumed"
 +[user@login00.darwin ~]$ GP_BASEDIR_PRIVS=2770
 +</code>
 +
 +
 +If the directory hierarchy does not yet exist, it can be setup using
 +<code bash>
 +[user@login00.darwin ~]$ mkdir -p -m $GP_BASEDIR_PRIVS "${GP_BASEDIR}"
 +[user@login00.darwin ~]$ mkdir -p -m $GP_BASEDIR_PRIVS "${G_BASEDIR}"
 +[user@login00.darwin ~]$ mkdir -p -m $GP_BASEDIR_PRIVS "${P_BASEDIR}"
 +</code>
 +
 +===== Source Preparation: Plumed =====
 +
 +In this example version 2.8.3 of Plumed will be built. Our standard recipes for Plumed will entail use of the Intel compilers, the MKL for BLAS/LAPACK/FFTW/ScaLAPACK/BLACS, and Open MPI for parallelism.
 +
 +Download and unpack "plumed-2.8.3.tgz" and create a directory to hold our base build of plumed-2.8.3, naming it with the version identifier:  ''v2.8.3''.  The source is then unpacked therein:
 +<code bash>
 +[user@login00.darwin ~]$ cd $P_BASEDIR
 +[user@login00.darwin Plumed]$ wget https://github.com/plumed/plumed2/releases/download/v2.8.3/plumed-2.8.3.tgz
 +[user@login00.darwin Plumed]$ tar -xvzf plumed-2.8.3.tgz
 +</code>
 +
 +===== Compilation and Installation: Plumed=====
 +
 +We use configure for compilation and installation on "v2.8.3" directory.
 +<code bash>
 +[user@login00.darwin ~]$ P_INSTALL_PREFIX="${P_BASEDIR}/v2.8.3"
 +[user@login00.darwin ~]$ export P_INSTALL_PREFIX
 +[user@login00.darwin ~]$ P_SRC_PREFIX="${P_BASEDIR}/plumed-2.8.3"
 +[user@login00.darwin ~]$ mkdir -m $GP_BASEDIR_PRIVS "$P_INSTALL_PREFIX"
 +[user@login00.darwin ~]$ cd "$P_SRC_PREFIX"
 +[user@login00.darwin plumed-2.8.3]$ vpkg_devrequire openmpi/4.1.5:intel-oneapi-2023
 +[user@login00.darwin plumed-2.8.3]$ ./configure --prefix=$P_INSTALL_PREFIX  CC=mpicc CXX=mpicxx FC=mpifort  LDFLAGS=-qmkl=sequential 
 +[user@login00.darwin plumed-2.8.3]$ make
 +[user@login00.darwin plumed-2.8.3]$ make install
 +</code>
 +Our current working directory is now the source directory.
 +Here, we create ''sourceme.sh'' :
 +<file make sourceme.sh>
 +TPATH="${P_INSTALL_PREFIX}"
 +export PATH="$TPATH/bin/:$PATH"
 +export LIBRARY_PATH="$TPATH/lib/:$LIBRARY_PATH"
 +export LD_LIBRARY_PATH="$TPATH/lib/:$LD_LIBRARY_PATH"
 +export PLUMED_KERNEL="$TPATH/lib/libplumedKernel.so"
 +export PLUMED_VIMPATH="$TPATH/lib/plumed/vim/"
 +export PYTHONPATH="$TPATH/lib/python/:$PYTHONPATH"
 +
 +export CPATH="$TPATH/include/:$CPATH”
 +export INCLUDE="$TPATH/include/:$INCLUDE”
 +</file>
 +
 +
 +Executing the script.
 +<code bash>
 +[user@login00.darwin plumed-2.8.3]$ source sourceme.sh
 +</code> 
 +
 +===== Source Preparation: Gromacs =====
 +
 +In this example version 2022.5 of Gromacs will be built. Our standard recipes for Gromacs will entail use of the Intel compilers, the MKL for BLAS/LAPACK/FFTW/ScaLAPACK/BLACS, and Open MPI for parallelism.
 +
 +Download and unpack "gromacs-2022.5.tar.gz" and create a directory to hold our base build of gromacs-2022.5, naming it with the version identifier:  ''v2022.5''.  The source is then unpacked therein:
 +<code bash>
 +[user@login00.darwin plumed-2.8.3]$ cd $G_BASEDIR
 +[user@login00.darwin Gromacs]$ wget https://ftp.gromacs.org/gromacs/gromacs-2022.5.tar.gz
 +[user@login00.darwin Gromacs]$ tar -xvzf gromacs-2022.5.tar.gz
 +</code>
 +
 +===== Compilation and Installation: Plumed-patched Gromacs =====
 +
 +We use cmake for compilation and installation on Plumed-patched Gromacs "v2022.5" directory.
 +<code bash>
 +[user@login00.darwin Gromacs]$ G_INSTALL_PREFIX="${G_BASEDIR}/v2022.5"
 +[user@login00.darwin Gromacs]$ export G_INSTALL_PREFIX
 +[user@login00.darwin Gromacs]$ G_SRC_PREFIX="${G_BASEDIR}/gromacs-2022.5"
 +[user@login00.darwin Gromacs]$ mkdir -m $GP_BASEDIR_PRIVS "$G_INSTALL_PREFIX"
 +[user@login00.darwin Gromacs]$ G_BUILDDIR="${G_SRC_PREFIX}/build"
 +[user@login00.darwin Gromacs]$ mkdir -m $GP_BASEDIR_PRIVS "$G_BUILDDIR"
 +[user@login00.darwin Gromacs]$ cd "$G_SRC_PREFIX"
 +</code>
 +Hear, we patch Plumed on Gromacs. 
 +<code bash>
 +[user@login00.darwin gromacs-2022.5]$ plumed patch -p --static
 +</code>
 +
 +Select 4:
 +<code bash>
 +1) gromacs-2019.6     5) gromacs-4.5.7    9) qespresso-5.0.2
 +2) gromacs-2020.7     6) namd-2.12   10) qespresso-6.2
 +3) gromacs-2021.7     7) namd-2.13
 +4) gromacs-2022.5     8) namd-2.14
 +</code>
 +
 +<code bash>
 +[user@login00.darwin gromacs-2022.5]$ cd "$G_BUILDDIR"
 +</code> 
 +Our current working directory is now the //build root//.
 +Here, we create ''SWMGR-build.sh'' :
 +<file make SWMGR-build.sh.darwin>
 +#!/bin/bash -l
 +
 +vpkg_require cmake/default openmpi/4.1.5:intel-oneapi-2023
 +
 +PREFIX="${G_INSTALL_PREFIX}"
 +
 +CPU_GMX_SIMD=AVX2_256
 +
 +BASE_BUILD_FLAGS="
 +    -DCMAKE_BUILD_TYPE=Release
 +    -DGMX_SIMD=${CPU_GMX_SIMD}
 +    -DGMX_FFT_LIBRARY=mkl
 +    -DMKL_MPI=openmpi
 +    -DBUILD_SHARED_LIBS=OFF
 +    -DGMX_PREFER_STATIC_LIBS=ON
 +"
 +# BASE_BUILD_FLAGS="$BASE_BUILD_FLAGS -DGMX_GPU=SYCL"
 +#    -DMKL_INCLUDE_DIR=$MKLROOT/include
 +#    -DMKL_LIBRARIES=-mkl=sequential
 +#"
 +
 +build_variant()
 +{
 +    local DBL=$1 MPI=$2
 +    local build_dir=""
 +    local build_flags=""
 +
 +    if [ $DBL -ne 0 ]; then
 +        build_dir="double"
 + build_flags="${build_flags} -DGMX_DOUBLE=on"
 +    else
 +        build_dir="single"
 +        build_flags="${build_flags} -DGMX_DOUBLE=off"
 +    fi
 +    if [ $MPI -eq 0 ]; then
 +        build_dir="${build_dir}-thread"
 +        build_flags="${build_flags} -DGMX_MPI=off -DGMX_THREAD_MPI=on -DCMAKE_DISABLE_FIND_PACKAGE_MPI=on"
 +        CC=icx CXX=icpx FC=ifx
 +    else
 +        build_dir="${build_dir}-mpi"
 +        build_flags="${build_flags} -DGMX_MPI=on -DGMX_THREAD_MPI=off"
 + CC=mpicc CXX=mpicxx FC=mpifort
 +    fi
 +    [ -d "build-${build_dir}" ] && rm -rf "build-${build_dir}"
 +    mkdir "build-${build_dir}" ; pushd "build-${build_dir}"
 +    CC=$CC CXX=$CXX FC=$FC \
 +        cmake \
 +     -DCMAKE_INSTALL_PREFIX="${PREFIX}/${build_dir}" \
 +            $BASE_BUILD_FLAGS $build_flags \
 + ../..
 +    if [ $? -eq 0 ]; then
 +        make -j 20
 +        if [ $? -eq 0 ]; then
 +            make install
 +        fi
 +    fi
 +    popd
 +}
 +
 +##
 +## SYCL only supports single precision.
 +##
 +#             DOUBLE?    MPI?
 +#build_variant 0          0
 +#build_variant 1          0
 +#build_variant 0          1
 +build_variant 1          1
 +
 +</file>
 +
 +<file make SWMGR-build.sh.caviness>
 +#!/bin/bash -l
 +
 +vpkg_require cmake/default openmpi/4.1.4:intel-oneapi-2023
 +
 +PREFIX="${G_INSTALL_PREFIX}"
 +
 +CPU_GMX_SIMD=AVX2_256
 +
 +BASE_BUILD_FLAGS="
 +    -DCMAKE_BUILD_TYPE=Release
 +    -DGMX_SIMD=${CPU_GMX_SIMD}
 +    -DGMX_FFT_LIBRARY=mkl
 +    -DMKL_MPI=openmpi
 +    -DBUILD_SHARED_LIBS=OFF
 +    -DGMX_PREFER_STATIC_LIBS=ON
 +"
 +# BASE_BUILD_FLAGS="$BASE_BUILD_FLAGS -DGMX_GPU=SYCL"
 +#    -DMKL_INCLUDE_DIR=$MKLROOT/include
 +#    -DMKL_LIBRARIES=-mkl=sequential
 +#"
 +
 +build_variant()
 +{
 +    local DBL=$1 MPI=$2
 +    local build_dir=""
 +    local build_flags=""
 +
 +    if [ $DBL -ne 0 ]; then
 +        build_dir="double"
 + build_flags="${build_flags} -DGMX_DOUBLE=on"
 +    else
 +        build_dir="single"
 +        build_flags="${build_flags} -DGMX_DOUBLE=off"
 +    fi
 +    if [ $MPI -eq 0 ]; then
 +        build_dir="${build_dir}-thread"
 +        build_flags="${build_flags} -DGMX_MPI=off -DGMX_THREAD_MPI=on -DCMAKE_DISABLE_FIND_PACKAGE_MPI=on"
 +        CC=icx CXX=icpx FC=ifx
 +    else
 +        build_dir="${build_dir}-mpi"
 +        build_flags="${build_flags} -DGMX_MPI=on -DGMX_THREAD_MPI=off"
 + CC=mpicc CXX=mpicxx FC=mpifort
 +    fi
 +    [ -d "build-${build_dir}" ] && rm -rf "build-${build_dir}"
 +    mkdir "build-${build_dir}" ; pushd "build-${build_dir}"
 +    CC=$CC CXX=$CXX FC=$FC \
 +        cmake \
 +     -DCMAKE_INSTALL_PREFIX="${PREFIX}/${build_dir}" \
 +            $BASE_BUILD_FLAGS $build_flags \
 + ../..
 +    if [ $? -eq 0 ]; then
 +        make -j 20
 +        if [ $? -eq 0 ]; then
 +            make install
 +        fi
 +    fi
 +    popd
 +}
 +
 +##
 +## SYCL only supports single precision.
 +##
 +#             DOUBLE?    MPI?
 +#build_variant 0          0
 +#build_variant 1          0
 +#build_variant 0          1
 +build_variant 1          1
 +
 +</file>
 +
 +Executing the script compile and install plumed-patched gromacs.
 +<code bash>
 +[user@login00.darwin build]$ ./SWMGR-build.sh
 +</code> 
 +
 +===== VALET Package Definition =====
 +
 +With this version of Gromacs_Plumed built, the remaining step is to leverage VALET for setup of the runtime environment when you use the software.  VALET automatically recognizes the standard directory layout, so configuring versions/variants of ''Gromacs_Plumed'' is very straightforward.  First, note your installation path:
 +<code bash>
 +[user@login00.darwin build]$ vpkg_rollback all
 +[user@login00.darwin build]$ cd
 +[user@login00.darwin ~]$ echo $GP_BASEDIR
 +/home/user/sw/Gromacs_Plumed
 +</code>
 +Since this build was done in the user's home directory, they were personal copies of the software and should use a //VALET package definition file// stored in ''~/.valet''
 +<code bash>
 +[user@login00.darwin ~]$ VALET_PKG_DIR=~/.valet ; VALET_PKG_DIR_MODE=0700
 +</code>
 +versus an installation made for an entire workgroup, which would store the VALET package definition files in ''$WORKDIR/sw/valet''
 +<code bash>
 +[user@login00.darwin ~]$ VALET_PKG_DIR="$WORKDIR/sw/valet" ; VALET_PKG_DIR_MODE=2770
 +</code>
 +Whichever scheme is in-use, ensure the directory exists:
 +<code bash>
 +[user@login00.darwin ~]$ mkdir -p --mode=$VALET_PKG_DIR_MODE "$VALET_PKG_DIR"
 +</code>
 +
 +VALET allows package definitions in a variety of formats (XML, JSON, YAML) but YAML tends to be the simplest format so we will use it here.
 +
 +==== Package section ====
 +
 +The //package section// of the definition file includes items that apply to all versions/variants of the software:
 +<code yaml>
 +Gromacs_Plumed:
 +    prefix: /home/user/sw/Gromacs_Plumed/Gromacs
 +    description: Gromacs_Plumed
 +    url: "https://www.gromacs.org/"
 +</code>
 +The //package identifier// is the top-level key in the document — ''Gromacs_Plumed'' — and the value of ''$GP_BASEDIR'' is the value of the ''prefix'' key in this section.  The URL and description are information taken from the official Gromacs web site.
 +
 +==== Versions ====
 +
 +The ''versions'' key is used to provide a list of the versions/variants of the software:
 +<code yaml>
 +gromacs-plumed:
 +    prefix: /home/user/sw/Gromacs_Plumed/Gromacs
 +    description: Gromacs_Plumed
 +    url: "https://www.gromacs.org/"
 +    
 +    default-version: "v2022.5"
 +    
 +    versions:
 +        "v2022.5":
 +            prefix: "v2022.5/double-mpi"
 +            description:  Gromacs patched with Plumed v2.8.3
 +            dependencies:
 +                 - openmpi/4.1.5:intel-oneapi-2023
 +               
 +</code>
 +
 +<WRAP center round tip 80%>
 +The version identifier ''v7.3'' is inferred to be the path prefix to the version in question here.  The package's prefix (''/home/user/sw/Gromacs_Plumed'') with the version identifier appended (''/home/user/sw/Gromacs_Plumed/Gromacs/v2022.5'') is implicit.
 +
 +The implicit behavior is overridden by providing a ''prefix'' key in the version definition:  a relative path is appended to the package's prefix, an absolute path is used as-is.
 +</WRAP>
 +
 +It is a good idea to specify which version definition should act as the default.  This yields the following package definition file
 +
 +<file yaml gromacs-plumed.vpkg_yaml.darwin>
 +gromacs-plumed:
 +    prefix: /home/user/sw/Gromacs_Plumed/Gromacs
 +    description: Gromacs_Plumed
 +    url: "https://www.gromacs.org/"
 +    
 +    default-version: "v2022.5"
 +    
 +    versions:
 +        "v2022.5":
 +            prefix: "v2022.5/double-mpi"
 +            description:  Gromacs patched with Plumed v2.8.3
 +            dependencies:
 +                 - openmpi/4.1.5:intel-oneapi-2023
 +</file>
 +
 +<file yaml gromacs-plumed.vpkg_yaml.caviness>
 +gromacs-plumed:
 +    prefix: /home/user/sw/Gromacs_Plumed/Gromacs
 +    description: Gromacs_Plumed
 +    url: "https://www.gromacs.org/"
 +    
 +    default-version: "v2022.5"
 +        
 +    versions:
 +        "v2022.5":
 +            prefix: "v2022.5/double-mpi"
 +            description:  Gromacs patched with Plumed v2.8.3
 +            dependencies:
 +                 - openmpi/4.1.4:intel-oneapi-2023
 +</file>
 +
 +saved at ''$VALET_PKG_DIR/gromacs-plumed.vpkg_yaml''.
 +
 +==== Checking the definition file ====
 +
 +The package definition file can be checked for proper syntax using the VALET command ''vpkg_check'':
 +<code bash>
 +[user@login00.darwin ~]$ vpkg_check "$VALET_PKG_DIR/gromacs-plumed.vpkg_yaml"
 +/home/user/.valet/gromacs-plumed.vpkg_yaml is OK
 +[gromacs-plumed] {
 +  contexts: all
 +  actions: {
 +    GROMACS_PLUMED_PREFIX=${VALET_PATH_PREFIX} (contexts: development)
 +  }
 +  https://www.gromacs.org/
 +  Gromacs_Plumed
 +  prefix: /home/user/sw/Gromacs_Plumed/Gromacs
 +  source file: /home/user/.valet/gromacs-plumed.vpkg_yaml
 +  default version: gromacs-plumed/v2022.5
 +  versions: {
 +    [gromacs-plumed/v2022.5] {
 +      contexts: all
 +      dependencies: {
 +        openmpi/4.1.5:intel-oneapi-2023
 +      }
 +      Gromacs patched with Plumed v2.8.3
 +      prefix: /home/user/sw/Gromacs_Plumed/Gromacs/v2022.5/double-mpi
 +      standard paths: {
 +        bin: /home/user/sw/Gromacs_Plumed/Gromacs/v2022.5/double-mpi/bin
 +        lib: /home/user/sw/Gromacs_Plumed/Gromacs/v2022.5/double-mpi/lib64
 +        man: /home/user/sw/Gromacs_Plumed/Gromacs/v2022.5/double-mpi/share/man
 +        include: /home/user/sw/Gromacs_Plumed/Gromacs/v2022.5/double-mpi/include
 +      }
 +    }
 +  }
 +}
 +
 +</code>
 +The file had no errors in its YAML syntax.  Notice also that the standard path (''bin'') is found and noted by VALET!
 +
 +==== Runtime environment ====
 +
 +To load **Gromacs_Plumed** v2022.5 into the runtime environment, the ''vpkg_require'' command is used:
 +<code bash>
 +[user@login00.darwin ~]$ vpkg_require gromacs-plumed/v2022.5
 +Adding dependency `binutils/2.35.1` to your environment
 +Adding dependency `gcc/12.2.0` to your environment
 +Adding dependency `intel-oneapi/2023.0.0.25537` to your environment
 +Adding dependency `ucx/1.13.1` to your environment
 +Adding dependency `openmpi/4.1.5:intel-oneapi-2023` to your environment
 +Adding package `gromacs-plumed/v2022.5` to your environment
 +[user@login00.darwin ~]$ which gmx_mpi_d
 +~/sw/Gromacs_Plumed/Gromacs/v2022.5/double-mpi/bin/gmx_mpi_d
 +</code>
 +
 +===== Scaling =====
 +
 +This is the wall time for MD run of 41057 atom, and nsteps=10000 with metadynamics by combining Gromacs and Plumed. The wall time decreases with an increased number of cores.
 +
 +{{:technical:recipes:scale_gromacs_plumed.jpg?400|}}