Korištenje

Verzija	Modul
4.1.5	siesta/4.1.5

% module load siesta/4.1.5
% siesta
Siesta Version  : 4.1.5
Architecture    : unknown
Compiler version: ifort (IFORT) 19.0.0.117 20180804
Compiler flags  : mpifort -O2 -fPIC -fp-model source -qopenmp -I/apps/netcdf-fortran/4.5.3/include/
PP flags        : -DFC_HAVE_ABORT -DMPI -qopenmp -DNCDF -DNCDF_4 -DNCDF_PARALLEL -DSIESTA__METIIS -DSIESTA__ELPA -I/apps/elpa/2020.11.001/avx2/include/elpa-2020.11.001/modules -DSIESTA__DIAG_2STAGE
Libraries       : libsiestaLAPACK.a libsiestaBLAS.a libfdict.a libncdf.a  -qopenmp -mkl=paralleel -i8 -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_sequential -lmkl_blas95_lp64 -L/apps/netcdf-c/4.7.4/lib/ -L/apps/netcdf-fortran/4.5.3/lib/ -L/apps/hdf5/1.12.0/lib/ -lnetcdff -lnetcdf  -lhdf5_fortran -lhdf5 -lz -L/apps/metis/5.1.0/lib/ -lmetis -L/apps/elpa/2020.11.001/avx2/lib/ -lelpa
PARALLEL version
NetCDF-4 support
NetCDF-4 MPI-IO support
METIS ordering support
...

Primjer skripte za opis posla:

#$ -N siesta-parallel
#$ -q p28.q
#$ -pe *mpi 16
#$ -cwd

module load siesta/4.1.5

export OMP_NUM_THREADS=4

mpirun --bind-to socket -hostfile $TMPDIR/machines -np 4 siesta < fe.fdf > fe.out

Instalacija

Programski paket SIESTA je pripremljen s kompajlerom Intel 2019, Intel MKL Scalapack i MPI implementacijom OpenMPI 3.0.0. Instalacija je obavljena pripremom pomoćnih biblioteka na koje se paket oslanja, dopremanjem izvornog koda, učitavanjem potrebnih modula za kompajler i MPI paralelizaciju te kompajliranjem samog paketa.

Priprema i instalacija se vršila prema uputstvima (uz male preinake specifične za Isabellu): https://gitlab.com/siesta-project/siesta/-/releases/v4.1.5/downloads/siesta.pdf

Izvorni kod se može pronaći na sjedištu: https://gitlab.com/siesta-project/siesta

Priprema

HD5

https://www.hdfgroup.org/downloads/hdf5/source-code/

module load mpi/openmpi3-intel-x86_64
CC=mpicc CXX=mpicxx FC=mpifort F90=mpif90 ./configure --prefix=/apps/hdf5/1.12.0 --enable-parallel --enable-fortran
make -j8
make install

NetCDF

https://parallel-netcdf.github.io/

module load mpi/openmpi3-intel-x86_64
./configure --prefix=/apps/pnetcdf/1.12.2 --enable-shared
make -j8
make install

https://www.unidata.ucar.edu/downloads/netcdf/

module load mpi/openmpi3-intel-x86_64

LD_LIBRARY_PATH="/apps/hdf5/1.12.0/lib/:/apps/pnetcdf/1.12.2/lib:$LD_LIBRARY_PATH"
LIBS="-lhdf5_hl -lhdf5" LDFLAGS="-L/apps/hdf5/1.12.0/lib
-L/apps/pnetcdf/1.12.2/lib" CC=mpicc CPPFLAGS="-I/apps/hdf5/1.12.0/include
-I/apps/pnetcdf/1.12.2/include" CXX=mpicxx ./configure
--prefix=/apps/netcdf-c/4.7.4/ --enable-pnetcdf

make -j8
make install

module load mpi/openmpi3-intel-x86_64

LD_LIBRARY_PATH="/apps/netcdf-c/4.7.4/lib/:$LD_LIBRARY_PATH" CC=mpicc
CXX=mpicxx FC=mpifort F90=ifort CPPFLAGS="-I/apps/netcdf-c/4.7.4/include/"
LDFLAGS="-L/apps/netcdf-c/4.7.4/lib/" ./configure
--prefix=/apps/netcdf-fortran/4.5.3/

make -j8
make install

METIS

http://glaros.dtc.umn.edu/gkhome/metis/metis/download

make config prefix=/apps/metis/5.1.0/ cc=icc
make install

ELPA

https://elpa.mpcdf.mpg.de/elpa-tar-archive

module load mpi/openmpi3-intel-x86_64

LIBS="-qopenmp -mkl=parallel -i8 -lmkl_core -lmkl_scalapack_lp64
-lmkl_blacs_openmpi_lp64 -lmkl_sequential -lmkl_blas95_lp64" 
./configure --prefix=/apps/elpa/2020.11.001/avx2

make -j8
make install

MUMPS

http://mumps.enseeiht.fr/index.php?page=dwnld#form

module load mpi/openmpi3-intel-x86_64

cd MUMPS_5.3.5
cp Make.inc/Makefile.INTEL.PAR Makefile.inc
make all
cp -R lib/ include/ /apps/mumps/5.3.5/

Kompajliranje i instalacija SIESTA

wget -c https://gitlab.com/siesta-project/siesta/-/package_files/6659062/download -O siesta-4.1.5.tar.gz
tar -v -xzf siesta-4.1.5.tar.gz

cd siesta-4.1.5/Obj/
sh ../Src/obj_setup.sh
cp intel.make arch.make
module load mpi/openmpi3-intel-x86_64

Datoteku arch.make je potrebno editirati i dodati postavke za MPI paralelizaciju, Intel kompajler te povezivanje na pomoćne biblioteke:

# parallel compile @ Isabella
CC = mpicc
FPP = $(FC) -E -P
FC = mpifort
FC_SERIAL = ifort

FFLAGS = -O2 -fPIC -fp-model source -qopenmp

MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE=.
FPPFLAGS += -DMPI -qopenmp
LIBS += -qopenmp -mkl=parallel -i8 -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_sequential -lmkl_blas95_lp64
LIBS += -L/apps/netcdf-c/4.7.4/lib/ -L/apps/netcdf-fortran/4.5.3/lib/ -L/apps/hdf5/1.12.0/lib/ -lnetcdff -lnetcdf  -lhdf5_fortran -lhdf5 -lz

# fdict
COMP_LIBS += libfdict.a
# ncdf
COMP_LIBS += libncdf.a
FPPFLAGS += -DNCDF -DNCDF_4 -DNCDF_PARALLEL
FFLAGS += -I/apps/netcdf-fortran/4.5.3/include/
# metis
LIBS += -L/apps/metis/5.1.0/lib/ -lmetis
FPPFLAGS += -DSIESTA__METIS
# elpa
LIBS += -L/apps/elpa/2020.11.001/avx2/lib/ -lelpa
FPPFLAGS += -DSIESTA__ELPA -I/apps/elpa/2020.11.001/avx2/include/elpa-2020.11.001/modules
# mumps
LIBS += -L/apps/mumps/5.3.5/lib/ -lzmumps -lmumps_common -lpord                                                                         
FPPFLAGS += -DSIESTA__MUMPS
FFLAGS += -I/apps/mumps/5.3.5/include/

make -j8
cp siesta /apps/siesta/4.1.5/avx2

cd siesta-4.1.5/Util
find . -name Makefile -exec sed -i 's/$(LDFLAGS)/$(LDFLAGS) -qopenmp/g' {} \;
./build_all
find . -type f -executable -exec cp {} /apps/siesta/4.1.5/avx2 \;
ls /apps/siesta/4.1.5/avx2/
h2o/           eig2bxsf*     fractional*       ionplot.sh*     pblas_prb*  simple*                   surf.py*
2dplot.py*     eigfat2plot*  get_chem_labels*  macroave*       pi3*        simple_mpi_parallel*      tbtrans*
Eig2DOS*       f2fmaster*    gnubands*         md2axsf*        plstm*      simple_mpi_serial*        ts2ts*
blacs_prb*     f2fslave*     grid1d*           mixps*          plsts*      simple_pipes_parallel*    tselecs.sh*
ccViz*         fat*          grid2d*           mprop*          protoNEB*   simple_pipes_serial*      vib2xsf*
clean_all.sh*  fcbuild*      hs2hsx*           optical*        rho2xsf*    simple_sockets_parallel*  vibra*
denchar*       fdf2grimme*   hsx2hs*           optical_input*  sies2arc*   simple_sockets_serial*    xv2xsf*
driver*        fmpdos*       int_explorer*     para*           siesta*     spin_texture*